2024-09-13T06:48:49.348 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T06:48:49.365 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T06:48:49.465 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900895 branch: reef description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} email: ceph-qa@ceph.com first_in_suite: false job_id: '7900895' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: reef ceph: conf: client: rbd cache: false global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 248 sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 sleep_before_teardown: 0 subset: 116/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 targets: smithi118.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_22:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T06:48:49.465 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa; will attempt to use it 2024-09-13T06:48:49.466 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa/tasks 2024-09-13T06:48:49.466 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T06:48:49.468 INFO:teuthology.task.internal:Checking packages... 2024-09-13T06:48:49.487 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '48a7de7e6e1701a88fd571c7e34908ac4b8ed412' 2024-09-13T06:48:49.487 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T06:48:49.487 INFO:teuthology.packaging:ref: None 2024-09-13T06:48:49.487 INFO:teuthology.packaging:tag: None 2024-09-13T06:48:49.488 INFO:teuthology.packaging:branch: reef 2024-09-13T06:48:49.488 INFO:teuthology.packaging:sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T06:48:49.488 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=reef 2024-09-13T06:48:49.716 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-843-g2c01458a-1focal 2024-09-13T06:48:49.719 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T06:48:49.725 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T06:48:49.725 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T06:48:49.739 INFO:teuthology.task.internal:Saving configuration 2024-09-13T06:48:49.753 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T06:48:49.760 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T06:48:49.794 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900895', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 06:41:58.580129', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T06:48:49.827 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900895', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 06:41:58.578665', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T06:48:49.827 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T06:48:49.833 INFO:teuthology.task.internal:roles: ubuntu@smithi118.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-13T06:48:49.833 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-13T06:48:49.833 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T06:48:49.913 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9ef0675990>, signals=[15]) 2024-09-13T06:48:49.913 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T06:48:49.919 INFO:teuthology.task.internal:Opening connections... 2024-09-13T06:48:49.919 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi118.front.sepia.ceph.com 2024-09-13T06:48:49.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T06:48:50.029 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-09-13T06:48:50.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T06:48:50.125 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T06:48:50.133 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2024-09-13T06:48:50.140 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2024-09-13T06:48:50.140 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2024-09-13T06:48:50.189 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="Ubuntu" 2024-09-13T06:48:50.189 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T06:48:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:ID=ubuntu 2024-09-13T06:48:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE=debian 2024-09-13T06:48:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T06:48:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="20.04" 2024-09-13T06:48:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T06:48:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T06:48:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T06:48:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T06:48:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_CODENAME=focal 2024-09-13T06:48:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:UBUNTU_CODENAME=focal 2024-09-13T06:48:50.191 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2024-09-13T06:48:50.217 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-09-13T06:48:50.223 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-09-13T06:48:50.224 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-09-13T06:48:50.229 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Ubuntu" 2024-09-13T06:48:50.229 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T06:48:50.230 INFO:teuthology.orchestra.run.smithi164.stdout:ID=ubuntu 2024-09-13T06:48:50.230 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE=debian 2024-09-13T06:48:50.230 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T06:48:50.230 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="20.04" 2024-09-13T06:48:50.230 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T06:48:50.230 INFO:teuthology.orchestra.run.smithi164.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T06:48:50.230 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T06:48:50.230 INFO:teuthology.orchestra.run.smithi164.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T06:48:50.230 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_CODENAME=focal 2024-09-13T06:48:50.230 INFO:teuthology.orchestra.run.smithi164.stdout:UBUNTU_CODENAME=focal 2024-09-13T06:48:50.231 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-09-13T06:48:50.278 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T06:48:50.288 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T06:48:50.294 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T06:48:50.294 DEBUG:teuthology.orchestra.run.smithi118:> test '!' -e /home/ubuntu/cephtest 2024-09-13T06:48:50.297 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-09-13T06:48:50.302 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T06:48:50.308 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T06:48:50.308 DEBUG:teuthology.orchestra.run.smithi118:> test -z $(ls -A /var/lib/ceph) 2024-09-13T06:48:50.346 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-09-13T06:48:50.359 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T06:48:50.431 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T06:48:50.448 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T06:48:50.448 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T06:48:50.449 DEBUG:teuthology.orchestra.run.smithi118:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T06:48:50.449 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T06:48:50.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T06:48:50.455 DEBUG:teuthology.orchestra.run.smithi118:> uname -r 2024-09-13T06:48:50.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T06:48:50.456 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-13T06:48:50.462 INFO:teuthology.orchestra.run.smithi164.stdout:5.4.0-124-generic 2024-09-13T06:48:50.463 INFO:teuthology.task.kernel:Running kernel on smithi164: 5.4.0-124-generic 2024-09-13T06:48:50.463 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get clean 2024-09-13T06:48:50.501 INFO:teuthology.orchestra.run.smithi118.stdout:5.4.0-124-generic 2024-09-13T06:48:50.502 INFO:teuthology.task.kernel:Running kernel on smithi118: 5.4.0-124-generic 2024-09-13T06:48:50.502 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get clean 2024-09-13T06:48:50.619 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get update 2024-09-13T06:48:50.779 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get update 2024-09-13T06:48:50.918 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T06:48:50.918 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-13T06:48:50.970 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-13T06:48:51.130 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T06:48:51.130 INFO:teuthology.orchestra.run.smithi118.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-13T06:48:51.130 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-13T06:48:51.190 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-13T06:48:51.326 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-13T06:48:51.397 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-13T06:48:51.518 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-13T06:48:51.540 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-13T06:48:51.698 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-13T06:48:51.700 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-13T06:48:51.711 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-13T06:48:51.715 INFO:teuthology.orchestra.run.smithi118.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-13T06:48:51.715 INFO:teuthology.orchestra.run.smithi118.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-13T06:48:51.715 INFO:teuthology.orchestra.run.smithi118.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-13T06:48:51.737 INFO:teuthology.orchestra.run.smithi118.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-13T06:48:51.740 INFO:teuthology.orchestra.run.smithi118.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T06:48:51.740 INFO:teuthology.orchestra.run.smithi118.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-13T06:48:51.745 INFO:teuthology.orchestra.run.smithi118.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-13T06:48:51.751 INFO:teuthology.orchestra.run.smithi118.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-13T06:48:51.753 INFO:teuthology.orchestra.run.smithi118.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-13T06:48:51.753 INFO:teuthology.orchestra.run.smithi118.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-13T06:48:51.754 INFO:teuthology.orchestra.run.smithi118.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-13T06:48:51.759 INFO:teuthology.orchestra.run.smithi118.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-13T06:48:51.762 INFO:teuthology.orchestra.run.smithi118.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-13T06:48:51.767 INFO:teuthology.orchestra.run.smithi118.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-13T06:48:51.807 INFO:teuthology.orchestra.run.smithi118.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-13T06:48:51.811 INFO:teuthology.orchestra.run.smithi118.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-13T06:48:51.811 INFO:teuthology.orchestra.run.smithi118.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-13T06:48:51.811 INFO:teuthology.orchestra.run.smithi118.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-13T06:48:51.832 INFO:teuthology.orchestra.run.smithi118.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-13T06:48:51.835 INFO:teuthology.orchestra.run.smithi118.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T06:48:51.835 INFO:teuthology.orchestra.run.smithi118.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-13T06:48:51.840 INFO:teuthology.orchestra.run.smithi118.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-13T06:48:51.848 INFO:teuthology.orchestra.run.smithi118.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-13T06:48:51.850 INFO:teuthology.orchestra.run.smithi118.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-13T06:48:51.850 INFO:teuthology.orchestra.run.smithi118.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-13T06:48:51.850 INFO:teuthology.orchestra.run.smithi118.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-13T06:48:51.851 INFO:teuthology.orchestra.run.smithi118.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-13T06:48:51.859 INFO:teuthology.orchestra.run.smithi118.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-13T06:48:51.859 INFO:teuthology.orchestra.run.smithi118.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-13T06:48:51.859 INFO:teuthology.orchestra.run.smithi118.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-13T06:48:51.860 INFO:teuthology.orchestra.run.smithi118.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-13T06:48:51.877 INFO:teuthology.orchestra.run.smithi118.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-13T06:48:51.878 INFO:teuthology.orchestra.run.smithi118.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-13T06:48:51.878 INFO:teuthology.orchestra.run.smithi118.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-13T06:48:51.900 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-13T06:48:51.914 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-13T06:48:51.922 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-13T06:48:51.922 INFO:teuthology.orchestra.run.smithi164.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-13T06:48:51.923 INFO:teuthology.orchestra.run.smithi164.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-13T06:48:52.012 INFO:teuthology.orchestra.run.smithi164.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-13T06:48:52.012 INFO:teuthology.orchestra.run.smithi164.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-13T06:48:52.019 INFO:teuthology.orchestra.run.smithi164.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T06:48:52.019 INFO:teuthology.orchestra.run.smithi164.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-13T06:48:52.029 INFO:teuthology.orchestra.run.smithi164.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-13T06:48:52.038 INFO:teuthology.orchestra.run.smithi164.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-13T06:48:52.039 INFO:teuthology.orchestra.run.smithi164.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-13T06:48:52.039 INFO:teuthology.orchestra.run.smithi164.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-13T06:48:52.070 INFO:teuthology.orchestra.run.smithi164.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-13T06:48:52.081 INFO:teuthology.orchestra.run.smithi164.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-13T06:48:52.083 INFO:teuthology.orchestra.run.smithi164.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-13T06:48:52.083 INFO:teuthology.orchestra.run.smithi164.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-13T06:48:52.083 INFO:teuthology.orchestra.run.smithi164.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-13T06:48:52.086 INFO:teuthology.orchestra.run.smithi164.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-13T06:48:52.158 INFO:teuthology.orchestra.run.smithi164.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-13T06:48:52.177 INFO:teuthology.orchestra.run.smithi164.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-13T06:48:52.184 INFO:teuthology.orchestra.run.smithi164.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T06:48:52.184 INFO:teuthology.orchestra.run.smithi164.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-13T06:48:52.204 INFO:teuthology.orchestra.run.smithi164.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-13T06:48:52.218 INFO:teuthology.orchestra.run.smithi164.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-13T06:48:52.222 INFO:teuthology.orchestra.run.smithi164.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-13T06:48:52.223 INFO:teuthology.orchestra.run.smithi164.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-13T06:48:52.223 INFO:teuthology.orchestra.run.smithi164.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-13T06:48:52.224 INFO:teuthology.orchestra.run.smithi164.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-13T06:48:52.275 INFO:teuthology.orchestra.run.smithi164.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-13T06:48:52.275 INFO:teuthology.orchestra.run.smithi164.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-13T06:48:52.276 INFO:teuthology.orchestra.run.smithi164.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-13T06:48:52.356 INFO:teuthology.orchestra.run.smithi164.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-13T06:48:52.357 INFO:teuthology.orchestra.run.smithi164.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-13T06:48:52.357 INFO:teuthology.orchestra.run.smithi164.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-13T06:48:52.358 INFO:teuthology.orchestra.run.smithi164.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-13T06:48:56.184 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 21.8 MB in 4s (6,143 kB/s) 2024-09-13T06:48:56.233 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 21.8 MB in 3s (6,704 kB/s) 2024-09-13T06:48:57.795 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-09-13T06:48:57.800 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-09-13T06:48:57.823 DEBUG:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T06:48:57.827 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T06:48:57.894 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-09-13T06:48:57.897 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-09-13T06:48:58.111 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2024-09-13T06:48:58.112 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2024-09-13T06:48:58.125 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2024-09-13T06:48:58.126 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2024-09-13T06:48:58.310 INFO:teuthology.orchestra.run.smithi118.stdout:The following additional packages will be installed: 2024-09-13T06:48:58.311 INFO:teuthology.orchestra.run.smithi118.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T06:48:58.311 INFO:teuthology.orchestra.run.smithi118.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-13T06:48:58.311 INFO:teuthology.orchestra.run.smithi118.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-13T06:48:58.312 INFO:teuthology.orchestra.run.smithi118.stdout:Suggested packages: 2024-09-13T06:48:58.312 INFO:teuthology.orchestra.run.smithi118.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-13T06:48:58.312 INFO:teuthology.orchestra.run.smithi118.stdout:Recommended packages: 2024-09-13T06:48:58.312 INFO:teuthology.orchestra.run.smithi118.stdout: thermald 2024-09-13T06:48:58.339 INFO:teuthology.orchestra.run.smithi164.stdout:The following additional packages will be installed: 2024-09-13T06:48:58.339 INFO:teuthology.orchestra.run.smithi164.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T06:48:58.339 INFO:teuthology.orchestra.run.smithi164.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-13T06:48:58.339 INFO:teuthology.orchestra.run.smithi164.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-13T06:48:58.340 INFO:teuthology.orchestra.run.smithi164.stdout:Suggested packages: 2024-09-13T06:48:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-13T06:48:58.341 INFO:teuthology.orchestra.run.smithi164.stdout:Recommended packages: 2024-09-13T06:48:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: thermald 2024-09-13T06:48:58.393 INFO:teuthology.orchestra.run.smithi118.stdout:The following NEW packages will be installed: 2024-09-13T06:48:58.393 INFO:teuthology.orchestra.run.smithi118.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T06:48:58.393 INFO:teuthology.orchestra.run.smithi118.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-13T06:48:58.393 INFO:teuthology.orchestra.run.smithi118.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-13T06:48:58.395 INFO:teuthology.orchestra.run.smithi118.stdout:The following packages will be upgraded: 2024-09-13T06:48:58.395 INFO:teuthology.orchestra.run.smithi118.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T06:48:58.402 INFO:teuthology.orchestra.run.smithi164.stdout:The following NEW packages will be installed: 2024-09-13T06:48:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T06:48:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-13T06:48:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-13T06:48:58.404 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages will be upgraded: 2024-09-13T06:48:58.405 INFO:teuthology.orchestra.run.smithi164.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T06:48:58.461 INFO:teuthology.orchestra.run.smithi164.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T06:48:58.461 INFO:teuthology.orchestra.run.smithi164.stdout:Need to get 77.2 MB of archives. 2024-09-13T06:48:58.461 INFO:teuthology.orchestra.run.smithi164.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-13T06:48:58.461 INFO:teuthology.orchestra.run.smithi164.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-13T06:48:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T06:48:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:Need to get 77.2 MB of archives. 2024-09-13T06:48:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-13T06:48:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-13T06:48:58.803 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-13T06:48:58.898 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-13T06:48:59.399 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-13T06:48:59.399 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-13T06:48:59.399 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-13T06:48:59.493 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-13T06:48:59.504 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-13T06:48:59.739 INFO:teuthology.orchestra.run.smithi118.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-13T06:49:00.013 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-13T06:49:00.228 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 77.2 MB in 1s (70.6 MB/s) 2024-09-13T06:49:00.410 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-13T06:49:01.491 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-13T06:49:01.491 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-13T06:49:01.491 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-13T06:49:01.847 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-13T06:49:01.866 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-13T06:49:02.181 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-13T06:49:02.185 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T06:49:02.315 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:02.570 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 77.2 MB in 3s (22.3 MB/s) 2024-09-13T06:49:02.748 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-13T06:49:04.424 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-13T06:49:04.437 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T06:49:04.488 INFO:teuthology.orchestra.run.smithi118.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-13T06:49:04.493 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T06:49:04.544 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:04.596 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:05.660 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-13T06:49:05.672 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T06:49:05.709 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:06.769 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-13T06:49:06.784 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T06:49:06.929 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:08.034 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-13T06:49:08.048 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T06:49:08.092 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:10.481 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T06:49:10.589 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T06:49:10.949 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T06:49:11.100 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T06:49:11.445 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-13T06:49:11.458 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-13T06:49:11.503 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T06:49:13.462 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T06:49:13.586 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T06:49:13.937 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T06:49:14.065 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T06:49:14.426 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-13T06:49:14.441 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-13T06:49:14.476 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T06:49:16.928 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-13T06:49:16.948 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T06:49:16.986 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:19.844 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-13T06:49:19.865 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T06:49:19.902 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:20.334 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T06:49:20.461 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T06:49:20.827 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:22.041 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:23.115 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T06:49:23.221 INFO:teuthology.orchestra.run.smithi164.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-13T06:49:23.221 INFO:teuthology.orchestra.run.smithi164.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-13T06:49:23.243 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T06:49:23.584 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:23.668 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:24.781 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:25.762 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T06:49:25.888 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-13T06:49:25.971 INFO:teuthology.orchestra.run.smithi118.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-13T06:49:25.971 INFO:teuthology.orchestra.run.smithi118.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-13T06:49:26.048 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:26.108 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:26.164 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-13T06:49:26.290 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-13T06:49:26.400 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:26.479 INFO:teuthology.orchestra.run.smithi164.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T06:49:26.479 INFO:teuthology.orchestra.run.smithi164.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-13T06:49:27.285 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T06:49:27.410 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-13T06:49:27.545 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:27.687 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-13T06:49:27.822 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-13T06:49:28.719 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T06:49:29.877 INFO:teuthology.orchestra.run.smithi118.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T06:49:29.877 INFO:teuthology.orchestra.run.smithi118.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-13T06:49:48.609 INFO:teuthology.orchestra.run.smithi164.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T06:49:48.609 INFO:teuthology.orchestra.run.smithi164.stdout:Sourcing file `/etc/default/grub' 2024-09-13T06:49:48.624 INFO:teuthology.orchestra.run.smithi164.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T06:49:48.639 INFO:teuthology.orchestra.run.smithi164.stdout:Generating grub configuration file ... 2024-09-13T06:49:48.980 INFO:teuthology.orchestra.run.smithi164.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T06:49:48.998 INFO:teuthology.orchestra.run.smithi164.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T06:49:49.487 INFO:teuthology.orchestra.run.smithi164.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T06:49:49.499 INFO:teuthology.orchestra.run.smithi164.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T06:49:51.850 INFO:teuthology.orchestra.run.smithi118.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T06:49:51.850 INFO:teuthology.orchestra.run.smithi118.stdout:Sourcing file `/etc/default/grub' 2024-09-13T06:49:51.856 INFO:teuthology.orchestra.run.smithi118.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T06:49:51.876 INFO:teuthology.orchestra.run.smithi118.stdout:Generating grub configuration file ... 2024-09-13T06:49:52.184 INFO:teuthology.orchestra.run.smithi118.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T06:49:52.203 INFO:teuthology.orchestra.run.smithi118.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T06:49:52.212 INFO:teuthology.orchestra.run.smithi164.stdout:done 2024-09-13T06:49:52.578 INFO:teuthology.orchestra.run.smithi118.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T06:49:52.590 INFO:teuthology.orchestra.run.smithi118.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T06:49:52.676 DEBUG:teuthology.orchestra.run.smithi164:> dpkg -s linux-image-generic 2024-09-13T06:49:52.719 INFO:teuthology.orchestra.run.smithi164.stdout:Package: linux-image-generic 2024-09-13T06:49:52.719 INFO:teuthology.orchestra.run.smithi164.stdout:Status: install ok installed 2024-09-13T06:49:52.719 INFO:teuthology.orchestra.run.smithi164.stdout:Priority: optional 2024-09-13T06:49:52.720 INFO:teuthology.orchestra.run.smithi164.stdout:Section: kernel 2024-09-13T06:49:52.720 INFO:teuthology.orchestra.run.smithi164.stdout:Installed-Size: 20 2024-09-13T06:49:52.720 INFO:teuthology.orchestra.run.smithi164.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T06:49:52.720 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: amd64 2024-09-13T06:49:52.720 INFO:teuthology.orchestra.run.smithi164.stdout:Source: linux-meta 2024-09-13T06:49:52.720 INFO:teuthology.orchestra.run.smithi164.stdout:Version: 5.4.0.195.193 2024-09-13T06:49:52.720 INFO:teuthology.orchestra.run.smithi164.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T06:49:52.720 INFO:teuthology.orchestra.run.smithi164.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T06:49:52.720 INFO:teuthology.orchestra.run.smithi164.stdout:Recommends: thermald 2024-09-13T06:49:52.720 INFO:teuthology.orchestra.run.smithi164.stdout:Description: Generic Linux kernel image 2024-09-13T06:49:52.720 INFO:teuthology.orchestra.run.smithi164.stdout: This package will always depend on the latest generic kernel image 2024-09-13T06:49:52.720 INFO:teuthology.orchestra.run.smithi164.stdout: available. 2024-09-13T06:49:52.721 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-13T06:49:52.721 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T06:49:52.721 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T06:49:52.721 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T06:49:52.721 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T06:49:52.721 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get clean 2024-09-13T06:49:52.806 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get update 2024-09-13T06:49:53.067 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T06:49:53.067 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T06:49:53.137 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T06:49:53.226 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T06:49:54.936 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-09-13T06:49:54.963 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T06:49:55.034 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-09-13T06:49:55.248 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2024-09-13T06:49:55.248 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2024-09-13T06:49:55.410 INFO:teuthology.orchestra.run.smithi118.stdout:done 2024-09-13T06:49:55.476 INFO:teuthology.orchestra.run.smithi164.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-13T06:49:55.477 INFO:teuthology.orchestra.run.smithi164.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T06:49:55.478 DEBUG:teuthology.orchestra.run.smithi164:> dpkg -s linux-image-generic 2024-09-13T06:49:55.500 INFO:teuthology.orchestra.run.smithi164.stdout:Package: linux-image-generic 2024-09-13T06:49:55.500 INFO:teuthology.orchestra.run.smithi164.stdout:Status: install ok installed 2024-09-13T06:49:55.500 INFO:teuthology.orchestra.run.smithi164.stdout:Priority: optional 2024-09-13T06:49:55.501 INFO:teuthology.orchestra.run.smithi164.stdout:Section: kernel 2024-09-13T06:49:55.501 INFO:teuthology.orchestra.run.smithi164.stdout:Installed-Size: 20 2024-09-13T06:49:55.501 INFO:teuthology.orchestra.run.smithi164.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T06:49:55.501 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: amd64 2024-09-13T06:49:55.501 INFO:teuthology.orchestra.run.smithi164.stdout:Source: linux-meta 2024-09-13T06:49:55.501 INFO:teuthology.orchestra.run.smithi164.stdout:Version: 5.4.0.195.193 2024-09-13T06:49:55.501 INFO:teuthology.orchestra.run.smithi164.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T06:49:55.501 INFO:teuthology.orchestra.run.smithi164.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T06:49:55.501 INFO:teuthology.orchestra.run.smithi164.stdout:Recommends: thermald 2024-09-13T06:49:55.501 INFO:teuthology.orchestra.run.smithi164.stdout:Description: Generic Linux kernel image 2024-09-13T06:49:55.501 INFO:teuthology.orchestra.run.smithi164.stdout: This package will always depend on the latest generic kernel image 2024-09-13T06:49:55.501 INFO:teuthology.orchestra.run.smithi164.stdout: available. 2024-09-13T06:49:55.502 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-09-13T06:49:55.506 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.ztWO7m7eLr 2024-09-13T06:49:55.506 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ztWO7m7eLr 2024-09-13T06:49:55.564 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.ztWO7m7eLr 2024-09-13T06:49:55.609 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.ztWO7m7eLr is 9KB 2024-09-13T06:49:55.621 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.ztWO7m7eLr 2024-09-13T06:49:55.628 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T06:49:55.683 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T06:49:55.683 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T06:49:55.683 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T06:49:55.717 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-13T06:49:55.717 DEBUG:teuthology.orchestra.run.smithi164:> sudo update-grub 2024-09-13T06:49:55.860 DEBUG:teuthology.orchestra.run.smithi118:> dpkg -s linux-image-generic 2024-09-13T06:49:55.912 INFO:teuthology.orchestra.run.smithi118.stdout:Package: linux-image-generic 2024-09-13T06:49:55.912 INFO:teuthology.orchestra.run.smithi118.stdout:Status: install ok installed 2024-09-13T06:49:55.913 INFO:teuthology.orchestra.run.smithi118.stdout:Priority: optional 2024-09-13T06:49:55.913 INFO:teuthology.orchestra.run.smithi118.stdout:Section: kernel 2024-09-13T06:49:55.913 INFO:teuthology.orchestra.run.smithi118.stdout:Installed-Size: 20 2024-09-13T06:49:55.913 INFO:teuthology.orchestra.run.smithi118.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T06:49:55.913 INFO:teuthology.orchestra.run.smithi118.stdout:Architecture: amd64 2024-09-13T06:49:55.913 INFO:teuthology.orchestra.run.smithi118.stdout:Source: linux-meta 2024-09-13T06:49:55.913 INFO:teuthology.orchestra.run.smithi118.stdout:Version: 5.4.0.195.193 2024-09-13T06:49:55.913 INFO:teuthology.orchestra.run.smithi118.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T06:49:55.913 INFO:teuthology.orchestra.run.smithi118.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T06:49:55.914 INFO:teuthology.orchestra.run.smithi118.stdout:Recommends: thermald 2024-09-13T06:49:55.914 INFO:teuthology.orchestra.run.smithi118.stdout:Description: Generic Linux kernel image 2024-09-13T06:49:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: This package will always depend on the latest generic kernel image 2024-09-13T06:49:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: available. 2024-09-13T06:49:55.914 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-13T06:49:55.914 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T06:49:55.914 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T06:49:55.915 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T06:49:55.915 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi118.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T06:49:55.915 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get clean 2024-09-13T06:49:56.010 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get update 2024-09-13T06:49:56.143 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T06:49:56.259 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T06:49:56.341 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T06:49:56.423 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T06:49:57.176 INFO:teuthology.orchestra.run.smithi164.stderr:Sourcing file `/etc/default/grub' 2024-09-13T06:49:57.181 INFO:teuthology.orchestra.run.smithi164.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T06:49:57.192 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-09-13T06:49:57.491 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T06:49:57.503 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T06:49:57.956 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T06:49:57.968 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T06:49:58.063 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-09-13T06:49:58.088 DEBUG:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T06:49:58.165 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-09-13T06:49:58.384 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2024-09-13T06:49:58.385 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2024-09-13T06:49:58.622 INFO:teuthology.orchestra.run.smithi118.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-13T06:49:58.623 INFO:teuthology.orchestra.run.smithi118.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T06:49:58.624 DEBUG:teuthology.orchestra.run.smithi118:> dpkg -s linux-image-generic 2024-09-13T06:49:58.641 INFO:teuthology.orchestra.run.smithi118.stdout:Package: linux-image-generic 2024-09-13T06:49:58.641 INFO:teuthology.orchestra.run.smithi118.stdout:Status: install ok installed 2024-09-13T06:49:58.642 INFO:teuthology.orchestra.run.smithi118.stdout:Priority: optional 2024-09-13T06:49:58.642 INFO:teuthology.orchestra.run.smithi118.stdout:Section: kernel 2024-09-13T06:49:58.642 INFO:teuthology.orchestra.run.smithi118.stdout:Installed-Size: 20 2024-09-13T06:49:58.642 INFO:teuthology.orchestra.run.smithi118.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T06:49:58.642 INFO:teuthology.orchestra.run.smithi118.stdout:Architecture: amd64 2024-09-13T06:49:58.642 INFO:teuthology.orchestra.run.smithi118.stdout:Source: linux-meta 2024-09-13T06:49:58.642 INFO:teuthology.orchestra.run.smithi118.stdout:Version: 5.4.0.195.193 2024-09-13T06:49:58.642 INFO:teuthology.orchestra.run.smithi118.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T06:49:58.642 INFO:teuthology.orchestra.run.smithi118.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T06:49:58.642 INFO:teuthology.orchestra.run.smithi118.stdout:Recommends: thermald 2024-09-13T06:49:58.642 INFO:teuthology.orchestra.run.smithi118.stdout:Description: Generic Linux kernel image 2024-09-13T06:49:58.642 INFO:teuthology.orchestra.run.smithi118.stdout: This package will always depend on the latest generic kernel image 2024-09-13T06:49:58.642 INFO:teuthology.orchestra.run.smithi118.stdout: available. 2024-09-13T06:49:58.643 DEBUG:teuthology.orchestra.run.smithi118:> mktemp 2024-09-13T06:49:58.649 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.hd6xn1Nl6M 2024-09-13T06:49:58.650 DEBUG:teuthology.orchestra.run.smithi118:> sudo cp /boot/grub/grub.cfg /tmp/tmp.hd6xn1Nl6M 2024-09-13T06:49:58.708 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.hd6xn1Nl6M 2024-09-13T06:49:58.750 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.hd6xn1Nl6M is 9KB 2024-09-13T06:49:58.763 DEBUG:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.hd6xn1Nl6M 2024-09-13T06:49:58.769 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T06:49:58.827 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T06:49:58.827 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T06:49:58.827 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T06:49:58.868 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-13T06:49:58.868 DEBUG:teuthology.orchestra.run.smithi118:> sudo update-grub 2024-09-13T06:49:59.655 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-09-13T06:49:59.656 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-09-13T06:50:00.505 INFO:teuthology.orchestra.run.smithi118.stderr:Sourcing file `/etc/default/grub' 2024-09-13T06:50:00.513 INFO:teuthology.orchestra.run.smithi118.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T06:50:00.525 INFO:teuthology.orchestra.run.smithi118.stderr:Generating grub configuration file ... 2024-09-13T06:50:00.768 INFO:teuthology.orchestra.run.smithi118.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T06:50:00.780 INFO:teuthology.orchestra.run.smithi118.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T06:50:01.133 INFO:teuthology.orchestra.run.smithi118.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T06:50:01.145 INFO:teuthology.orchestra.run.smithi118.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T06:50:02.761 INFO:teuthology.orchestra.run.smithi118.stderr:done 2024-09-13T06:50:02.763 DEBUG:teuthology.orchestra.run.smithi118:> sudo shutdown -r now 2024-09-13T06:50:29.684 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T06:50:29.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-13T06:50:29.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T06:50:32.769 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T06:50:32.770 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-13T06:50:32.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T06:50:48.253 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-13T06:50:51.325 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-09-13T06:50:57.259 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-13T06:50:57.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T06:51:00.329 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-13T06:51:00.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T06:51:00.356 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-13T06:51:03.421 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-09-13T06:51:12.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-13T06:51:12.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T06:51:15.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-13T06:51:15.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T06:51:18.493 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-09-13T06:51:33.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-13T06:51:33.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T06:52:05.298 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-09-13T06:52:05.871 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-13T06:52:05.871 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-195-generic"... 2024-09-13T06:52:05.871 DEBUG:teuthology.orchestra.run.smithi118:> uname -r 2024-09-13T06:52:05.878 INFO:teuthology.orchestra.run.smithi118.stdout:5.4.0-195-generic 2024-09-13T06:52:05.878 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-13T06:52:05.878 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T06:52:05.878 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T06:52:06.879 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T06:52:06.879 DEBUG:teuthology.orchestra.run.smithi118:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T06:52:06.978 INFO:teuthology.orchestra.run.smithi118.stdout:ttyS1 2024-09-13T06:52:07.006 DEBUG:teuthology.parallel:result is None 2024-09-13T06:52:12.394 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T06:52:27.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-13T06:52:27.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T06:52:27.672 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-13T06:52:28.253 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-13T06:52:28.253 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-195-generic"... 2024-09-13T06:52:28.253 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-13T06:52:28.301 INFO:teuthology.orchestra.run.smithi164.stdout:5.4.0-195-generic 2024-09-13T06:52:28.302 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-13T06:52:28.302 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T06:52:28.302 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T06:52:29.304 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T06:52:29.304 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T06:52:29.386 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2024-09-13T06:52:29.425 DEBUG:teuthology.parallel:result is None 2024-09-13T06:52:29.425 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T06:52:29.432 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T06:52:29.433 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T06:52:29.436 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T06:52:29.443 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T06:52:29.493 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T06:52:29.520 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T06:52:29.520 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T06:52:29.522 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T06:52:29.540 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T06:52:29.546 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T06:52:29.546 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T06:52:29.575 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T06:52:29.600 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T06:52:29.600 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T06:52:29.608 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T06:52:29.610 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T06:52:29.612 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T06:52:29.620 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T06:52:29.621 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T06:52:29.623 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T06:52:29.637 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T06:52:29.646 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T06:52:29.646 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T06:52:29.649 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T06:52:29.685 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T06:52:29.700 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T06:52:29.707 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T06:52:29.707 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T06:52:29.734 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T06:52:29.741 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T06:52:29.748 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T06:52:29.748 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T06:52:29.775 DEBUG:teuthology.orchestra.run.smithi118:> sudo service rsyslog restart 2024-09-13T06:52:29.778 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-09-13T06:52:29.870 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T06:52:29.878 INFO:teuthology.task.internal:Starting timer... 2024-09-13T06:52:29.878 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T06:52:29.888 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T06:52:29.896 DEBUG:teuthology.task.selinux:Excluding smithi118: OS 'ubuntu' does not support SELinux 2024-09-13T06:52:29.896 DEBUG:teuthology.task.selinux:Excluding smithi164: OS 'ubuntu' does not support SELinux 2024-09-13T06:52:29.896 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T06:52:29.896 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T06:52:29.896 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T06:52:29.896 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T06:52:29.904 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T06:52:29.917 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T06:52:29.918 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi118.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T06:58:05.168 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi118.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-09-13T06:58:05.169 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-13T06:58:05.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T06:58:05.269 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-09-13T06:58:05.391 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-09-13T06:58:05.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-13T06:58:05.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T06:58:05.500 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-13T06:58:05.630 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-13T06:58:05.631 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T06:58:05.642 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T06:58:05.642 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T06:58:05.643 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T06:58:05.645 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T06:58:05.645 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T06:58:05.679 INFO:teuthology.orchestra.run.smithi118.stdout:13 Sep 06:58:05 ntpd[13739]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T06:58:05.679 INFO:teuthology.orchestra.run.smithi118.stdout:13 Sep 06:58:05 ntpd[13739]: Command line: ntpd -gq 2024-09-13T06:58:05.681 INFO:teuthology.orchestra.run.smithi164.stdout:13 Sep 06:58:05 ntpd[13709]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T06:58:05.681 INFO:teuthology.orchestra.run.smithi164.stdout:13 Sep 06:58:05 ntpd[13709]: Command line: ntpd -gq 2024-09-13T06:58:05.681 INFO:teuthology.orchestra.run.smithi118.stdout:13 Sep 06:58:05 ntpd[13739]: proto: precision = 0.082 usec (-23) 2024-09-13T06:58:05.681 INFO:teuthology.orchestra.run.smithi118.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T06:58:05.682 INFO:teuthology.orchestra.run.smithi118.stdout:13 Sep 06:58:05 ntpd[13739]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T06:58:05.682 INFO:teuthology.orchestra.run.smithi118.stdout:13 Sep 06:58:05 ntpd[13739]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T06:58:05.682 INFO:teuthology.orchestra.run.smithi118.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T06:58:05.683 INFO:teuthology.orchestra.run.smithi118.stdout:13 Sep 06:58:05 ntpd[13739]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T06:58:05.683 INFO:teuthology.orchestra.run.smithi118.stdout:13 Sep 06:58:05 ntpd[13739]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T06:58:05.683 INFO:teuthology.orchestra.run.smithi118.stdout:13 Sep 06:58:05 ntpd[13739]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T06:58:05.683 INFO:teuthology.orchestra.run.smithi118.stdout:13 Sep 06:58:05 ntpd[13739]: Listen normally on 3 enp3s0f1 172.21.15.118:123 2024-09-13T06:58:05.683 INFO:teuthology.orchestra.run.smithi118.stdout:13 Sep 06:58:05 ntpd[13739]: Listen normally on 4 lo [::1]:123 2024-09-13T06:58:05.683 INFO:teuthology.orchestra.run.smithi118.stdout:13 Sep 06:58:05 ntpd[13739]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d725%5]:123 2024-09-13T06:58:05.683 INFO:teuthology.orchestra.run.smithi118.stdout:13 Sep 06:58:05 ntpd[13739]: Listening on routing socket on fd #22 for interface updates 2024-09-13T06:58:05.684 INFO:teuthology.orchestra.run.smithi164.stdout:13 Sep 06:58:05 ntpd[13709]: proto: precision = 0.078 usec (-24) 2024-09-13T06:58:05.684 INFO:teuthology.orchestra.run.smithi164.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T06:58:05.684 INFO:teuthology.orchestra.run.smithi164.stdout:13 Sep 06:58:05 ntpd[13709]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T06:58:05.685 INFO:teuthology.orchestra.run.smithi164.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T06:58:05.686 INFO:teuthology.orchestra.run.smithi164.stdout:13 Sep 06:58:05 ntpd[13709]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T06:58:05.686 INFO:teuthology.orchestra.run.smithi164.stdout:13 Sep 06:58:05 ntpd[13709]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T06:58:05.686 INFO:teuthology.orchestra.run.smithi164.stdout:13 Sep 06:58:05 ntpd[13709]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T06:58:05.686 INFO:teuthology.orchestra.run.smithi164.stdout:13 Sep 06:58:05 ntpd[13709]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T06:58:05.686 INFO:teuthology.orchestra.run.smithi164.stdout:13 Sep 06:58:05 ntpd[13709]: Listen normally on 3 enp3s0f1 172.21.15.164:123 2024-09-13T06:58:05.686 INFO:teuthology.orchestra.run.smithi164.stdout:13 Sep 06:58:05 ntpd[13709]: Listen normally on 4 lo [::1]:123 2024-09-13T06:58:05.686 INFO:teuthology.orchestra.run.smithi164.stdout:13 Sep 06:58:05 ntpd[13709]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:73f3%5]:123 2024-09-13T06:58:05.686 INFO:teuthology.orchestra.run.smithi164.stdout:13 Sep 06:58:05 ntpd[13709]: Listening on routing socket on fd #22 for interface updates 2024-09-13T06:58:06.680 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:06 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:06.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:06 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:06.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:06 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:06.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:06 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:06.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:06 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:06.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:06 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:07.680 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:07 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:07.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:07 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:07.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:07 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:07.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:07 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:07.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:07 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:07.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:07 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:08 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:08 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:08 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:08 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:08 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:08 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:08.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:08 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:08.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:08 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:08.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:08 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:08.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:08 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:08.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:08 ntpd[13709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:08.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:08 ntpd[13709]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:09.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:09 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:09.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:09 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:09.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:09 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:09.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:09 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:09.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:09 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:09.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:09 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:09.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:09 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:09.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:09 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:09.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:09.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:09.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:09.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:09.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:09.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:09.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:09.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:09.684 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:09.684 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:09.684 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:09.684 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:09.684 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:09.684 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:09 ntpd[13709]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:10.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:10 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:10.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:10 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:10.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:10 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:10.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:10 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:10 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:10 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:10 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:10 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:10 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:10 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:10 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:10 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:10.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:10 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:10.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:10 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:10.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:10 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:10.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:10 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:10.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:10 ntpd[13709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:10.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:10 ntpd[13709]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:11.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:11 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:11.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:11 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:11.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:11 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:11.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:11 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:11.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:11 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:11.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:11 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:11.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:11 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:11.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:11 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:11.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:11 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:11.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:11 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:11.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:11 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:11.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:11 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:11.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:11 ntpd[13709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:11.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:11 ntpd[13709]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:11.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:11 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:11.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:11 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:11.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:11 ntpd[13709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:11.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:11 ntpd[13709]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:12.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:12 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:12.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:12 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:12.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:12 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:12.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:12 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:12.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:12 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:12.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:12 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:12.681 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:12 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:12.682 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:12 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:12.682 INFO:teuthology.orchestra.run.smithi118.stdout:13 Sep 06:58:12 ntpd[13739]: ntpd: time slew +0.000222 s 2024-09-13T06:58:12.682 INFO:teuthology.orchestra.run.smithi118.stdout:ntpd: time slew +0.000222s 2024-09-13T06:58:12.682 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:12 ntpd[13739]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T06:58:12.683 INFO:teuthology.orchestra.run.smithi118.stderr:13 Sep 06:58:12 ntpd[13739]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T06:58:12.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:12 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:12.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:12 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:12.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:12 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:12.684 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:12 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:12.684 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:12 ntpd[13709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:12.684 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:12 ntpd[13709]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:12.752 INFO:teuthology.orchestra.run.smithi118.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T06:58:12.753 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================== 2024-09-13T06:58:12.753 INFO:teuthology.orchestra.run.smithi118.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T06:58:12.753 INFO:teuthology.orchestra.run.smithi118.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T06:58:12.753 INFO:teuthology.orchestra.run.smithi118.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T06:58:12.753 INFO:teuthology.orchestra.run.smithi118.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T06:58:13.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:13 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:13.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:13 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:13.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:13 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:13.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:13 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:13.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:13 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:13.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:13 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:13.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:13 ntpd[13709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:13.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:13 ntpd[13709]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:13.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:13 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:13.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:13 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:13.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:13 ntpd[13709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:13.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:13 ntpd[13709]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:14.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:14 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:14.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:14 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:14.682 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:14 ntpd[13709]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T06:58:14.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:14 ntpd[13709]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T06:58:14.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:14 ntpd[13709]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T06:58:14.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:14 ntpd[13709]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T06:58:14.683 INFO:teuthology.orchestra.run.smithi164.stdout:13 Sep 06:58:14 ntpd[13709]: ntpd: time slew -0.002447 s 2024-09-13T06:58:14.683 INFO:teuthology.orchestra.run.smithi164.stdout:ntpd: time slew -0.002447s 2024-09-13T06:58:14.683 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:14 ntpd[13709]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T06:58:14.684 INFO:teuthology.orchestra.run.smithi164.stderr:13 Sep 06:58:14 ntpd[13709]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T06:58:14.741 INFO:teuthology.orchestra.run.smithi164.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T06:58:14.742 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================== 2024-09-13T06:58:14.742 INFO:teuthology.orchestra.run.smithi164.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T06:58:14.742 INFO:teuthology.orchestra.run.smithi164.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T06:58:14.742 INFO:teuthology.orchestra.run.smithi164.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T06:58:14.742 INFO:teuthology.orchestra.run.smithi164.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T06:58:14.743 INFO:teuthology.run_tasks:Running task install... 2024-09-13T06:58:14.754 DEBUG:teuthology.task.install:project ceph 2024-09-13T06:58:14.754 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}} 2024-09-13T06:58:14.754 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'} 2024-09-13T06:58:14.754 INFO:teuthology.task.install:Using flavor: default 2024-09-13T06:58:14.760 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T06:58:14.760 INFO:teuthology.task.install:extra packages: [] 2024-09-13T06:58:14.761 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-key list | grep Ceph 2024-09-13T06:58:14.761 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-key list | grep Ceph 2024-09-13T06:58:14.861 INFO:teuthology.orchestra.run.smithi118.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T06:58:14.886 INFO:teuthology.orchestra.run.smithi164.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T06:58:14.923 INFO:teuthology.orchestra.run.smithi118.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T06:58:14.923 INFO:teuthology.orchestra.run.smithi118.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T06:58:14.925 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T06:58:14.925 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T06:58:14.946 INFO:teuthology.orchestra.run.smithi164.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T06:58:14.946 INFO:teuthology.orchestra.run.smithi164.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T06:58:14.947 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T06:58:14.947 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T06:58:15.077 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default/ 2024-09-13T06:58:15.077 INFO:teuthology.task.install.deb:Package version is 18.2.4-820-g48a7de7e-1focal 2024-09-13T06:58:15.093 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default/ 2024-09-13T06:58:15.093 INFO:teuthology.task.install.deb:Package version is 18.2.4-820-g48a7de7e-1focal 2024-09-13T06:58:15.203 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T06:58:15.203 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T06:58:15.216 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T06:58:15.217 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T06:58:15.220 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get update 2024-09-13T06:58:15.234 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get update 2024-09-13T06:58:15.359 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T06:58:15.371 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T06:58:15.371 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T06:58:15.377 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T06:58:15.394 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T06:58:15.413 INFO:teuthology.orchestra.run.smithi118.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal InRelease 2024-09-13T06:58:15.423 INFO:teuthology.orchestra.run.smithi164.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal InRelease 2024-09-13T06:58:15.434 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T06:58:15.445 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T06:58:15.457 INFO:teuthology.orchestra.run.smithi118.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T06:58:15.468 INFO:teuthology.orchestra.run.smithi164.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T06:58:15.475 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T06:58:15.478 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T06:58:15.490 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T06:58:15.510 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main i386 Packages [3,433 B] 2024-09-13T06:58:15.523 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main i386 Packages [3,433 B] 2024-09-13T06:58:15.557 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T06:58:15.638 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T06:58:16.019 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 28.7 kB in 1s (45.0 kB/s) 2024-09-13T06:58:16.030 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 28.7 kB in 1s (45.6 kB/s) 2024-09-13T06:58:17.171 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-09-13T06:58:17.184 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-09-13T06:58:17.198 DEBUG:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-820-g48a7de7e-1focal cephadm=18.2.4-820-g48a7de7e-1focal ceph-mds=18.2.4-820-g48a7de7e-1focal ceph-mgr=18.2.4-820-g48a7de7e-1focal ceph-common=18.2.4-820-g48a7de7e-1focal ceph-fuse=18.2.4-820-g48a7de7e-1focal ceph-test=18.2.4-820-g48a7de7e-1focal ceph-volume=18.2.4-820-g48a7de7e-1focal radosgw=18.2.4-820-g48a7de7e-1focal python3-rados=18.2.4-820-g48a7de7e-1focal python3-rgw=18.2.4-820-g48a7de7e-1focal python3-cephfs=18.2.4-820-g48a7de7e-1focal python3-rbd=18.2.4-820-g48a7de7e-1focal libcephfs2=18.2.4-820-g48a7de7e-1focal libcephfs-dev=18.2.4-820-g48a7de7e-1focal librados2=18.2.4-820-g48a7de7e-1focal librbd1=18.2.4-820-g48a7de7e-1focal rbd-fuse=18.2.4-820-g48a7de7e-1focal 2024-09-13T06:58:17.208 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-820-g48a7de7e-1focal cephadm=18.2.4-820-g48a7de7e-1focal ceph-mds=18.2.4-820-g48a7de7e-1focal ceph-mgr=18.2.4-820-g48a7de7e-1focal ceph-common=18.2.4-820-g48a7de7e-1focal ceph-fuse=18.2.4-820-g48a7de7e-1focal ceph-test=18.2.4-820-g48a7de7e-1focal ceph-volume=18.2.4-820-g48a7de7e-1focal radosgw=18.2.4-820-g48a7de7e-1focal python3-rados=18.2.4-820-g48a7de7e-1focal python3-rgw=18.2.4-820-g48a7de7e-1focal python3-cephfs=18.2.4-820-g48a7de7e-1focal python3-rbd=18.2.4-820-g48a7de7e-1focal libcephfs2=18.2.4-820-g48a7de7e-1focal libcephfs-dev=18.2.4-820-g48a7de7e-1focal librados2=18.2.4-820-g48a7de7e-1focal librbd1=18.2.4-820-g48a7de7e-1focal rbd-fuse=18.2.4-820-g48a7de7e-1focal 2024-09-13T06:58:17.269 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-09-13T06:58:17.276 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-09-13T06:58:17.471 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2024-09-13T06:58:17.472 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2024-09-13T06:58:17.472 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2024-09-13T06:58:17.472 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2024-09-13T06:58:17.662 INFO:teuthology.orchestra.run.smithi118.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T06:58:17.664 INFO:teuthology.orchestra.run.smithi118.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T06:58:17.664 INFO:teuthology.orchestra.run.smithi118.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T06:58:17.664 INFO:teuthology.orchestra.run.smithi118.stdout:The following additional packages will be installed: 2024-09-13T06:58:17.664 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T06:58:17.664 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T06:58:17.664 INFO:teuthology.orchestra.run.smithi118.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T06:58:17.664 INFO:teuthology.orchestra.run.smithi118.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T06:58:17.664 INFO:teuthology.orchestra.run.smithi118.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T06:58:17.668 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T06:58:17.668 INFO:teuthology.orchestra.run.smithi164.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T06:58:17.668 INFO:teuthology.orchestra.run.smithi164.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T06:58:17.669 INFO:teuthology.orchestra.run.smithi164.stdout:The following additional packages will be installed: 2024-09-13T06:58:17.669 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T06:58:17.669 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T06:58:17.669 INFO:teuthology.orchestra.run.smithi164.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T06:58:17.669 INFO:teuthology.orchestra.run.smithi164.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T06:58:17.669 INFO:teuthology.orchestra.run.smithi164.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T06:58:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T06:58:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T06:58:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T06:58:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T06:58:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2024-09-13T06:58:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-09-13T06:58:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T06:58:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T06:58:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T06:58:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T06:58:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-13T06:58:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T06:58:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T06:58:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T06:58:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T06:58:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T06:58:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T06:58:17.673 INFO:teuthology.orchestra.run.smithi164.stdout:Suggested packages: 2024-09-13T06:58:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T06:58:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T06:58:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T06:58:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T06:58:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T06:58:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T06:58:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T06:58:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T06:58:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T06:58:17.674 INFO:teuthology.orchestra.run.smithi164.stdout:Recommended packages: 2024-09-13T06:58:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: btrfs-tools 2024-09-13T06:58:17.674 INFO:teuthology.orchestra.run.smithi118.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T06:58:17.674 INFO:teuthology.orchestra.run.smithi118.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T06:58:17.675 INFO:teuthology.orchestra.run.smithi118.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T06:58:17.675 INFO:teuthology.orchestra.run.smithi118.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T06:58:17.675 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2024-09-13T06:58:17.675 INFO:teuthology.orchestra.run.smithi118.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-09-13T06:58:17.675 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T06:58:17.675 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T06:58:17.675 INFO:teuthology.orchestra.run.smithi118.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T06:58:17.675 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T06:58:17.675 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-13T06:58:17.676 INFO:teuthology.orchestra.run.smithi118.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T06:58:17.676 INFO:teuthology.orchestra.run.smithi118.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T06:58:17.676 INFO:teuthology.orchestra.run.smithi118.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T06:58:17.676 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T06:58:17.676 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T06:58:17.676 INFO:teuthology.orchestra.run.smithi118.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T06:58:17.676 INFO:teuthology.orchestra.run.smithi118.stdout:Suggested packages: 2024-09-13T06:58:17.676 INFO:teuthology.orchestra.run.smithi118.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T06:58:17.676 INFO:teuthology.orchestra.run.smithi118.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T06:58:17.676 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T06:58:17.676 INFO:teuthology.orchestra.run.smithi118.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T06:58:17.676 INFO:teuthology.orchestra.run.smithi118.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T06:58:17.676 INFO:teuthology.orchestra.run.smithi118.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T06:58:17.677 INFO:teuthology.orchestra.run.smithi118.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T06:58:17.677 INFO:teuthology.orchestra.run.smithi118.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T06:58:17.677 INFO:teuthology.orchestra.run.smithi118.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T06:58:17.677 INFO:teuthology.orchestra.run.smithi118.stdout:Recommended packages: 2024-09-13T06:58:17.677 INFO:teuthology.orchestra.run.smithi118.stdout: btrfs-tools 2024-09-13T06:58:17.783 INFO:teuthology.orchestra.run.smithi118.stdout:The following NEW packages will be installed: 2024-09-13T06:58:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T06:58:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T06:58:17.783 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T06:58:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-13T06:58:17.786 INFO:teuthology.orchestra.run.smithi118.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-13T06:58:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-13T06:58:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-13T06:58:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-asyncssh-doc 2024-09-13T06:58:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: python-natsort-doc python-pastedeploy-tpl python3-asyncssh 2024-09-13T06:58:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2024-09-13T06:58:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T06:58:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2024-09-13T06:58:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T06:58:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T06:58:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T06:58:17.787 INFO:teuthology.orchestra.run.smithi118.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T06:58:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T06:58:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-13T06:58:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2024-09-13T06:58:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2024-09-13T06:58:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-13T06:58:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2024-09-13T06:58:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2024-09-13T06:58:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webtest python3-werkzeug qttranslations5-l10n radosgw rbd-fuse socat 2024-09-13T06:58:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: xmlstarlet 2024-09-13T06:58:17.788 INFO:teuthology.orchestra.run.smithi118.stdout:The following packages will be upgraded: 2024-09-13T06:58:17.788 INFO:teuthology.orchestra.run.smithi118.stdout: librados2 librbd1 2024-09-13T06:58:17.789 INFO:teuthology.orchestra.run.smithi164.stdout:The following NEW packages will be installed: 2024-09-13T06:58:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T06:58:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T06:58:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T06:58:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-13T06:58:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-13T06:58:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-13T06:58:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-13T06:58:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-asyncssh-doc 2024-09-13T06:58:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: python-natsort-doc python-pastedeploy-tpl python3-asyncssh 2024-09-13T06:58:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2024-09-13T06:58:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T06:58:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2024-09-13T06:58:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T06:58:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T06:58:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T06:58:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T06:58:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T06:58:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-13T06:58:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2024-09-13T06:58:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2024-09-13T06:58:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-13T06:58:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2024-09-13T06:58:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2024-09-13T06:58:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webtest python3-werkzeug qttranslations5-l10n radosgw rbd-fuse socat 2024-09-13T06:58:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet 2024-09-13T06:58:17.793 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages will be upgraded: 2024-09-13T06:58:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 librbd1 2024-09-13T06:58:17.863 INFO:teuthology.orchestra.run.smithi118.stdout:2 upgraded, 110 newly installed, 0 to remove and 295 not upgraded. 2024-09-13T06:58:17.863 INFO:teuthology.orchestra.run.smithi118.stdout:Need to get 199 MB of archives. 2024-09-13T06:58:17.863 INFO:teuthology.orchestra.run.smithi118.stdout:After this operation, 977 MB of additional disk space will be used. 2024-09-13T06:58:17.863 INFO:teuthology.orchestra.run.smithi118.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T06:58:17.933 INFO:teuthology.orchestra.run.smithi118.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-820-g48a7de7e-1focal [3,456 kB] 2024-09-13T06:58:17.941 INFO:teuthology.orchestra.run.smithi164.stdout:2 upgraded, 110 newly installed, 0 to remove and 295 not upgraded. 2024-09-13T06:58:17.941 INFO:teuthology.orchestra.run.smithi164.stdout:Need to get 199 MB of archives. 2024-09-13T06:58:17.941 INFO:teuthology.orchestra.run.smithi164.stdout:After this operation, 977 MB of additional disk space will be used. 2024-09-13T06:58:17.942 INFO:teuthology.orchestra.run.smithi164.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-820-g48a7de7e-1focal [3,456 kB] 2024-09-13T06:58:17.946 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T06:58:17.949 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T06:58:17.967 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T06:58:17.987 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T06:58:18.019 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T06:58:18.022 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T06:58:18.028 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T06:58:18.029 INFO:teuthology.orchestra.run.smithi118.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T06:58:18.030 INFO:teuthology.orchestra.run.smithi118.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T06:58:18.031 INFO:teuthology.orchestra.run.smithi118.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T06:58:18.031 INFO:teuthology.orchestra.run.smithi118.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T06:58:18.034 INFO:teuthology.orchestra.run.smithi118.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T06:58:18.035 INFO:teuthology.orchestra.run.smithi118.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T06:58:18.037 INFO:teuthology.orchestra.run.smithi118.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T06:58:18.054 INFO:teuthology.orchestra.run.smithi118.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T06:58:18.054 INFO:teuthology.orchestra.run.smithi118.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T06:58:18.055 INFO:teuthology.orchestra.run.smithi118.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T06:58:18.056 INFO:teuthology.orchestra.run.smithi118.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T06:58:18.057 INFO:teuthology.orchestra.run.smithi118.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T06:58:18.057 INFO:teuthology.orchestra.run.smithi118.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T06:58:18.058 INFO:teuthology.orchestra.run.smithi118.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T06:58:18.059 INFO:teuthology.orchestra.run.smithi118.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T06:58:18.061 INFO:teuthology.orchestra.run.smithi118.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T06:58:18.071 INFO:teuthology.orchestra.run.smithi118.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T06:58:18.080 INFO:teuthology.orchestra.run.smithi118.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T06:58:18.085 INFO:teuthology.orchestra.run.smithi118.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T06:58:18.086 INFO:teuthology.orchestra.run.smithi118.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T06:58:18.087 INFO:teuthology.orchestra.run.smithi118.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T06:58:18.088 INFO:teuthology.orchestra.run.smithi118.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T06:58:18.089 INFO:teuthology.orchestra.run.smithi118.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T06:58:18.092 INFO:teuthology.orchestra.run.smithi118.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T06:58:18.092 INFO:teuthology.orchestra.run.smithi118.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T06:58:18.094 INFO:teuthology.orchestra.run.smithi118.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T06:58:18.105 INFO:teuthology.orchestra.run.smithi118.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T06:58:18.107 INFO:teuthology.orchestra.run.smithi118.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T06:58:18.108 INFO:teuthology.orchestra.run.smithi118.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T06:58:18.122 INFO:teuthology.orchestra.run.smithi118.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T06:58:18.123 INFO:teuthology.orchestra.run.smithi118.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T06:58:18.124 INFO:teuthology.orchestra.run.smithi118.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T06:58:18.125 INFO:teuthology.orchestra.run.smithi118.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T06:58:18.126 INFO:teuthology.orchestra.run.smithi118.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T06:58:18.127 INFO:teuthology.orchestra.run.smithi118.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T06:58:18.128 INFO:teuthology.orchestra.run.smithi118.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T06:58:18.139 INFO:teuthology.orchestra.run.smithi118.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T06:58:18.140 INFO:teuthology.orchestra.run.smithi118.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T06:58:18.142 INFO:teuthology.orchestra.run.smithi118.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T06:58:18.157 INFO:teuthology.orchestra.run.smithi118.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T06:58:18.158 INFO:teuthology.orchestra.run.smithi118.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T06:58:18.159 INFO:teuthology.orchestra.run.smithi118.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T06:58:18.160 INFO:teuthology.orchestra.run.smithi118.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T06:58:18.186 INFO:teuthology.orchestra.run.smithi118.stdout:Get:52 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-820-g48a7de7e-1focal [4,039 kB] 2024-09-13T06:58:18.213 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-820-g48a7de7e-1focal [4,039 kB] 2024-09-13T06:58:18.241 INFO:teuthology.orchestra.run.smithi118.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T06:58:18.257 INFO:teuthology.orchestra.run.smithi118.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T06:58:18.259 INFO:teuthology.orchestra.run.smithi118.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T06:58:18.347 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-820-g48a7de7e-1focal [746 kB] 2024-09-13T06:58:18.367 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-820-g48a7de7e-1focal [387 kB] 2024-09-13T06:58:18.384 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-820-g48a7de7e-1focal [31.9 kB] 2024-09-13T06:58:18.384 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-820-g48a7de7e-1focal [200 kB] 2024-09-13T06:58:18.387 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-820-g48a7de7e-1focal [62.5 kB] 2024-09-13T06:58:18.387 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-820-g48a7de7e-1focal [367 kB] 2024-09-13T06:58:18.389 INFO:teuthology.orchestra.run.smithi164.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T06:58:18.403 INFO:teuthology.orchestra.run.smithi164.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-820-g48a7de7e-1focal [6,628 kB] 2024-09-13T06:58:18.403 INFO:teuthology.orchestra.run.smithi164.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T06:58:18.406 INFO:teuthology.orchestra.run.smithi118.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T06:58:18.406 INFO:teuthology.orchestra.run.smithi118.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-820-g48a7de7e-1focal [746 kB] 2024-09-13T06:58:18.406 INFO:teuthology.orchestra.run.smithi118.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T06:58:18.408 INFO:teuthology.orchestra.run.smithi118.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T06:58:18.409 INFO:teuthology.orchestra.run.smithi118.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T06:58:18.410 INFO:teuthology.orchestra.run.smithi118.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T06:58:18.411 INFO:teuthology.orchestra.run.smithi118.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T06:58:18.411 INFO:teuthology.orchestra.run.smithi118.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T06:58:18.413 INFO:teuthology.orchestra.run.smithi118.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T06:58:18.414 INFO:teuthology.orchestra.run.smithi118.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T06:58:18.419 INFO:teuthology.orchestra.run.smithi118.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-820-g48a7de7e-1focal [387 kB] 2024-09-13T06:58:18.422 INFO:teuthology.orchestra.run.smithi118.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-820-g48a7de7e-1focal [31.9 kB] 2024-09-13T06:58:18.422 INFO:teuthology.orchestra.run.smithi118.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-820-g48a7de7e-1focal [200 kB] 2024-09-13T06:58:18.423 INFO:teuthology.orchestra.run.smithi118.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T06:58:18.424 INFO:teuthology.orchestra.run.smithi118.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-820-g48a7de7e-1focal [62.5 kB] 2024-09-13T06:58:18.425 INFO:teuthology.orchestra.run.smithi118.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-820-g48a7de7e-1focal [367 kB] 2024-09-13T06:58:18.428 INFO:teuthology.orchestra.run.smithi118.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-820-g48a7de7e-1focal [6,628 kB] 2024-09-13T06:58:18.433 INFO:teuthology.orchestra.run.smithi118.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T06:58:18.434 INFO:teuthology.orchestra.run.smithi118.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T06:58:18.436 INFO:teuthology.orchestra.run.smithi118.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T06:58:18.437 INFO:teuthology.orchestra.run.smithi118.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T06:58:18.441 INFO:teuthology.orchestra.run.smithi118.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T06:58:18.443 INFO:teuthology.orchestra.run.smithi118.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T06:58:18.458 INFO:teuthology.orchestra.run.smithi118.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T06:58:18.461 INFO:teuthology.orchestra.run.smithi118.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T06:58:18.462 INFO:teuthology.orchestra.run.smithi118.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T06:58:18.486 INFO:teuthology.orchestra.run.smithi164.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T06:58:18.498 INFO:teuthology.orchestra.run.smithi118.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T06:58:18.499 INFO:teuthology.orchestra.run.smithi118.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T06:58:18.499 INFO:teuthology.orchestra.run.smithi118.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T06:58:18.500 INFO:teuthology.orchestra.run.smithi118.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T06:58:18.511 INFO:teuthology.orchestra.run.smithi118.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T06:58:18.599 INFO:teuthology.orchestra.run.smithi164.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-820-g48a7de7e-1focal [113 kB] 2024-09-13T06:58:18.600 INFO:teuthology.orchestra.run.smithi164.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-820-g48a7de7e-1focal [478 kB] 2024-09-13T06:58:18.624 INFO:teuthology.orchestra.run.smithi164.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-820-g48a7de7e-1focal [25.2 MB] 2024-09-13T06:58:18.625 INFO:teuthology.orchestra.run.smithi118.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T06:58:18.628 INFO:teuthology.orchestra.run.smithi118.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T06:58:18.629 INFO:teuthology.orchestra.run.smithi118.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T06:58:18.630 INFO:teuthology.orchestra.run.smithi118.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T06:58:18.646 INFO:teuthology.orchestra.run.smithi118.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-820-g48a7de7e-1focal [113 kB] 2024-09-13T06:58:18.648 INFO:teuthology.orchestra.run.smithi118.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-820-g48a7de7e-1focal [478 kB] 2024-09-13T06:58:18.658 INFO:teuthology.orchestra.run.smithi118.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-820-g48a7de7e-1focal [25.2 MB] 2024-09-13T06:58:18.728 INFO:teuthology.orchestra.run.smithi164.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T06:58:18.738 INFO:teuthology.orchestra.run.smithi164.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T06:58:18.766 INFO:teuthology.orchestra.run.smithi164.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T06:58:18.774 INFO:teuthology.orchestra.run.smithi164.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T06:58:18.775 INFO:teuthology.orchestra.run.smithi164.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T06:58:18.778 INFO:teuthology.orchestra.run.smithi164.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T06:58:18.780 INFO:teuthology.orchestra.run.smithi164.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T06:58:18.791 INFO:teuthology.orchestra.run.smithi164.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T06:58:18.794 INFO:teuthology.orchestra.run.smithi164.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T06:58:18.811 INFO:teuthology.orchestra.run.smithi164.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T06:58:18.881 INFO:teuthology.orchestra.run.smithi164.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T06:58:18.881 INFO:teuthology.orchestra.run.smithi164.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T06:58:18.882 INFO:teuthology.orchestra.run.smithi164.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T06:58:18.884 INFO:teuthology.orchestra.run.smithi164.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T06:58:18.885 INFO:teuthology.orchestra.run.smithi164.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T06:58:18.885 INFO:teuthology.orchestra.run.smithi164.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T06:58:18.888 INFO:teuthology.orchestra.run.smithi164.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T06:58:18.889 INFO:teuthology.orchestra.run.smithi164.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T06:58:18.892 INFO:teuthology.orchestra.run.smithi164.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T06:58:18.973 INFO:teuthology.orchestra.run.smithi164.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T06:58:18.974 INFO:teuthology.orchestra.run.smithi164.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T06:58:18.983 INFO:teuthology.orchestra.run.smithi164.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T06:58:18.983 INFO:teuthology.orchestra.run.smithi164.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T06:58:18.984 INFO:teuthology.orchestra.run.smithi164.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T06:58:18.985 INFO:teuthology.orchestra.run.smithi164.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T06:58:18.987 INFO:teuthology.orchestra.run.smithi164.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T06:58:18.989 INFO:teuthology.orchestra.run.smithi164.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T06:58:18.989 INFO:teuthology.orchestra.run.smithi164.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T06:58:19.055 INFO:teuthology.orchestra.run.smithi164.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T06:58:19.073 INFO:teuthology.orchestra.run.smithi164.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T06:58:19.076 INFO:teuthology.orchestra.run.smithi164.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T06:58:19.079 INFO:teuthology.orchestra.run.smithi164.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T06:58:19.079 INFO:teuthology.orchestra.run.smithi164.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T06:58:19.080 INFO:teuthology.orchestra.run.smithi164.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T06:58:19.081 INFO:teuthology.orchestra.run.smithi164.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T06:58:19.082 INFO:teuthology.orchestra.run.smithi164.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T06:58:19.082 INFO:teuthology.orchestra.run.smithi164.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T06:58:19.137 INFO:teuthology.orchestra.run.smithi164.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T06:58:19.218 INFO:teuthology.orchestra.run.smithi164.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T06:58:19.222 INFO:teuthology.orchestra.run.smithi164.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T06:58:19.222 INFO:teuthology.orchestra.run.smithi164.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T06:58:19.225 INFO:teuthology.orchestra.run.smithi164.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T06:58:19.228 INFO:teuthology.orchestra.run.smithi164.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T06:58:19.230 INFO:teuthology.orchestra.run.smithi164.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T06:58:19.230 INFO:teuthology.orchestra.run.smithi164.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T06:58:19.231 INFO:teuthology.orchestra.run.smithi164.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T06:58:19.497 INFO:teuthology.orchestra.run.smithi164.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-820-g48a7de7e-1focal [9,612 kB] 2024-09-13T06:58:19.555 INFO:teuthology.orchestra.run.smithi164.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T06:58:19.630 INFO:teuthology.orchestra.run.smithi164.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T06:58:19.633 INFO:teuthology.orchestra.run.smithi164.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T06:58:19.656 INFO:teuthology.orchestra.run.smithi164.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T06:58:19.658 INFO:teuthology.orchestra.run.smithi164.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T06:58:19.659 INFO:teuthology.orchestra.run.smithi164.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T06:58:19.659 INFO:teuthology.orchestra.run.smithi164.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T06:58:19.660 INFO:teuthology.orchestra.run.smithi164.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T06:58:19.660 INFO:teuthology.orchestra.run.smithi164.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T06:58:19.661 INFO:teuthology.orchestra.run.smithi164.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T06:58:19.661 INFO:teuthology.orchestra.run.smithi118.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-820-g48a7de7e-1focal [9,612 kB] 2024-09-13T06:58:19.701 INFO:teuthology.orchestra.run.smithi164.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-820-g48a7de7e-1focal [238 kB] 2024-09-13T06:58:19.703 INFO:teuthology.orchestra.run.smithi164.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-820-g48a7de7e-1focal [148 kB] 2024-09-13T06:58:19.714 INFO:teuthology.orchestra.run.smithi164.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-820-g48a7de7e-1focal [1,575 kB] 2024-09-13T06:58:19.719 INFO:teuthology.orchestra.run.smithi164.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T06:58:19.719 INFO:teuthology.orchestra.run.smithi164.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T06:58:19.798 INFO:teuthology.orchestra.run.smithi164.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-820-g48a7de7e-1focal [6,640 kB] 2024-09-13T06:58:19.801 INFO:teuthology.orchestra.run.smithi164.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T06:58:19.806 INFO:teuthology.orchestra.run.smithi164.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T06:58:19.809 INFO:teuthology.orchestra.run.smithi164.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T06:58:19.813 INFO:teuthology.orchestra.run.smithi164.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T06:58:19.814 INFO:teuthology.orchestra.run.smithi164.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T06:58:19.817 INFO:teuthology.orchestra.run.smithi164.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T06:58:19.819 INFO:teuthology.orchestra.run.smithi164.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T06:58:19.822 INFO:teuthology.orchestra.run.smithi118.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-820-g48a7de7e-1focal [238 kB] 2024-09-13T06:58:19.832 INFO:teuthology.orchestra.run.smithi118.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-820-g48a7de7e-1focal [148 kB] 2024-09-13T06:58:19.834 INFO:teuthology.orchestra.run.smithi118.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-820-g48a7de7e-1focal [1,575 kB] 2024-09-13T06:58:19.874 INFO:teuthology.orchestra.run.smithi118.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-820-g48a7de7e-1focal [6,640 kB] 2024-09-13T06:58:19.892 INFO:teuthology.orchestra.run.smithi164.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T06:58:19.896 INFO:teuthology.orchestra.run.smithi164.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T06:58:19.898 INFO:teuthology.orchestra.run.smithi164.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T06:58:20.025 INFO:teuthology.orchestra.run.smithi118.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-820-g48a7de7e-1focal [24.5 MB] 2024-09-13T06:58:20.062 INFO:teuthology.orchestra.run.smithi164.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T06:58:20.063 INFO:teuthology.orchestra.run.smithi164.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T06:58:20.063 INFO:teuthology.orchestra.run.smithi164.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T06:58:20.064 INFO:teuthology.orchestra.run.smithi164.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T06:58:20.077 INFO:teuthology.orchestra.run.smithi164.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T06:58:20.077 INFO:teuthology.orchestra.run.smithi164.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T06:58:20.079 INFO:teuthology.orchestra.run.smithi164.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T06:58:20.079 INFO:teuthology.orchestra.run.smithi164.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T06:58:20.079 INFO:teuthology.orchestra.run.smithi164.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T06:58:20.149 INFO:teuthology.orchestra.run.smithi164.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-820-g48a7de7e-1focal [24.5 MB] 2024-09-13T06:58:20.687 INFO:teuthology.orchestra.run.smithi118.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-820-g48a7de7e-1focal [13.8 kB] 2024-09-13T06:58:20.688 INFO:teuthology.orchestra.run.smithi118.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-820-g48a7de7e-1focal [855 kB] 2024-09-13T06:58:20.701 INFO:teuthology.orchestra.run.smithi118.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-820-g48a7de7e-1focal [2,295 kB] 2024-09-13T06:58:20.717 INFO:teuthology.orchestra.run.smithi118.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-820-g48a7de7e-1focal [238 kB] 2024-09-13T06:58:20.719 INFO:teuthology.orchestra.run.smithi118.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-820-g48a7de7e-1focal [143 kB] 2024-09-13T06:58:20.720 INFO:teuthology.orchestra.run.smithi118.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-820-g48a7de7e-1focal [7,015 kB] 2024-09-13T06:58:20.898 INFO:teuthology.orchestra.run.smithi118.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-820-g48a7de7e-1focal [7,868 kB] 2024-09-13T06:58:21.088 INFO:teuthology.orchestra.run.smithi118.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-820-g48a7de7e-1focal [13.9 kB] 2024-09-13T06:58:21.088 INFO:teuthology.orchestra.run.smithi118.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-820-g48a7de7e-1focal [46.9 MB] 2024-09-13T06:58:21.425 INFO:teuthology.orchestra.run.smithi164.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-820-g48a7de7e-1focal [13.8 kB] 2024-09-13T06:58:21.426 INFO:teuthology.orchestra.run.smithi164.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-820-g48a7de7e-1focal [855 kB] 2024-09-13T06:58:21.438 INFO:teuthology.orchestra.run.smithi164.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-820-g48a7de7e-1focal [2,295 kB] 2024-09-13T06:58:21.498 INFO:teuthology.orchestra.run.smithi164.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-820-g48a7de7e-1focal [238 kB] 2024-09-13T06:58:21.520 INFO:teuthology.orchestra.run.smithi164.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-820-g48a7de7e-1focal [143 kB] 2024-09-13T06:58:21.523 INFO:teuthology.orchestra.run.smithi164.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-820-g48a7de7e-1focal [7,015 kB] 2024-09-13T06:58:21.840 INFO:teuthology.orchestra.run.smithi164.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-820-g48a7de7e-1focal [7,868 kB] 2024-09-13T06:58:22.132 INFO:teuthology.orchestra.run.smithi164.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-820-g48a7de7e-1focal [13.9 kB] 2024-09-13T06:58:22.132 INFO:teuthology.orchestra.run.smithi164.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-820-g48a7de7e-1focal [46.9 MB] 2024-09-13T06:58:22.536 INFO:teuthology.orchestra.run.smithi118.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-820-g48a7de7e-1focal [120 kB] 2024-09-13T06:58:22.538 INFO:teuthology.orchestra.run.smithi118.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-820-g48a7de7e-1focal [37.4 kB] 2024-09-13T06:58:22.539 INFO:teuthology.orchestra.run.smithi118.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-820-g48a7de7e-1focal [13.5 MB] 2024-09-13T06:58:22.889 INFO:teuthology.orchestra.run.smithi118.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-820-g48a7de7e-1focal [107 kB] 2024-09-13T06:58:23.411 INFO:teuthology.orchestra.run.smithi164.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-820-g48a7de7e-1focal [120 kB] 2024-09-13T06:58:23.414 INFO:teuthology.orchestra.run.smithi164.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-820-g48a7de7e-1focal [37.4 kB] 2024-09-13T06:58:23.481 INFO:teuthology.orchestra.run.smithi164.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-820-g48a7de7e-1focal [13.5 MB] 2024-09-13T06:58:23.539 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 199 MB in 5s (39.0 MB/s) 2024-09-13T06:58:23.656 INFO:teuthology.orchestra.run.smithi164.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-820-g48a7de7e-1focal [107 kB] 2024-09-13T06:58:23.717 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T06:58:23.772 INFO:teuthology.orchestra.run.smithi118.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-13T06:58:23.776 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T06:58:23.859 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T06:58:24.279 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T06:58:24.295 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T06:58:24.305 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 199 MB in 6s (34.0 MB/s) 2024-09-13T06:58:24.337 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T06:58:24.474 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T06:58:24.530 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-13T06:58:24.534 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T06:58:24.623 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T06:58:24.674 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T06:58:24.691 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../002-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T06:58:24.724 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T06:58:24.976 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T06:58:24.994 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T06:58:25.034 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T06:58:25.219 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T06:58:25.239 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../003-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T06:58:25.288 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T06:58:25.396 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T06:58:25.412 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../002-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T06:58:25.454 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T06:58:25.864 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T06:58:25.881 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../004-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T06:58:25.922 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T06:58:25.941 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T06:58:25.958 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../003-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T06:58:26.009 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T06:58:26.284 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T06:58:26.301 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../005-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T06:58:26.351 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T06:58:26.511 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T06:58:26.527 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../004-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T06:58:26.593 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T06:58:26.779 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T06:58:26.795 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../006-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T06:58:26.837 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T06:58:26.930 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T06:58:26.947 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../005-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T06:58:26.988 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T06:58:27.235 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../007-librbd1_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:27.340 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librbd1 (18.2.4-820-g48a7de7e-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T06:58:27.366 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T06:58:27.383 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../006-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T06:58:27.424 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T06:58:27.804 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../007-librbd1_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:27.930 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librbd1 (18.2.4-820-g48a7de7e-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T06:58:28.061 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../008-librados2_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:28.195 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librados2 (18.2.4-820-g48a7de7e-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T06:58:28.619 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../008-librados2_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:28.759 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librados2 (18.2.4-820-g48a7de7e-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T06:58:29.074 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T06:58:29.090 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../009-libcephfs2_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:29.132 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libcephfs2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:29.455 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T06:58:29.477 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../009-libcephfs2_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:29.502 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rados. 2024-09-13T06:58:29.513 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libcephfs2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:29.518 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../010-python3-rados_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:29.560 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rados (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:29.840 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rados. 2024-09-13T06:58:29.858 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../010-python3-rados_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:29.899 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rados (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:29.946 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T06:58:29.962 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../011-python3-ceph-argparse_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:58:30.004 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-ceph-argparse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:30.253 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T06:58:30.272 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../012-python3-cephfs_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:30.285 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T06:58:30.302 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../011-python3-ceph-argparse_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:58:30.307 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cephfs (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:30.343 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-ceph-argparse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:30.587 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T06:58:30.603 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../012-python3-cephfs_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:30.635 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T06:58:30.637 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cephfs (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:30.652 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../013-python3-ceph-common_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:58:30.710 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:30.923 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T06:58:30.941 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../013-python3-ceph-common_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:58:30.982 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:31.055 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T06:58:31.073 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../014-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T06:58:31.113 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T06:58:31.301 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T06:58:31.318 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../014-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T06:58:31.385 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T06:58:31.415 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T06:58:31.435 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../015-python3-rbd_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:31.474 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rbd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:31.637 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T06:58:31.653 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../015-python3-rbd_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:31.695 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rbd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:31.860 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T06:58:31.877 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../016-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T06:58:31.910 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T06:58:32.063 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T06:58:32.081 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../016-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T06:58:32.123 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T06:58:32.246 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T06:58:32.262 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../017-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T06:58:32.338 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T06:58:32.409 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T06:58:32.429 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../017-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T06:58:32.467 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T06:58:32.658 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T06:58:32.676 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../018-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T06:58:32.716 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T06:58:32.838 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T06:58:32.854 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../018-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T06:58:32.896 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T06:58:33.086 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package librgw2. 2024-09-13T06:58:33.102 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../019-librgw2_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:33.135 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librgw2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:33.249 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librgw2. 2024-09-13T06:58:33.264 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../019-librgw2_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:33.307 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librgw2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:34.126 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T06:58:34.143 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:34.209 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:34.338 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T06:58:34.356 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:34.404 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:34.554 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liboath0. 2024-09-13T06:58:34.570 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T06:58:34.612 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T06:58:34.724 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liboath0. 2024-09-13T06:58:34.740 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T06:58:34.774 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T06:58:34.932 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T06:58:34.950 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:34.990 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libradosstriper1 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:35.094 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T06:58:35.110 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:35.152 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libradosstriper1 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:35.385 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-common. 2024-09-13T06:58:35.400 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../023-ceph-common_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:35.434 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:35.647 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-common. 2024-09-13T06:58:35.662 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../023-ceph-common_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:35.705 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:37.923 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-base. 2024-09-13T06:58:37.941 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../024-ceph-base_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:37.992 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-base (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:38.378 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-base. 2024-09-13T06:58:38.394 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../024-ceph-base_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:38.446 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-base (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:39.136 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T06:58:39.155 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T06:58:39.194 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T06:58:39.523 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T06:58:39.539 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../026-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T06:58:39.573 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T06:58:39.674 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T06:58:39.692 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T06:58:39.740 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T06:58:39.868 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T06:58:39.885 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T06:58:39.917 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T06:58:40.111 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T06:58:40.129 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../026-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T06:58:40.202 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T06:58:40.204 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T06:58:40.220 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T06:58:40.284 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T06:58:40.539 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T06:58:40.559 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T06:58:40.582 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-mako. 2024-09-13T06:58:40.597 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T06:58:40.599 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T06:58:40.640 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T06:58:40.908 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T06:58:40.928 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T06:58:40.967 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T06:58:40.993 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T06:58:41.010 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T06:58:41.051 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T06:58:41.295 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-mako. 2024-09-13T06:58:41.312 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T06:58:41.337 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T06:58:41.357 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T06:58:41.361 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T06:58:41.396 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T06:58:41.698 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T06:58:41.714 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T06:58:41.732 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-webob. 2024-09-13T06:58:41.750 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T06:58:41.773 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T06:58:41.790 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T06:58:42.076 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T06:58:42.084 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T06:58:42.091 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T06:58:42.100 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T06:58:42.134 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T06:58:42.142 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T06:58:42.454 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T06:58:42.471 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T06:58:42.478 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-webob. 2024-09-13T06:58:42.494 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T06:58:42.512 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T06:58:42.537 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T06:58:42.857 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T06:58:42.875 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T06:58:42.881 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T06:58:42.897 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T06:58:42.918 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T06:58:42.939 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T06:58:43.251 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T06:58:43.267 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T06:58:43.285 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T06:58:43.303 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T06:58:43.317 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T06:58:43.343 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T06:58:43.630 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-paste. 2024-09-13T06:58:43.649 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T06:58:43.679 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T06:58:43.699 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T06:58:43.707 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T06:58:43.749 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T06:58:44.040 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T06:58:44.058 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T06:58:44.098 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T06:58:44.107 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T06:58:44.123 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T06:58:44.165 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T06:58:44.426 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T06:58:44.443 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T06:58:44.477 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-paste. 2024-09-13T06:58:44.484 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T06:58:44.496 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T06:58:44.551 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T06:58:44.787 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T06:58:44.803 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T06:58:44.845 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T06:58:44.905 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T06:58:44.924 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T06:58:44.963 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T06:58:45.174 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T06:58:45.191 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T06:58:45.232 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T06:58:45.274 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T06:58:45.290 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T06:58:45.366 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T06:58:45.553 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T06:58:45.573 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T06:58:45.611 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T06:58:45.669 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T06:58:45.686 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T06:58:45.728 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T06:58:45.931 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T06:58:45.950 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../043-ceph-mgr-modules-core_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:58:45.990 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:46.079 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T06:58:46.097 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T06:58:46.137 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T06:58:46.353 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T06:58:46.373 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../044-libsqlite3-mod-ceph_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:46.411 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:46.485 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T06:58:46.504 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T06:58:46.543 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T06:58:46.752 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T06:58:46.768 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../045-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T06:58:46.809 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T06:58:46.838 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T06:58:46.856 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../043-ceph-mgr-modules-core_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:58:46.888 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:47.112 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T06:58:47.130 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../046-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T06:58:47.170 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T06:58:47.318 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T06:58:47.334 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../044-libsqlite3-mod-ceph_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:47.376 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:47.549 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T06:58:47.569 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../047-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T06:58:47.607 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T06:58:47.712 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T06:58:47.732 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../045-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T06:58:47.770 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T06:58:47.960 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T06:58:47.978 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../048-ceph-mgr_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:48.018 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:48.090 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T06:58:48.106 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../046-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T06:58:48.149 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T06:58:48.496 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T06:58:48.514 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../049-ceph-mon_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:48.553 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T06:58:48.554 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mon (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:48.569 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../047-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T06:58:48.602 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T06:58:48.968 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T06:58:48.985 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../048-ceph-mgr_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:49.051 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:49.588 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T06:58:49.605 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../049-ceph-mon_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:49.646 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mon (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:50.234 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T06:58:50.251 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../050-ceph-osd_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:50.292 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-osd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:51.449 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T06:58:51.466 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../050-ceph-osd_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:51.508 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-osd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:52.437 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph. 2024-09-13T06:58:52.453 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../051-ceph_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:52.495 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:52.864 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T06:58:52.882 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../052-ceph-fuse_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:52.923 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:53.334 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T06:58:53.351 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../053-ceph-mds_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:53.392 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mds (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:53.786 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph. 2024-09-13T06:58:53.803 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../051-ceph_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:53.844 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:53.912 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package cephadm. 2024-09-13T06:58:53.930 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../054-cephadm_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:53.970 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:54.197 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T06:58:54.213 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../052-ceph-fuse_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:54.255 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:54.306 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T06:58:54.326 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../055-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T06:58:54.364 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T06:58:54.660 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T06:58:54.675 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../053-ceph-mds_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:54.709 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mds (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:54.717 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T06:58:54.737 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T06:58:54.775 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T06:58:55.187 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T06:58:55.204 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:58:55.245 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:55.262 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package cephadm. 2024-09-13T06:58:55.278 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../054-cephadm_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:58:55.320 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:55.607 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T06:58:55.626 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../058-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T06:58:55.665 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T06:58:55.673 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T06:58:55.692 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../055-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T06:58:55.740 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T06:58:55.976 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-attr. 2024-09-13T06:58:55.995 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../059-python3-attr_19.3.0-2_all.deb ... 2024-09-13T06:58:56.035 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T06:58:56.176 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T06:58:56.193 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T06:58:56.234 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T06:58:56.404 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T06:58:56.420 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../060-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T06:58:56.462 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T06:58:56.596 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T06:58:56.616 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:58:56.671 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:58:56.782 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-py. 2024-09-13T06:58:56.801 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../061-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T06:58:56.840 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T06:58:57.041 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T06:58:57.058 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../058-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T06:58:57.099 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T06:58:57.178 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T06:58:57.197 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../062-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T06:58:57.236 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T06:58:57.410 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-attr. 2024-09-13T06:58:57.427 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../059-python3-attr_19.3.0-2_all.deb ... 2024-09-13T06:58:57.469 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T06:58:57.522 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T06:58:57.542 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../063-python3-packaging_20.3-1_all.deb ... 2024-09-13T06:58:57.580 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T06:58:57.838 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T06:58:57.855 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../060-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T06:58:57.905 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T06:58:57.916 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T06:58:57.933 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../064-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T06:58:57.975 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T06:58:58.250 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-py. 2024-09-13T06:58:58.261 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T06:58:58.266 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../061-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T06:58:58.278 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../065-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T06:58:58.316 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T06:58:58.344 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T06:58:58.661 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T06:58:58.680 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../062-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T06:58:58.681 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T06:58:58.699 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../066-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T06:58:58.719 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T06:58:58.730 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T06:58:58.975 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T06:58:58.991 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../067-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T06:58:59.030 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T06:58:59.047 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../063-python3-packaging_20.3-1_all.deb ... 2024-09-13T06:58:59.049 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T06:58:59.089 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T06:58:59.335 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T06:58:59.354 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../068-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T06:58:59.394 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T06:58:59.450 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T06:58:59.466 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../064-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T06:58:59.508 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T06:58:59.755 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-routes. 2024-09-13T06:58:59.771 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../069-python3-routes_2.4.1-2_all.deb ... 2024-09-13T06:58:59.813 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T06:58:59.861 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T06:58:59.880 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../065-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T06:58:59.918 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T06:59:00.141 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T06:59:00.157 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../070-ceph-mgr-dashboard_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:59:00.199 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:00.274 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T06:59:00.292 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../066-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T06:59:00.332 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T06:59:00.593 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T06:59:00.609 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../067-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T06:59:00.651 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T06:59:01.012 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T06:59:01.028 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../068-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T06:59:01.070 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T06:59:01.440 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-routes. 2024-09-13T06:59:01.456 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../069-python3-routes_2.4.1-2_all.deb ... 2024-09-13T06:59:01.498 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T06:59:01.597 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T06:59:01.614 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../071-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T06:59:01.654 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T06:59:01.859 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T06:59:01.875 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../070-ceph-mgr-dashboard_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:59:01.917 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:01.974 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T06:59:01.989 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../072-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T06:59:02.032 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T06:59:02.360 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T06:59:02.378 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../073-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T06:59:02.418 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T06:59:03.608 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T06:59:03.628 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../071-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T06:59:03.666 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T06:59:03.795 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T06:59:03.811 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T06:59:03.853 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T06:59:04.019 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T06:59:04.035 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../072-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T06:59:04.077 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T06:59:04.383 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T06:59:04.402 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../075-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T06:59:04.405 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T06:59:04.422 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../073-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T06:59:04.449 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T06:59:04.530 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T06:59:04.854 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T06:59:04.868 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../076-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T06:59:04.903 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T06:59:05.971 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T06:59:05.988 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:59:06.037 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:06.073 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T06:59:06.092 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T06:59:06.132 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T06:59:07.110 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T06:59:07.126 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../075-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T06:59:07.165 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T06:59:07.168 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T06:59:07.181 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../078-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T06:59:07.215 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T06:59:07.513 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T06:59:07.529 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T06:59:07.571 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T06:59:07.580 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T06:59:07.596 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../076-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T06:59:07.638 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T06:59:07.916 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T06:59:07.932 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T06:59:07.983 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T06:59:08.381 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T06:59:08.396 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../081-python3-rsa_4.0-3_all.deb ... 2024-09-13T06:59:08.438 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T06:59:08.465 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T06:59:08.482 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:59:08.522 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:08.784 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T06:59:08.800 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T06:59:08.841 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T06:59:09.163 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T06:59:09.180 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../083-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T06:59:09.247 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T06:59:09.532 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T06:59:09.551 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../084-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T06:59:09.590 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T06:59:09.676 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T06:59:09.693 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../078-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T06:59:09.734 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T06:59:09.940 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T06:59:09.956 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../085-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T06:59:09.998 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T06:59:10.079 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T06:59:10.099 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T06:59:10.145 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T06:59:10.302 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T06:59:10.321 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../086-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T06:59:10.360 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T06:59:10.549 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T06:59:10.566 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T06:59:10.607 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T06:59:10.688 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T06:59:10.704 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../087-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T06:59:10.746 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T06:59:11.029 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T06:59:11.048 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../081-python3-rsa_4.0-3_all.deb ... 2024-09-13T06:59:11.087 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T06:59:11.329 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T06:59:11.347 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../088-ceph-mgr-k8sevents_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:59:11.386 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:11.412 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T06:59:11.430 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T06:59:11.470 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T06:59:11.714 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T06:59:11.731 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../089-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T06:59:11.772 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T06:59:11.816 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T06:59:11.833 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../083-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T06:59:11.874 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T06:59:12.151 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T06:59:12.168 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../090-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T06:59:12.202 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T06:59:12.209 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T06:59:12.218 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../084-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T06:59:12.260 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T06:59:12.487 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package jq. 2024-09-13T06:59:12.506 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../091-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T06:59:12.545 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T06:59:12.631 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T06:59:12.651 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../085-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T06:59:12.689 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T06:59:12.823 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package socat. 2024-09-13T06:59:12.839 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../092-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T06:59:12.881 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T06:59:12.985 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T06:59:13.001 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../086-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T06:59:13.043 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T06:59:13.176 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T06:59:13.192 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../093-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T06:59:13.242 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T06:59:13.446 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T06:59:13.462 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../087-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T06:59:13.505 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T06:59:13.597 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-test. 2024-09-13T06:59:13.616 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../094-ceph-test_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:59:13.655 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-test (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:14.198 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T06:59:14.216 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../088-ceph-mgr-k8sevents_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:59:14.273 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:14.619 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T06:59:14.637 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../089-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T06:59:14.677 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T06:59:15.115 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T06:59:15.133 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../090-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T06:59:15.173 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T06:59:15.492 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package jq. 2024-09-13T06:59:15.510 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../091-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T06:59:15.550 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T06:59:15.837 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package socat. 2024-09-13T06:59:15.854 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../092-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T06:59:15.903 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T06:59:16.198 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T06:59:16.219 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../093-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T06:59:16.298 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T06:59:16.644 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-test. 2024-09-13T06:59:16.662 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../094-ceph-test_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:59:16.711 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-test (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:17.793 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T06:59:17.813 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../095-ceph-volume_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:59:17.851 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-volume (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:18.174 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T06:59:18.191 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T06:59:18.232 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T06:59:18.728 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T06:59:18.746 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../097-libcephfs-dev_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:59:18.786 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libcephfs-dev (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:19.164 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T06:59:19.182 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T06:59:19.242 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T06:59:19.580 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T06:59:19.596 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T06:59:19.639 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T06:59:20.008 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T06:59:20.024 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T06:59:20.066 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T06:59:21.073 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T06:59:21.090 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../095-ceph-volume_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T06:59:21.123 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-volume (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:21.460 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T06:59:21.477 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T06:59:21.510 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T06:59:21.689 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T06:59:21.711 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T06:59:21.763 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T06:59:22.022 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T06:59:22.041 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../097-libcephfs-dev_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:59:22.080 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libcephfs-dev (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:22.091 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T06:59:22.108 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T06:59:22.149 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T06:59:22.476 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T06:59:22.477 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T06:59:22.494 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T06:59:22.496 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T06:59:22.536 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T06:59:22.552 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T06:59:22.864 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T06:59:22.881 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T06:59:22.884 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T06:59:22.902 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T06:59:22.925 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T06:59:22.942 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T06:59:23.287 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T06:59:23.306 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T06:59:23.345 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T06:59:23.387 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T06:59:23.406 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../105-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T06:59:23.445 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T06:59:23.757 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T06:59:23.775 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../106-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T06:59:23.839 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T06:59:24.159 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T06:59:24.179 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T06:59:24.209 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T06:59:24.554 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T06:59:24.572 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../108-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T06:59:24.612 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T06:59:24.881 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T06:59:24.901 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../109-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T06:59:24.939 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T06:59:25.232 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T06:59:25.249 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T06:59:25.289 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T06:59:25.671 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package radosgw. 2024-09-13T06:59:25.676 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T06:59:25.688 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../110-radosgw_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:59:25.694 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T06:59:25.728 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking radosgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:25.734 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T06:59:26.020 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T06:59:26.040 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T06:59:26.111 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T06:59:26.465 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T06:59:26.482 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T06:59:26.523 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T06:59:27.018 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T06:59:27.037 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../105-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T06:59:27.101 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T06:59:27.124 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T06:59:27.140 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../111-rbd-fuse_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:59:27.215 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking rbd-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:27.422 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T06:59:27.441 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../106-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T06:59:27.480 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T06:59:27.539 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T06:59:27.825 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T06:59:27.837 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T06:59:27.842 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T06:59:27.886 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T06:59:27.981 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T06:59:28.107 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T06:59:28.213 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T06:59:28.232 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../108-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T06:59:28.280 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T06:59:28.300 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:28.566 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T06:59:28.583 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../109-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T06:59:28.624 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T06:59:28.836 INFO:teuthology.orchestra.run.smithi164.stdout:Adding system user cephadm....done 2024-09-13T06:59:28.991 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T06:59:29.117 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T06:59:29.337 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package radosgw. 2024-09-13T06:59:29.354 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../110-radosgw_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:59:29.395 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking radosgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:29.452 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T06:59:29.711 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T06:59:29.837 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T06:59:29.997 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T06:59:30.315 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T06:59:30.742 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T06:59:30.798 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T06:59:30.817 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../111-rbd-fuse_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T06:59:30.856 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking rbd-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:31.018 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T06:59:31.178 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T06:59:31.311 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T06:59:31.514 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T06:59:31.579 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T06:59:31.623 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T06:59:31.782 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T06:59:31.880 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-ceph-argparse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:31.995 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:32.214 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T06:59:32.341 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T06:59:32.579 INFO:teuthology.orchestra.run.smithi118.stdout:Adding system user cephadm....done 2024-09-13T06:59:32.625 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T06:59:32.735 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T06:59:32.887 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T06:59:32.942 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T06:59:33.043 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T06:59:33.361 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T06:59:33.403 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T06:59:33.640 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T06:59:33.679 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T06:59:33.766 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T06:59:33.934 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T06:59:33.973 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T06:59:34.261 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T06:59:34.348 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T06:59:34.649 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T06:59:34.655 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T06:59:34.948 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T06:59:34.984 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T06:59:35.233 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T06:59:35.285 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T06:59:35.518 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T06:59:35.569 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T06:59:35.670 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T06:59:35.796 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T06:59:35.819 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-ceph-argparse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:36.098 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T06:59:36.145 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T06:59:36.271 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T06:59:36.547 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T06:59:36.799 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T06:59:36.840 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T06:59:36.991 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T06:59:37.201 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T06:59:37.586 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T06:59:37.659 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T06:59:37.686 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:37.986 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T06:59:38.013 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T06:59:38.147 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T06:59:38.287 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T06:59:38.507 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T06:59:38.630 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T06:59:38.633 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T06:59:38.759 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T06:59:38.931 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T06:59:39.052 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T06:59:39.290 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T06:59:39.361 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T06:59:39.487 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T06:59:39.583 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T06:59:39.613 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T06:59:39.859 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T06:59:39.982 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T06:59:40.034 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T06:59:40.137 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T06:59:40.274 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T06:59:40.421 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T06:59:40.592 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T06:59:40.718 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T06:59:40.881 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T06:59:41.002 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T06:59:41.249 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T06:59:41.311 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T06:59:41.613 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T06:59:41.635 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T06:59:41.769 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:41.999 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T06:59:42.087 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T06:59:42.255 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T06:59:42.257 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T06:59:42.383 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T06:59:42.598 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T06:59:42.742 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T06:59:42.749 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T06:59:42.875 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T06:59:43.110 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T06:59:43.184 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T06:59:43.261 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T06:59:43.362 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T06:59:43.495 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T06:59:43.621 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T06:59:43.705 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T06:59:43.772 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T06:59:43.990 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T06:59:44.090 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T06:59:44.383 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T06:59:44.667 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T06:59:44.827 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T06:59:45.111 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T06:59:45.404 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T06:59:45.445 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T06:59:45.571 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T06:59:45.746 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T06:59:45.948 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T06:59:46.151 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T06:59:46.248 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T06:59:46.425 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T06:59:46.551 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T06:59:46.552 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T06:59:46.684 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T06:59:46.852 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T06:59:47.002 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T06:59:47.204 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T06:59:47.328 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T06:59:47.355 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T06:59:47.489 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T06:59:47.705 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T06:59:47.849 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T06:59:48.023 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T06:59:48.133 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T06:59:49.053 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T06:59:49.154 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T06:59:49.313 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T06:59:49.439 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T06:59:49.598 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T06:59:49.724 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T06:59:49.740 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T06:59:49.866 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T06:59:50.100 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T06:59:50.176 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T06:59:50.402 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T06:59:50.460 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T06:59:50.712 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T06:59:50.769 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T06:59:50.837 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T06:59:51.114 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librados2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:51.189 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T06:59:51.239 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T06:59:51.390 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librgw2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:51.516 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:51.524 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T06:59:51.728 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libcephfs2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:51.862 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T06:59:51.909 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T06:59:52.164 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libradosstriper1 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:52.285 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T06:59:52.289 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T06:59:52.591 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librbd1 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:52.734 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:52.860 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:53.089 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T06:59:53.089 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T06:59:53.341 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T06:59:53.467 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T06:59:53.544 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libcephfs-dev (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:53.627 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T06:59:53.671 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rados (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:53.753 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T06:59:53.788 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T06:59:54.087 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T06:59:54.213 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T06:59:54.506 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T06:59:54.709 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rbd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:54.832 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T06:59:54.860 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up rbd-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:54.986 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:55.120 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cephfs (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:55.150 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T06:59:55.246 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:55.503 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librados2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:55.670 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T06:59:55.797 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librgw2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:55.824 INFO:teuthology.orchestra.run.smithi164.stdout:Adding group ceph....done 2024-09-13T06:59:55.923 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:56.147 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libcephfs2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:56.200 INFO:teuthology.orchestra.run.smithi164.stdout:Adding system user ceph....done 2024-09-13T06:59:56.277 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T06:59:56.360 INFO:teuthology.orchestra.run.smithi164.stdout:Setting system user ceph properties....done 2024-09-13T06:59:56.382 INFO:teuthology.orchestra.run.smithi164.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T06:59:56.528 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T06:59:56.570 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libradosstriper1 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:56.738 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T06:59:56.841 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T06:59:57.040 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librbd1 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:57.166 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:57.306 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-test (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:57.342 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:57.432 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up radosgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:57.562 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T06:59:57.562 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T06:59:57.926 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T06:59:57.927 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T06:59:58.045 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libcephfs-dev (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:58.171 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rados (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:58.338 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T06:59:58.358 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-base (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:58.738 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T06:59:59.193 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mds (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:59.284 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rbd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:59.402 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up rbd-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:59.423 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T06:59:59.423 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T06:59:59.528 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:59.654 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cephfs (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:59.822 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T06:59:59.878 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:00.118 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T07:00:00.118 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T07:00:00.426 INFO:teuthology.orchestra.run.smithi118.stdout:Adding group ceph....done 2024-09-13T07:00:00.555 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-osd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:00.786 INFO:teuthology.orchestra.run.smithi118.stdout:Adding system user ceph....done 2024-09-13T07:00:00.987 INFO:teuthology.orchestra.run.smithi118.stdout:Setting system user ceph properties....done 2024-09-13T07:00:01.012 INFO:teuthology.orchestra.run.smithi118.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T07:00:01.050 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T07:00:01.050 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T07:00:01.163 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T07:00:01.456 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:01.480 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T07:00:01.582 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:01.741 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mon (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:01.942 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-test (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:01.962 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T07:00:01.962 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T07:00:02.076 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up radosgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:02.402 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:02.544 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:02.559 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T07:00:02.559 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T07:00:02.679 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:02.822 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-volume (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:03.028 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-base (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:03.183 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T07:00:03.334 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T07:00:03.401 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T07:00:03.652 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T07:00:03.855 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mds (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:04.138 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T07:00:04.138 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T07:00:04.590 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:04.816 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T07:00:04.817 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T07:00:05.267 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-osd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:05.819 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T07:00:05.819 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T07:00:06.277 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:06.395 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:06.505 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T07:00:06.546 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mon (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:06.752 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T07:00:06.752 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T07:00:07.071 INFO:teuthology.orchestra.run.smithi164.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T07:00:07.079 DEBUG:teuthology.parallel:result is None 2024-09-13T07:00:07.189 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:07.324 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:07.450 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:07.667 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-volume (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T07:00:08.282 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T07:00:08.832 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T07:00:09.117 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T07:00:11.953 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T07:00:12.532 INFO:teuthology.orchestra.run.smithi118.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T07:00:12.536 DEBUG:teuthology.parallel:result is None 2024-09-13T07:00:12.537 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T07:00:12.685 DEBUG:teuthology.orchestra.run.smithi118:> dpkg-query -W -f '${Version}' ceph 2024-09-13T07:00:12.707 INFO:teuthology.orchestra.run.smithi118.stdout:18.2.4-820-g48a7de7e-1focal 2024-09-13T07:00:12.708 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820-g48a7de7e-1focal 2024-09-13T07:00:12.708 INFO:teuthology.task.install:The correct ceph version 18.2.4-820-g48a7de7e-1focal is installed. 2024-09-13T07:00:12.710 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T07:00:12.857 DEBUG:teuthology.orchestra.run.smithi164:> dpkg-query -W -f '${Version}' ceph 2024-09-13T07:00:12.881 INFO:teuthology.orchestra.run.smithi164.stdout:18.2.4-820-g48a7de7e-1focal 2024-09-13T07:00:12.881 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820-g48a7de7e-1focal 2024-09-13T07:00:12.881 INFO:teuthology.task.install:The correct ceph version 18.2.4-820-g48a7de7e-1focal is installed. 2024-09-13T07:00:12.883 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T07:00:12.884 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:12.884 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T07:00:12.900 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:12.901 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T07:00:12.939 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T07:00:12.939 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:12.939 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T07:00:12.954 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T07:00:13.014 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:13.014 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T07:00:13.030 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T07:00:13.043 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T07:00:13.043 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:13.043 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T07:00:13.059 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T07:00:13.117 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:13.117 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T07:00:13.134 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T07:00:13.190 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T07:00:13.191 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:13.191 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T07:00:13.207 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T07:00:13.265 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:13.265 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T07:00:13.281 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T07:00:13.339 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T07:00:13.415 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T07:00:13.415 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 777 /var/log/ceph 2024-09-13T07:00:13.417 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /var/log/ceph 2024-09-13T07:00:13.433 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T07:00:13.433 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T07:00:13.476 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T07:00:13.491 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T07:00:13.492 DEBUG:teuthology.orchestra.run.smithi118:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T07:00:13.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T07:00:13.514 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T07:00:13.514 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T07:00:13.514 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900895', 'branch': 'reef', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/rbd_fsx_nocache}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7900895', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 248, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'sleep_before_teardown': 0, 'subset': '116/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880', 'targets': {'smithi118.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi164.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_22:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T07:00:13.515 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T07:00:13.517 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T07:00:13.554 DEBUG:teuthology.orchestra.run.smithi118:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T07:00:13.564 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T07:00:13.578 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:13.578 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T07:00:13.585 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T07:00:13.585 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_1 2024-09-13T07:00:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T07:00:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:00:13.633 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 360 Links: 1 2024-09-13T07:00:13.633 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:00:13.633 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-13 06:56:39.587126684 +0000 2024-09-13T07:00:13.633 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-13 06:56:39.391131199 +0000 2024-09-13T07:00:13.633 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-13 06:56:39.391131199 +0000 2024-09-13T07:00:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2024-09-13T07:00:13.634 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T07:00:13.645 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-13T07:00:13.645 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-13T07:00:13.645 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000306067 s, 1.7 MB/s 2024-09-13T07:00:13.646 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T07:00:13.655 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_2 2024-09-13T07:00:13.705 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T07:00:13.705 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:00:13.705 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 364 Links: 1 2024-09-13T07:00:13.705 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:00:13.708 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-13 06:56:39.975117746 +0000 2024-09-13T07:00:13.708 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-13 06:56:39.775122353 +0000 2024-09-13T07:00:13.708 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-13 06:56:39.775122353 +0000 2024-09-13T07:00:13.708 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2024-09-13T07:00:13.709 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T07:00:13.721 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-13T07:00:13.721 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-13T07:00:13.722 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000292639 s, 1.7 MB/s 2024-09-13T07:00:13.722 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T07:00:13.775 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_3 2024-09-13T07:00:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T07:00:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:00:13.782 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 371 Links: 1 2024-09-13T07:00:13.782 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:00:13.782 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-13 06:56:40.191112769 +0000 2024-09-13T07:00:13.782 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-13 06:56:40.187112861 +0000 2024-09-13T07:00:13.782 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-13 06:56:40.187112861 +0000 2024-09-13T07:00:13.783 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2024-09-13T07:00:13.783 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T07:00:13.795 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-13T07:00:13.795 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-13T07:00:13.795 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000208049 s, 2.5 MB/s 2024-09-13T07:00:13.796 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T07:00:13.804 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_4 2024-09-13T07:00:13.853 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T07:00:13.853 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:00:13.853 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 379 Links: 1 2024-09-13T07:00:13.853 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:00:13.853 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-09-13 06:56:40.855097471 +0000 2024-09-13T07:00:13.853 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-09-13 06:56:40.691101249 +0000 2024-09-13T07:00:13.853 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-09-13 06:56:40.691101249 +0000 2024-09-13T07:00:13.853 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2024-09-13T07:00:13.854 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T07:00:13.868 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-09-13T07:00:13.869 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-09-13T07:00:13.869 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000291577 s, 1.8 MB/s 2024-09-13T07:00:13.870 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T07:00:13.877 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T07:00:13.877 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:13.877 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T07:00:13.884 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T07:00:13.885 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-09-13T07:00:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T07:00:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:00:13.935 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 368 Links: 1 2024-09-13T07:00:13.935 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:00:13.935 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-13 06:56:39.226825998 +0000 2024-09-13T07:00:13.935 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-13 06:56:39.222826090 +0000 2024-09-13T07:00:13.935 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-13 06:56:39.222826090 +0000 2024-09-13T07:00:13.936 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2024-09-13T07:00:13.936 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T07:00:13.946 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-13T07:00:13.947 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-13T07:00:13.947 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000353839 s, 1.4 MB/s 2024-09-13T07:00:13.948 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T07:00:13.955 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-09-13T07:00:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T07:00:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:00:14.003 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 377 Links: 1 2024-09-13T07:00:14.003 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:00:14.003 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-13 06:56:39.862811338 +0000 2024-09-13T07:00:14.003 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-13 06:56:39.654816132 +0000 2024-09-13T07:00:14.003 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-13 06:56:39.654816132 +0000 2024-09-13T07:00:14.003 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2024-09-13T07:00:14.003 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T07:00:14.016 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-13T07:00:14.016 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-13T07:00:14.017 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000399728 s, 1.3 MB/s 2024-09-13T07:00:14.017 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T07:00:14.026 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-09-13T07:00:14.079 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T07:00:14.079 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:00:14.079 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 384 Links: 1 2024-09-13T07:00:14.079 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:00:14.079 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-13 06:56:40.254802302 +0000 2024-09-13T07:00:14.079 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-13 06:56:40.250802395 +0000 2024-09-13T07:00:14.079 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-13 06:56:40.250802395 +0000 2024-09-13T07:00:14.079 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2024-09-13T07:00:14.080 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T07:00:14.091 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-13T07:00:14.091 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-13T07:00:14.091 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000402208 s, 1.3 MB/s 2024-09-13T07:00:14.093 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T07:00:14.102 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-09-13T07:00:14.153 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T07:00:14.153 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T07:00:14.153 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 387 Links: 1 2024-09-13T07:00:14.153 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T07:00:14.153 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-13 06:56:40.878787919 +0000 2024-09-13T07:00:14.153 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-13 06:56:40.658792990 +0000 2024-09-13T07:00:14.153 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-13 06:56:40.658792990 +0000 2024-09-13T07:00:14.153 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2024-09-13T07:00:14.153 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T07:00:14.168 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-13T07:00:14.168 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-13T07:00:14.168 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000350377 s, 1.5 MB/s 2024-09-13T07:00:14.169 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T07:00:14.221 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T07:00:14.221 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi118.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T07:00:14.221 INFO:tasks.ceph:Generating config... 2024-09-13T07:00:14.222 INFO:tasks.ceph:[client] rbd cache = False 2024-09-13T07:00:14.222 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-13T07:00:14.222 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-13T07:00:14.222 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-13T07:00:14.222 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T07:00:14.223 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T07:00:14.223 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T07:00:14.223 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T07:00:14.223 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T07:00:14.223 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-13T07:00:14.223 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-13T07:00:14.223 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-13T07:00:14.223 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T07:00:14.223 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T07:00:14.223 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T07:00:14.223 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T07:00:14.223 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T07:00:14.223 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T07:00:14.223 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T07:00:14.224 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T07:00:14.224 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T07:00:14.224 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T07:00:14.224 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-13T07:00:14.224 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T07:00:14.224 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-13T07:00:14.224 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T07:00:14.224 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-13T07:00:14.224 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T07:00:14.224 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T07:00:14.266 INFO:teuthology.orchestra.run.smithi118.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T07:00:14.285 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T07:00:14.328 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T07:00:14.342 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.118'), ('mon.c', '[v2:172.21.15.118:3301,v1:172.21.15.118:6790]'), ('mon.b', '172.21.15.164')] 2024-09-13T07:00:14.343 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790],172.21.15.164', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': False}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-13T07:00:14.343 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:14.344 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T07:00:14.349 DEBUG:teuthology.orchestra.run.smithi118:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.118 --addv c '[v2:172.21.15.118:3301,v1:172.21.15.118:6790]' --add b 172.21.15.164 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:14.437 INFO:teuthology.orchestra.run.smithi118.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:14.437 INFO:teuthology.orchestra.run.smithi118.stdout:monmaptool: generated fsid 25bb30d1-9541-4e40-8659-18d83bb1bd8a 2024-09-13T07:00:14.438 INFO:teuthology.orchestra.run.smithi118.stdout:setting min_mon_release = pacific 2024-09-13T07:00:14.438 INFO:teuthology.orchestra.run.smithi118.stdout:epoch 0 2024-09-13T07:00:14.438 INFO:teuthology.orchestra.run.smithi118.stdout:fsid 25bb30d1-9541-4e40-8659-18d83bb1bd8a 2024-09-13T07:00:14.438 INFO:teuthology.orchestra.run.smithi118.stdout:last_changed 2024-09-13T07:00:14.438210+0000 2024-09-13T07:00:14.438 INFO:teuthology.orchestra.run.smithi118.stdout:created 2024-09-13T07:00:14.438210+0000 2024-09-13T07:00:14.438 INFO:teuthology.orchestra.run.smithi118.stdout:min_mon_release 16 (pacific) 2024-09-13T07:00:14.438 INFO:teuthology.orchestra.run.smithi118.stdout:election_strategy: 1 2024-09-13T07:00:14.438 INFO:teuthology.orchestra.run.smithi118.stdout:0: [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon.a 2024-09-13T07:00:14.438 INFO:teuthology.orchestra.run.smithi118.stdout:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-09-13T07:00:14.438 INFO:teuthology.orchestra.run.smithi118.stdout:2: [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] mon.c 2024-09-13T07:00:14.438 INFO:teuthology.orchestra.run.smithi118.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T07:00:14.440 DEBUG:teuthology.orchestra.run.smithi118:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T07:00:14.446 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 25bb30d1-9541-4e40-8659-18d83bb1bd8a... 2024-09-13T07:00:14.447 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T07:00:14.449 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T07:00:14.481 INFO:teuthology.orchestra.run.smithi118.stdout:[global] 2024-09-13T07:00:14.481 INFO:teuthology.orchestra.run.smithi118.stdout: chdir = "" 2024-09-13T07:00:14.481 INFO:teuthology.orchestra.run.smithi118.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T07:00:14.481 INFO:teuthology.orchestra.run.smithi118.stdout: auth supported = cephx 2024-09-13T07:00:14.481 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.481 INFO:teuthology.orchestra.run.smithi118.stdout: filestore xattr use omap = true 2024-09-13T07:00:14.481 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.481 INFO:teuthology.orchestra.run.smithi118.stdout: mon clock drift allowed = 1.000 2024-09-13T07:00:14.481 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.482 INFO:teuthology.orchestra.run.smithi118.stdout: osd crush chooseleaf type = 0 2024-09-13T07:00:14.482 INFO:teuthology.orchestra.run.smithi118.stdout: auth debug = true 2024-09-13T07:00:14.482 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.482 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on old message = true 2024-09-13T07:00:14.482 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on bug = true 2024-09-13T07:00:14.482 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.482 INFO:teuthology.orchestra.run.smithi118.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T07:00:14.482 INFO:teuthology.orchestra.run.smithi118.stdout: mon pg warn max object skew = 0 2024-09-13T07:00:14.482 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.482 INFO:teuthology.orchestra.run.smithi118.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T07:00:14.482 INFO:teuthology.orchestra.run.smithi118.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T07:00:14.482 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.482 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default size = 2 2024-09-13T07:00:14.483 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.483 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow primary affinity = true 2024-09-13T07:00:14.483 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow pg remap = true 2024-09-13T07:00:14.483 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on legacy crush tunables = false 2024-09-13T07:00:14.483 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on crush straw calc version zero = false 2024-09-13T07:00:14.483 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on no sortbitwise = false 2024-09-13T07:00:14.483 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on osd down out interval zero = false 2024-09-13T07:00:14.483 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on too few osds = false 2024-09-13T07:00:14.483 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T07:00:14.483 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T07:00:14.483 INFO:teuthology.orchestra.run.smithi118.stdout: mon_allow_pool_size_one = true 2024-09-13T07:00:14.483 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.484 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T07:00:14.484 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.484 INFO:teuthology.orchestra.run.smithi118.stdout: osd default data pool replay window = 5 2024-09-13T07:00:14.484 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.484 INFO:teuthology.orchestra.run.smithi118.stdout: mon allow pool delete = true 2024-09-13T07:00:14.484 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.484 INFO:teuthology.orchestra.run.smithi118.stdout: mon cluster log file level = debug 2024-09-13T07:00:14.484 INFO:teuthology.orchestra.run.smithi118.stdout: debug asserts on shutdown = true 2024-09-13T07:00:14.484 INFO:teuthology.orchestra.run.smithi118.stdout: mon health detail to clog = false 2024-09-13T07:00:14.484 INFO:teuthology.orchestra.run.smithi118.stdout: mon host = "172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790],172.21.15.164" 2024-09-13T07:00:14.484 INFO:teuthology.orchestra.run.smithi118.stdout: mon client directed command retry = 5 2024-09-13T07:00:14.485 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on pool no app = False 2024-09-13T07:00:14.485 INFO:teuthology.orchestra.run.smithi118.stdout: ms inject socket failures = 5000 2024-09-13T07:00:14.485 INFO:teuthology.orchestra.run.smithi118.stdout: fsid = 25bb30d1-9541-4e40-8659-18d83bb1bd8a 2024-09-13T07:00:14.485 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.485 INFO:teuthology.orchestra.run.smithi118.stdout:[osd] 2024-09-13T07:00:14.485 INFO:teuthology.orchestra.run.smithi118.stdout: osd journal size = 100 2024-09-13T07:00:14.485 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.485 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub load threshold = 5.0 2024-09-13T07:00:14.485 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub max interval = 600 2024-09-13T07:00:14.485 INFO:teuthology.orchestra.run.smithi118.stdout: osd mclock profile = high_recovery_ops 2024-09-13T07:00:14.485 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.485 INFO:teuthology.orchestra.run.smithi118.stdout: osd recover clone overlap = true 2024-09-13T07:00:14.485 INFO:teuthology.orchestra.run.smithi118.stdout: osd recovery max chunk = 1048576 2024-09-13T07:00:14.486 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.486 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug shutdown = true 2024-09-13T07:00:14.486 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug op order = true 2024-09-13T07:00:14.486 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug verify stray on activate = true 2024-09-13T07:00:14.486 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.486 INFO:teuthology.orchestra.run.smithi118.stdout: osd open classes on start = true 2024-09-13T07:00:14.486 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug pg log writeout = true 2024-09-13T07:00:14.486 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.486 INFO:teuthology.orchestra.run.smithi118.stdout: osd deep scrub update digest min age = 30 2024-09-13T07:00:14.486 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.486 INFO:teuthology.orchestra.run.smithi118.stdout: osd map max advance = 10 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: journal zero on create = true 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: filestore ondisk finisher threads = 3 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: filestore apply finisher threads = 3 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: bdev debug aio = true 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug misdirected ops = true 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: bdev async discard = True 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: bdev enable discard = True 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore allocator = bitmap 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore block size = 96636764160 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore fsck on mount = True 2024-09-13T07:00:14.487 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluefs = 1/20 2024-09-13T07:00:14.488 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluestore = 1/20 2024-09-13T07:00:14.488 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2024-09-13T07:00:14.488 INFO:teuthology.orchestra.run.smithi118.stdout: debug osd = 20 2024-09-13T07:00:14.488 INFO:teuthology.orchestra.run.smithi118.stdout: debug rocksdb = 4/10 2024-09-13T07:00:14.488 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T07:00:14.488 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd full ratio = 0.9 2024-09-13T07:00:14.488 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd nearfull ratio = 0.8 2024-09-13T07:00:14.488 INFO:teuthology.orchestra.run.smithi118.stdout: osd blocked scrub grace period = 3600 2024-09-13T07:00:14.489 INFO:teuthology.orchestra.run.smithi118.stdout: osd failsafe full ratio = 0.95 2024-09-13T07:00:14.489 INFO:teuthology.orchestra.run.smithi118.stdout: osd max markdown count = 1000 2024-09-13T07:00:14.489 INFO:teuthology.orchestra.run.smithi118.stdout: osd objectstore = bluestore 2024-09-13T07:00:14.489 INFO:teuthology.orchestra.run.smithi118.stdout: osd shutdown pgref assert = True 2024-09-13T07:00:14.489 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.489 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr] 2024-09-13T07:00:14.489 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2024-09-13T07:00:14.489 INFO:teuthology.orchestra.run.smithi118.stdout: debug mgr = 20 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout: mgr/telemetry/nag = false 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout:[mon] 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout: debug paxos = 20 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout: mon data avail warn = 5 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout: mon mgr mkfs grace = 240 2024-09-13T07:00:14.490 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd reporter subtree level = osd 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd prime pg temp = true 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: # 1m isn't quite enough 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: mon_down_mkfs_grace = 2m 2024-09-13T07:00:14.491 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.492 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_filestore_osds = false 2024-09-13T07:00:14.492 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:14.492 INFO:teuthology.orchestra.run.smithi118.stdout:[client] 2024-09-13T07:00:14.492 INFO:teuthology.orchestra.run.smithi118.stdout: rgw cache enabled = true 2024-09-13T07:00:14.492 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable ops log = true 2024-09-13T07:00:14.492 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable usage log = true 2024-09-13T07:00:14.492 INFO:teuthology.orchestra.run.smithi118.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T07:00:14.492 INFO:teuthology.orchestra.run.smithi118.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T07:00:14.492 INFO:teuthology.orchestra.run.smithi118.stdout: rbd cache = False 2024-09-13T07:00:14.492 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.a] 2024-09-13T07:00:14.492 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.c] 2024-09-13T07:00:14.492 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.b] 2024-09-13T07:00:14.494 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2024-09-13T07:00:14.494 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2024-09-13T07:00:14.494 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T07:00:14.494 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2024-09-13T07:00:14.494 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.494 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2024-09-13T07:00:14.494 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.494 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2024-09-13T07:00:14.495 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.495 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2024-09-13T07:00:14.495 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2024-09-13T07:00:14.495 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.495 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2024-09-13T07:00:14.495 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2024-09-13T07:00:14.495 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.495 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T07:00:14.495 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2024-09-13T07:00:14.495 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.495 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T07:00:14.495 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T07:00:14.495 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2024-09-13T07:00:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2024-09-13T07:00:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2024-09-13T07:00:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2024-09-13T07:00:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2024-09-13T07:00:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2024-09-13T07:00:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2024-09-13T07:00:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2024-09-13T07:00:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T07:00:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T07:00:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: mon_allow_pool_size_one = true 2024-09-13T07:00:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790],172.21.15.164" 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: mon client directed command retry = 5 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on pool no app = False 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: ms inject socket failures = 5000 2024-09-13T07:00:14.497 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 25bb30d1-9541-4e40-8659-18d83bb1bd8a 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout: osd mclock profile = high_recovery_ops 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2024-09-13T07:00:14.498 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.499 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: bdev async discard = True 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: bdev enable discard = True 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore allocator = bitmap 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 1/20 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 1/20 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 4/10 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2024-09-13T07:00:14.500 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout: osd blocked scrub grace period = 3600 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout: osd max markdown count = 1000 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout: osd shutdown pgref assert = True 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-09-13T07:00:14.501 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T07:00:14.502 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T07:00:14.503 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T07:00:14.503 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.503 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T07:00:14.503 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T07:00:14.503 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T07:00:14.503 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.503 INFO:teuthology.orchestra.run.smithi164.stdout: # 1m isn't quite enough 2024-09-13T07:00:14.503 INFO:teuthology.orchestra.run.smithi164.stdout: mon_down_mkfs_grace = 2m 2024-09-13T07:00:14.503 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.503 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_filestore_osds = false 2024-09-13T07:00:14.503 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-13T07:00:14.503 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2024-09-13T07:00:14.503 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2024-09-13T07:00:14.504 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2024-09-13T07:00:14.504 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2024-09-13T07:00:14.504 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T07:00:14.504 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T07:00:14.504 INFO:teuthology.orchestra.run.smithi164.stdout: rbd cache = False 2024-09-13T07:00:14.504 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2024-09-13T07:00:14.504 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2024-09-13T07:00:14.504 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2024-09-13T07:00:14.504 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T07:00:14.504 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T07:00:14.559 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T07:00:14.560 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:14.560 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T07:00:14.567 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:14.567 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T07:00:14.573 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi118.front.sepia.ceph.com 2024-09-13T07:00:14.574 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:14.574 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T07:00:14.574 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T07:00:14.597 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:14.597 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:14.603 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi164.front.sepia.ceph.com 2024-09-13T07:00:14.603 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:14.603 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T07:00:14.603 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T07:00:14.630 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:14.630 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:14.635 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T07:00:14.635 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T07:00:14.635 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T07:00:14.694 INFO:teuthology.orchestra.run.smithi118.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T07:00:14.697 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T07:00:14.749 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T07:00:14.763 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T07:00:14.763 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T07:00:14.763 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T07:00:14.805 INFO:teuthology.orchestra.run.smithi118.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T07:00:14.819 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T07:00:14.862 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-13T07:00:14.876 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T07:00:14.877 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi118.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T07:00:14.877 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T07:00:14.894 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T07:00:14.894 INFO:tasks.ceph:role: osd.0 2024-09-13T07:00:14.894 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-13T07:00:14.894 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T07:00:14.936 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:00:14.936 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:00:14.936 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:00:14.936 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 2024-09-13T07:00:14.937 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:00:14.937 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:00:14.937 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:00:14.937 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T07:00:14.937 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:00:14.937 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:00:14.938 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2024-09-13T07:00:14.938 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T07:00:15.004 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T07:00:15.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:00:15.015 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T07:00:15.015 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T07:00:15.027 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T07:00:15.027 INFO:tasks.ceph:role: osd.1 2024-09-13T07:00:15.027 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-13T07:00:15.027 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T07:00:15.068 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:00:15.069 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:00:15.069 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:00:15.069 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 2024-09-13T07:00:15.069 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:00:15.069 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:00:15.069 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:00:15.069 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T07:00:15.069 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:00:15.069 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:00:15.070 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2024-09-13T07:00:15.070 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T07:00:15.090 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T07:00:15.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:00:15.147 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T07:00:15.148 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T07:00:15.159 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T07:00:15.159 INFO:tasks.ceph:role: osd.2 2024-09-13T07:00:15.159 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-13T07:00:15.159 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T07:00:15.200 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:00:15.200 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:00:15.200 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:00:15.200 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 2024-09-13T07:00:15.201 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:00:15.201 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:00:15.201 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:00:15.201 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T07:00:15.201 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:00:15.201 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:00:15.202 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2024-09-13T07:00:15.202 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T07:00:15.218 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T07:00:15.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:00:15.273 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T07:00:15.273 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T07:00:15.283 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T07:00:15.283 INFO:tasks.ceph:role: osd.3 2024-09-13T07:00:15.284 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-13T07:00:15.284 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T07:00:15.323 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:00:15.323 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:00:15.323 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:00:15.323 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 2024-09-13T07:00:15.323 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:00:15.323 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:00:15.324 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:00:15.324 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T07:00:15.324 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:00:15.324 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:00:15.325 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2024-09-13T07:00:15.325 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T07:00:15.386 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T07:00:15.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:00:15.396 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T07:00:15.397 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:15.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:15.466+0000 7fbd26e46880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T07:00:15.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:15.466+0000 7fbd26e46880 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T07:00:15.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:15.466+0000 7fbd26e46880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T07:00:15.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:15.466+0000 7fbd26e46880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T07:00:15.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:15.466+0000 7fbd26e46880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T07:00:15.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:15.466+0000 7fbd26e46880 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T07:00:18.478 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T07:00:18.496 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:18.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:18.538+0000 7f07eaf29880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T07:00:18.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:18.538+0000 7f07eaf29880 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T07:00:18.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:18.538+0000 7f07eaf29880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T07:00:18.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:18.538+0000 7f07eaf29880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T07:00:18.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:18.538+0000 7f07eaf29880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T07:00:18.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:18.538+0000 7f07eaf29880 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T07:00:21.544 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T07:00:21.560 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:21.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:21.598+0000 7fac47ed0880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T07:00:21.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:21.598+0000 7fac47ed0880 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T07:00:21.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:21.598+0000 7fac47ed0880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T07:00:21.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:21.598+0000 7fac47ed0880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T07:00:21.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:21.598+0000 7fac47ed0880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T07:00:21.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:21.598+0000 7fac47ed0880 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T07:00:24.601 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T07:00:24.619 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:24.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:24.706+0000 7f4c741d5880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T07:00:24.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:24.706+0000 7f4c741d5880 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T07:00:24.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:24.706+0000 7f4c741d5880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T07:00:24.709 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:24.706+0000 7f4c741d5880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T07:00:24.709 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:24.706+0000 7f4c741d5880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T07:00:24.709 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:24.706+0000 7f4c741d5880 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T07:00:27.681 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T07:00:27.696 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T07:00:27.714 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T07:00:27.714 INFO:tasks.ceph:role: osd.4 2024-09-13T07:00:27.714 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-13T07:00:27.714 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T07:00:28.246 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:00:28.246 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:00:28.247 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:00:28.247 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 2024-09-13T07:00:28.247 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:00:28.247 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:00:28.247 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:00:28.247 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T07:00:28.247 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:00:28.247 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:00:28.248 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-09-13T07:00:28.248 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T07:00:28.354 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T07:00:28.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:00:28.367 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T07:00:28.367 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T07:00:28.380 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T07:00:28.380 INFO:tasks.ceph:role: osd.5 2024-09-13T07:00:28.380 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-13T07:00:28.380 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T07:00:28.882 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:00:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:00:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:00:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 2024-09-13T07:00:28.882 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:00:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:00:28.882 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:00:28.882 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T07:00:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:00:28.883 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:00:28.884 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-09-13T07:00:28.884 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T07:00:28.936 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T07:00:28.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:00:28.947 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T07:00:28.947 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T07:00:29.004 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T07:00:29.004 INFO:tasks.ceph:role: osd.6 2024-09-13T07:00:29.004 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-13T07:00:29.004 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T07:00:29.479 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:00:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:00:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:00:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 2024-09-13T07:00:29.480 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:00:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:00:29.480 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:00:29.480 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T07:00:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:00:29.480 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:00:29.481 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-09-13T07:00:29.482 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T07:00:29.541 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T07:00:29.553 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T07:00:29.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:00:29.554 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T07:00:29.609 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T07:00:29.609 INFO:tasks.ceph:role: osd.7 2024-09-13T07:00:29.609 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-13T07:00:29.609 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T07:00:30.131 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T07:00:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T07:00:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T07:00:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 2024-09-13T07:00:30.132 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T07:00:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-09-13T07:00:30.132 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T07:00:30.132 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T07:00:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T07:00:30.132 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T07:00:30.133 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-09-13T07:00:30.133 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T07:00:30.189 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T07:00:30.201 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T07:00:30.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:00:30.202 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:30.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:30.279+0000 7fb3a59f6880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T07:00:30.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:30.283+0000 7fb3a59f6880 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T07:00:30.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:30.283+0000 7fb3a59f6880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T07:00:30.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:30.283+0000 7fb3a59f6880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T07:00:30.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:30.283+0000 7fb3a59f6880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T07:00:30.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:30.287+0000 7fb3a59f6880 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T07:00:33.297 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T07:00:33.314 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:33.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:33.398+0000 7f4260d51880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T07:00:33.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:33.398+0000 7f4260d51880 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T07:00:33.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:33.398+0000 7f4260d51880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T07:00:33.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:33.398+0000 7f4260d51880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T07:00:33.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:33.398+0000 7f4260d51880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T07:00:33.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:33.402+0000 7f4260d51880 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T07:00:36.407 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T07:00:36.421 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:36.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:36.462+0000 7f5f53eef880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T07:00:36.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:36.462+0000 7f5f53eef880 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T07:00:36.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:36.462+0000 7f5f53eef880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T07:00:36.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:36.462+0000 7f5f53eef880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T07:00:36.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:36.462+0000 7f5f53eef880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T07:00:36.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:36.462+0000 7f5f53eef880 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T07:00:39.471 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T07:00:39.488 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:39.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:39.526+0000 7f9136c38880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T07:00:39.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:39.526+0000 7f9136c38880 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T07:00:39.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:39.526+0000 7f9136c38880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T07:00:39.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:39.526+0000 7f9136c38880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T07:00:39.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:39.526+0000 7f9136c38880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T07:00:39.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-13T07:00:39.526+0000 7f9136c38880 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T07:00:42.555 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T07:00:42.572 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T07:00:42.572 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:42.573 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T07:00:42.589 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:42.589 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T07:00:42.645 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:42.645 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T07:00:42.654 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:42.654 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T07:00:42.665 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:42.665 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T07:00:42.721 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:42.722 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T07:00:42.738 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:42.738 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T07:00:42.753 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:42.754 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T07:00:42.811 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:42.811 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T07:00:42.820 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:42.821 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T07:00:42.832 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:42.832 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T07:00:42.839 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:42.839 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-13T07:00:42.844 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T07:00:42.844 DEBUG:teuthology.orchestra.run.smithi118:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T07:00:42.884 DEBUG:teuthology.orchestra.run.smithi164:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T07:00:42.896 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr.y] 2024-09-13T07:00:42.896 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQD+4uNmJ6NxKRAA6iVTAlR/6JUZQYbiGKr06w== 2024-09-13T07:00:42.896 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.0] 2024-09-13T07:00:42.896 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQD/4uNmG+f8GxAA9nXAydY7PcI2qBxxOygSAg== 2024-09-13T07:00:42.896 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.1] 2024-09-13T07:00:42.896 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAC4+NmWalPIBAARhZ/CPCDQ/Llquek7qDjWQ== 2024-09-13T07:00:42.896 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.2] 2024-09-13T07:00:42.897 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAF4+NmGL/NIxAAxDsmGgFx9ySGgcTqkIY3bg== 2024-09-13T07:00:42.897 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.3] 2024-09-13T07:00:42.897 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAI4+NmGSY9KhAA6lB6QiE93XJszopfZnWg+A== 2024-09-13T07:00:42.897 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr.x] 2024-09-13T07:00:42.897 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQD+4uNm6aaqLBAA436QoyEi77klZQQIbPyWag== 2024-09-13T07:00:42.897 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.4] 2024-09-13T07:00:42.897 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAO4+NmNGbREBAA0nTFcCzNsAosKHxKaLKSKw== 2024-09-13T07:00:42.897 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.5] 2024-09-13T07:00:42.897 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAR4+NmSmEOGBAAONP0nhEtdcP0ixExgWypIQ== 2024-09-13T07:00:42.897 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.6] 2024-09-13T07:00:42.897 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAU4+Nm+eK1GxAAO+eC0qN6sItuc8ZgDKsmbQ== 2024-09-13T07:00:42.897 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.7] 2024-09-13T07:00:42.897 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQAX4+NmHTaeHxAAClyyidtGFU0Dyq9ZMDerHw== 2024-09-13T07:00:42.898 INFO:teuthology.orchestra.run.smithi118.stdout:[client.0] 2024-09-13T07:00:42.898 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQD+4uNmP9EOMBAAVPHreQoazWoAJEIj8cMJEQ== 2024-09-13T07:00:42.898 INFO:teuthology.orchestra.run.smithi118.stdout:[client.1] 2024-09-13T07:00:42.898 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQD+4uNm0aNmMxAAXDvzBWdMWgrgRBwe0Wow+A== 2024-09-13T07:00:42.900 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.y] 2024-09-13T07:00:42.900 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQD+4uNmJ6NxKRAA6iVTAlR/6JUZQYbiGKr06w== 2024-09-13T07:00:42.900 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.0] 2024-09-13T07:00:42.900 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQD/4uNmG+f8GxAA9nXAydY7PcI2qBxxOygSAg== 2024-09-13T07:00:42.900 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.1] 2024-09-13T07:00:42.900 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQAC4+NmWalPIBAARhZ/CPCDQ/Llquek7qDjWQ== 2024-09-13T07:00:42.900 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.2] 2024-09-13T07:00:42.900 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQAF4+NmGL/NIxAAxDsmGgFx9ySGgcTqkIY3bg== 2024-09-13T07:00:42.900 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.3] 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQAI4+NmGSY9KhAA6lB6QiE93XJszopfZnWg+A== 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.x] 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQD+4uNm6aaqLBAA436QoyEi77klZQQIbPyWag== 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.4] 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQAO4+NmNGbREBAA0nTFcCzNsAosKHxKaLKSKw== 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.5] 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQAR4+NmSmEOGBAAONP0nhEtdcP0ixExgWypIQ== 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.6] 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQAU4+Nm+eK1GxAAO+eC0qN6sItuc8ZgDKsmbQ== 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.7] 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQAX4+NmHTaeHxAAClyyidtGFU0Dyq9ZMDerHw== 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQD+4uNmP9EOMBAAVPHreQoazWoAJEIj8cMJEQ== 2024-09-13T07:00:42.901 INFO:teuthology.orchestra.run.smithi164.stdout:[client.1] 2024-09-13T07:00:42.902 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQD+4uNm0aNmMxAAXDvzBWdMWgrgRBwe0Wow+A== 2024-09-13T07:00:42.902 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T07:00:42.905 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T07:00:42.951 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:42.954 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.002 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.005 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.053 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.055 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.100 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.103 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.149 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T07:00:43.152 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T07:00:43.199 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.201 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.251 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.254 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.301 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.304 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.398 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.401 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T07:00:43.453 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T07:00:43.455 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T07:00:43.505 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T07:00:43.507 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T07:00:43.559 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T07:00:43.559 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T07:00:43.574 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T07:00:43.652 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T07:00:43.668 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T07:00:43.678 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T07:00:43.769 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T07:00:43.784 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T07:00:43.800 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T07:00:43.894 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T07:00:43.911 DEBUG:teuthology.orchestra.run.smithi118:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:43.914 DEBUG:teuthology.orchestra.run.smithi164:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T07:00:43.918 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T07:00:43.919 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T07:00:43.919 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T07:00:43.960 INFO:tasks.ceph.mon.a:Started 2024-09-13T07:00:43.960 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T07:00:43.960 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T07:00:43.963 INFO:tasks.ceph.mon.c:Started 2024-09-13T07:00:43.963 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T07:00:43.963 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T07:00:43.965 INFO:tasks.ceph.mon.b:Started 2024-09-13T07:00:43.966 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T07:00:43.966 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T07:00:43.966 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T07:00:43.968 INFO:tasks.ceph.mgr.y:Started 2024-09-13T07:00:43.968 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T07:00:43.968 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T07:00:43.971 INFO:tasks.ceph.mgr.x:Started 2024-09-13T07:00:43.971 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T07:00:43.971 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T07:00:49.541 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:49.537+0000 7f438a22b2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T07:00:49.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:00:49.557+0000 7fbdf1d9b700 0 --2- 172.21.15.118:0/3813739784 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7fbde0004870 0x7fbdf41d4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T07:00:49.672 INFO:teuthology.orchestra.run.smithi118.stderr:adjusted tunables profile to default 2024-09-13T07:00:49.685 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T07:00:49.685 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T07:00:49.686 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:49.686 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T07:00:49.701 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:49.701 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T07:00:49.709 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:49.709 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T07:00:49.718 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-09-13T07:00:49.718 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T07:00:49.727 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:49.727 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T07:00:49.744 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:49.744 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T07:00:49.799 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:49.799 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T07:00:49.809 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-13T07:00:49.809 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T07:00:49.824 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 21736eee-00cd-4eef-9725-eca7776957b4 0 2024-09-13T07:00:50.044 INFO:teuthology.orchestra.run.smithi164.stdout:0 2024-09-13T07:00:50.058 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 05818db2-841d-406d-be17-8fbdab4fff69 1 2024-09-13T07:00:50.264 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-09-13T07:00:50.278 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 2387761c-42d2-481b-b5b5-b16d3a943b22 2 2024-09-13T07:00:50.308 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:50.302+0000 7fd723a9a2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T07:00:50.312 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:50.309+0000 7f438a22b2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T07:00:50.369 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:50.365+0000 7f438a22b2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T07:00:50.483 INFO:teuthology.orchestra.run.smithi164.stdout:2 2024-09-13T07:00:50.496 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 56500e90-f76b-47c8-a649-893400359c53 3 2024-09-13T07:00:50.530 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:50.525+0000 7f438a22b2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T07:00:50.704 INFO:teuthology.orchestra.run.smithi164.stdout:3 2024-09-13T07:00:50.718 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new f594a612-aa07-4565-a09e-08368cf397b6 4 2024-09-13T07:00:50.740 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:50.737+0000 7f438a22b2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T07:00:50.790 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:50.785+0000 7f438a22b2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T07:00:50.921 INFO:teuthology.orchestra.run.smithi164.stdout:4 2024-09-13T07:00:50.936 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 1013b81f-e6b4-430b-938d-88759d43b881 5 2024-09-13T07:00:50.960 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:50.957+0000 7f438a22b2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T07:00:51.022 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:51.021+0000 7f438a22b2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T07:00:51.044 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:51.042+0000 7fd723a9a2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T07:00:51.086 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:51.081+0000 7f438a22b2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T07:00:51.105 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:51.102+0000 7fd723a9a2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T07:00:51.142 INFO:teuthology.orchestra.run.smithi164.stdout:5 2024-09-13T07:00:51.157 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 367f5f5b-5919-4534-b88c-229f327e4950 6 2024-09-13T07:00:51.285 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:51.282+0000 7fd723a9a2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T07:00:51.368 INFO:teuthology.orchestra.run.smithi164.stdout:6 2024-09-13T07:00:51.382 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 90d9e486-e1ae-4305-80a6-6f483e534643 7 2024-09-13T07:00:51.485 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:51.481+0000 7f438a22b2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T07:00:51.506 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:51.502+0000 7fd723a9a2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T07:00:51.556 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:51.550+0000 7fd723a9a2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T07:00:51.584 INFO:teuthology.orchestra.run.smithi164.stdout:7 2024-09-13T07:00:51.595 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T07:00:51.595 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T07:00:51.597 INFO:tasks.ceph.osd.0:Started 2024-09-13T07:00:51.597 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T07:00:51.598 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T07:00:51.600 INFO:tasks.ceph.osd.1:Started 2024-09-13T07:00:51.600 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T07:00:51.600 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T07:00:51.602 INFO:tasks.ceph.osd.2:Started 2024-09-13T07:00:51.602 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T07:00:51.602 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T07:00:51.604 INFO:tasks.ceph.osd.3:Started 2024-09-13T07:00:51.604 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T07:00:51.604 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T07:00:51.606 INFO:tasks.ceph.osd.4:Started 2024-09-13T07:00:51.607 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T07:00:51.607 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T07:00:51.609 INFO:tasks.ceph.osd.5:Started 2024-09-13T07:00:51.609 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T07:00:51.609 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T07:00:51.612 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:51.609+0000 7f438a22b2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T07:00:51.613 INFO:tasks.ceph.osd.6:Started 2024-09-13T07:00:51.613 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T07:00:51.613 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T07:00:51.615 INFO:tasks.ceph.osd.7:Started 2024-09-13T07:00:51.615 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T07:00:51.756 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:51.750+0000 7fd723a9a2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T07:00:51.801 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:00:51.802 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":10,"fsid":"25bb30d1-9541-4e40-8659-18d83bb1bd8a","created":"2024-09-13T07:00:49.323883+0000","modified":"2024-09-13T07:00:51.579203+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T07:00:51.579203+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"21736eee-00cd-4eef-9725-eca7776957b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"05818db2-841d-406d-be17-8fbdab4fff69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2387761c-42d2-481b-b5b5-b16d3a943b22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"56500e90-f76b-47c8-a649-893400359c53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f594a612-aa07-4565-a09e-08368cf397b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1013b81f-e6b4-430b-938d-88759d43b881","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"367f5f5b-5919-4534-b88c-229f327e4950","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"90d9e486-e1ae-4305-80a6-6f483e534643","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:00:51.819 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:51.814+0000 7fd723a9a2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T07:00:51.821 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T07:00:51.822 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T07:00:51.883 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:51.878+0000 7fd723a9a2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T07:00:51.960 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:51.957+0000 7f438a22b2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T07:00:51.968 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:00:51.965+0000 7faf999e6880 -1 Falling back to public interface 2024-09-13T07:00:51.977 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:00:51.974+0000 7f0276adf880 -1 Falling back to public interface 2024-09-13T07:00:52.012 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:00:52.009+0000 7fdd88fa6880 -1 Falling back to public interface 2024-09-13T07:00:52.141 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:52.137+0000 7f438a22b2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T07:00:52.208 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:52.205+0000 7f438a22b2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T07:00:52.232 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:00:52.229+0000 7fb49796b880 -1 Falling back to public interface 2024-09-13T07:00:52.232 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:00:52.229+0000 7f4a89157880 -1 Falling back to public interface 2024-09-13T07:00:52.234 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:00:52.230+0000 7f014b748880 -1 Falling back to public interface 2024-09-13T07:00:52.235 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:00:52.230+0000 7f23385fc880 -1 Falling back to public interface 2024-09-13T07:00:52.235 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:00:52.230+0000 7f9dfefad880 -1 Falling back to public interface 2024-09-13T07:00:52.281 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:52.278+0000 7fd723a9a2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T07:00:52.368 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:52.365+0000 7f438a22b2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T07:00:52.399 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:52.394+0000 7fd723a9a2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T07:00:52.627 DEBUG:teuthology.orchestra.run.smithi118:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T07:00:52.731 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:52.726+0000 7fd723a9a2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T07:00:52.804 INFO:teuthology.misc.health.smithi118.stdout: 2024-09-13T07:00:52.805 INFO:teuthology.misc.health.smithi118.stdout:{"epoch":10,"fsid":"25bb30d1-9541-4e40-8659-18d83bb1bd8a","created":"2024-09-13T07:00:49.323883+0000","modified":"2024-09-13T07:00:51.579203+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T07:00:51.579203+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"21736eee-00cd-4eef-9725-eca7776957b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"05818db2-841d-406d-be17-8fbdab4fff69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2387761c-42d2-481b-b5b5-b16d3a943b22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"56500e90-f76b-47c8-a649-893400359c53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f594a612-aa07-4565-a09e-08368cf397b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1013b81f-e6b4-430b-938d-88759d43b881","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"367f5f5b-5919-4534-b88c-229f327e4950","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"90d9e486-e1ae-4305-80a6-6f483e534643","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:00:52.816 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T07:00:52.916 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:52.910+0000 7fd723a9a2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T07:00:52.978 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:52.974+0000 7fd723a9a2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T07:00:53.140 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:53.137+0000 7f438a22b2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T07:00:53.141 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:53.138+0000 7fd723a9a2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T07:00:53.206 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:53.201+0000 7f438a22b2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T07:00:53.357 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:53.353+0000 7f438a22b2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T07:00:53.421 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:53.417+0000 7f438a22b2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T07:00:53.496 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:53.493+0000 7f438a22b2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T07:00:53.810 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:53.809+0000 7f438a22b2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T07:00:53.887 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:53.885+0000 7f438a22b2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T07:00:53.935 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:53.930+0000 7fd723a9a2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T07:00:54.005 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:54.002+0000 7fd723a9a2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T07:00:54.172 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:54.166+0000 7fd723a9a2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T07:00:54.240 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:54.234+0000 7fd723a9a2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T07:00:54.315 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:54.310+0000 7fd723a9a2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T07:00:54.559 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:54.557+0000 7f438a22b2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T07:00:54.643 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:54.638+0000 7fd723a9a2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T07:00:54.721 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:54.718+0000 7fd723a9a2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T07:00:55.460 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-13T07:00:55.454+0000 7fd723a9a2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T07:00:55.616 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:55.613+0000 7f4370da2700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T07:00:55.620 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:00:55.617+0000 7f4370da2700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T07:00:55.822 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:00:55.817+0000 7faf999e6880 -1 osd.3 0 log_to_monitors true 2024-09-13T07:00:55.849 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:00:55.845+0000 7fdd88fa6880 -1 osd.2 0 log_to_monitors true 2024-09-13T07:00:55.860 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:00:55.854+0000 7f0276adf880 -1 osd.5 0 log_to_monitors true 2024-09-13T07:00:55.867 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:00:55.862+0000 7f23385fc880 -1 osd.4 0 log_to_monitors true 2024-09-13T07:00:56.013 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:00:56.010+0000 7f014b748880 -1 osd.6 0 log_to_monitors true 2024-09-13T07:00:56.014 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:00:56.010+0000 7f9dfefad880 -1 osd.7 0 log_to_monitors true 2024-09-13T07:00:56.070 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:00:56.069+0000 7f4a89157880 -1 osd.0 0 log_to_monitors true 2024-09-13T07:00:56.110 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:00:56.109+0000 7fb49796b880 -1 osd.1 0 log_to_monitors true 2024-09-13T07:00:59.141 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:00:59.137+0000 7faf969ad700 -1 osd.3 0 waiting for initial osdmap 2024-09-13T07:00:59.146 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:00:59.145+0000 7faf8d2d7700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:00:59.154 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:00:59.153+0000 7fb492f87700 -1 osd.1 0 waiting for initial osdmap 2024-09-13T07:00:59.160 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:00:59.157+0000 7fb48b25c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:00:59.225 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:00:59.221+0000 7f4a84773700 -1 osd.0 0 waiting for initial osdmap 2024-09-13T07:00:59.229 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:00:59.225+0000 7f4a7ca48700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:00:59.236 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:00:59.233+0000 7fdd845c2700 -1 osd.2 0 waiting for initial osdmap 2024-09-13T07:00:59.240 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:00:59.237+0000 7fdd7c897700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:00:59.623 DEBUG:teuthology.orchestra.run.smithi118:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T07:00:59.712 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:00:59.698+0000 7f0146d64700 -1 osd.6 0 waiting for initial osdmap 2024-09-13T07:00:59.716 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:00:59.710+0000 7f013f039700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:00:59.735 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:00:59.730+0000 7f2333c18700 -1 osd.4 0 waiting for initial osdmap 2024-09-13T07:00:59.740 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:00:59.738+0000 7f232beed700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:00:59.809 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:00:59.806+0000 7f9dfa5c9700 -1 osd.7 0 waiting for initial osdmap 2024-09-13T07:00:59.814 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:00:59.810+0000 7f9df289e700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:00:59.831 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:00:59.826+0000 7f02720fb700 -1 osd.5 0 waiting for initial osdmap 2024-09-13T07:00:59.835 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:00:59.830+0000 7f026a3d0700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:00:59.921 INFO:teuthology.misc.health.smithi118.stdout: 2024-09-13T07:00:59.922 INFO:teuthology.misc.health.smithi118.stdout:{"epoch":12,"fsid":"25bb30d1-9541-4e40-8659-18d83bb1bd8a","created":"2024-09-13T07:00:49.323883+0000","modified":"2024-09-13T07:00:57.649442+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T07:00:51.579203+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"21736eee-00cd-4eef-9725-eca7776957b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"05818db2-841d-406d-be17-8fbdab4fff69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2387761c-42d2-481b-b5b5-b16d3a943b22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"56500e90-f76b-47c8-a649-893400359c53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f594a612-aa07-4565-a09e-08368cf397b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1013b81f-e6b4-430b-938d-88759d43b881","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"367f5f5b-5919-4534-b88c-229f327e4950","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"90d9e486-e1ae-4305-80a6-6f483e534643","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:00:59.939 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T07:01:06.746 DEBUG:teuthology.orchestra.run.smithi118:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T07:01:07.049 INFO:teuthology.misc.health.smithi118.stdout: 2024-09-13T07:01:07.049 INFO:teuthology.misc.health.smithi118.stdout:{"epoch":16,"fsid":"25bb30d1-9541-4e40-8659-18d83bb1bd8a","created":"2024-09-13T07:00:49.323883+0000","modified":"2024-09-13T07:01:03.162929+0000","last_up_change":"2024-09-13T07:01:00.146832+0000","last_in_change":"2024-09-13T07:00:51.579203+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T07:01:00.675365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21736eee-00cd-4eef-9725-eca7776957b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6819","nonce":3731386737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6823","nonce":3731386737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6831","nonce":3731386737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6827","nonce":3731386737}]},"public_addr":"172.21.15.118:6819/3731386737","cluster_addr":"172.21.15.118:6823/3731386737","heartbeat_back_addr":"172.21.15.118:6831/3731386737","heartbeat_front_addr":"172.21.15.118:6827/3731386737","state":["exists","up"]},{"osd":1,"uuid":"05818db2-841d-406d-be17-8fbdab4fff69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6818","nonce":876310814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6822","nonce":876310814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6830","nonce":876310814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6826","nonce":876310814}]},"public_addr":"172.21.15.118:6818/876310814","cluster_addr":"172.21.15.118:6822/876310814","heartbeat_back_addr":"172.21.15.118:6830/876310814","heartbeat_front_addr":"172.21.15.118:6826/876310814","state":["exists","up"]},{"osd":2,"uuid":"2387761c-42d2-481b-b5b5-b16d3a943b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6809","nonce":690188762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6811","nonce":690188762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6815","nonce":690188762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6813","nonce":690188762}]},"public_addr":"172.21.15.118:6809/690188762","cluster_addr":"172.21.15.118:6811/690188762","heartbeat_back_addr":"172.21.15.118:6815/690188762","heartbeat_front_addr":"172.21.15.118:6813/690188762","state":["exists","up"]},{"osd":3,"uuid":"56500e90-f76b-47c8-a649-893400359c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6801","nonce":2198175481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6803","nonce":2198175481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6807","nonce":2198175481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6805","nonce":2198175481}]},"public_addr":"172.21.15.118:6801/2198175481","cluster_addr":"172.21.15.118:6803/2198175481","heartbeat_back_addr":"172.21.15.118:6807/2198175481","heartbeat_front_addr":"172.21.15.118:6805/2198175481","state":["exists","up"]},{"osd":4,"uuid":"f594a612-aa07-4565-a09e-08368cf397b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6813","nonce":1835601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6819","nonce":1835601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6830","nonce":1835601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6825","nonce":1835601651}]},"public_addr":"172.21.15.164:6813/1835601651","cluster_addr":"172.21.15.164:6819/1835601651","heartbeat_back_addr":"172.21.15.164:6830/1835601651","heartbeat_front_addr":"172.21.15.164:6825/1835601651","state":["exists","up"]},{"osd":5,"uuid":"1013b81f-e6b4-430b-938d-88759d43b881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2853696331},{"type":"v1","addr":"172.21.15.164:6801","nonce":2853696331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2853696331},{"type":"v1","addr":"172.21.15.164:6803","nonce":2853696331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2853696331},{"type":"v1","addr":"172.21.15.164:6807","nonce":2853696331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2853696331},{"type":"v1","addr":"172.21.15.164:6805","nonce":2853696331}]},"public_addr":"172.21.15.164:6801/2853696331","cluster_addr":"172.21.15.164:6803/2853696331","heartbeat_back_addr":"172.21.15.164:6807/2853696331","heartbeat_front_addr":"172.21.15.164:6805/2853696331","state":["exists","up"]},{"osd":6,"uuid":"367f5f5b-5919-4534-b88c-229f327e4950","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6812","nonce":131309660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6815","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6818","nonce":131309660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6831","nonce":131309660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6824","nonce":131309660}]},"public_addr":"172.21.15.164:6812/131309660","cluster_addr":"172.21.15.164:6818/131309660","heartbeat_back_addr":"172.21.15.164:6831/131309660","heartbeat_front_addr":"172.21.15.164:6824/131309660","state":["exists","up"]},{"osd":7,"uuid":"90d9e486-e1ae-4305-80a6-6f483e534643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6811","nonce":652173699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6816","nonce":652173699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6829","nonce":652173699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6823","nonce":652173699}]},"public_addr":"172.21.15.164:6811/652173699","cluster_addr":"172.21.15.164:6816/652173699","heartbeat_back_addr":"172.21.15.164:6829/652173699","heartbeat_front_addr":"172.21.15.164:6823/652173699","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.059233+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.083130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.848896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.847456+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.914135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.834284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.003715+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.033514+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:01:07.062 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T07:01:07.062 INFO:tasks.ceph:Creating RBD pool 2024-09-13T07:01:07.063 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-13T07:01:08.204 INFO:teuthology.orchestra.run.smithi118.stderr:pool 'rbd' created 2024-09-13T07:01:08.215 DEBUG:teuthology.orchestra.run.smithi118:> rbd --cluster ceph pool init rbd 2024-09-13T07:01:11.237 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T07:01:11.237 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T07:01:11.237 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T07:01:11.238 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T07:01:11.238 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T07:01:11.597 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:11.613 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":437589447},{"type":"v1","addr":"172.21.15.118:6833","nonce":437589447}]},"active_addr":"172.21.15.118:6833/437589447","active_change":"2024-09-13T07:00:54.582328+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4119,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":933071383}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":2504144367}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":899539299}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":2165840565}]}]} 2024-09-13T07:01:11.617 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T07:01:11.617 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T07:01:11.618 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T07:01:11.923 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:11.923 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":20,"fsid":"25bb30d1-9541-4e40-8659-18d83bb1bd8a","created":"2024-09-13T07:00:49.323883+0000","modified":"2024-09-13T07:01:11.210625+0000","last_up_change":"2024-09-13T07:01:00.146832+0000","last_in_change":"2024-09-13T07:00:51.579203+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T07:01:00.675365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T07:01:07.404263+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21736eee-00cd-4eef-9725-eca7776957b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6819","nonce":3731386737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6823","nonce":3731386737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6831","nonce":3731386737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6827","nonce":3731386737}]},"public_addr":"172.21.15.118:6819/3731386737","cluster_addr":"172.21.15.118:6823/3731386737","heartbeat_back_addr":"172.21.15.118:6831/3731386737","heartbeat_front_addr":"172.21.15.118:6827/3731386737","state":["exists","up"]},{"osd":1,"uuid":"05818db2-841d-406d-be17-8fbdab4fff69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6818","nonce":876310814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6822","nonce":876310814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6830","nonce":876310814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6826","nonce":876310814}]},"public_addr":"172.21.15.118:6818/876310814","cluster_addr":"172.21.15.118:6822/876310814","heartbeat_back_addr":"172.21.15.118:6830/876310814","heartbeat_front_addr":"172.21.15.118:6826/876310814","state":["exists","up"]},{"osd":2,"uuid":"2387761c-42d2-481b-b5b5-b16d3a943b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6809","nonce":690188762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6811","nonce":690188762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6815","nonce":690188762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6813","nonce":690188762}]},"public_addr":"172.21.15.118:6809/690188762","cluster_addr":"172.21.15.118:6811/690188762","heartbeat_back_addr":"172.21.15.118:6815/690188762","heartbeat_front_addr":"172.21.15.118:6813/690188762","state":["exists","up"]},{"osd":3,"uuid":"56500e90-f76b-47c8-a649-893400359c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6801","nonce":2198175481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6803","nonce":2198175481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6807","nonce":2198175481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6805","nonce":2198175481}]},"public_addr":"172.21.15.118:6801/2198175481","cluster_addr":"172.21.15.118:6803/2198175481","heartbeat_back_addr":"172.21.15.118:6807/2198175481","heartbeat_front_addr":"172.21.15.118:6805/2198175481","state":["exists","up"]},{"osd":4,"uuid":"f594a612-aa07-4565-a09e-08368cf397b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6813","nonce":1835601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6819","nonce":1835601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6830","nonce":1835601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6825","nonce":1835601651}]},"public_addr":"172.21.15.164:6813/1835601651","cluster_addr":"172.21.15.164:6819/1835601651","heartbeat_back_addr":"172.21.15.164:6830/1835601651","heartbeat_front_addr":"172.21.15.164:6825/1835601651","state":["exists","up"]},{"osd":5,"uuid":"1013b81f-e6b4-430b-938d-88759d43b881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2853696331},{"type":"v1","addr":"172.21.15.164:6801","nonce":2853696331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2853696331},{"type":"v1","addr":"172.21.15.164:6803","nonce":2853696331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2853696331},{"type":"v1","addr":"172.21.15.164:6807","nonce":2853696331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2853696331},{"type":"v1","addr":"172.21.15.164:6805","nonce":2853696331}]},"public_addr":"172.21.15.164:6801/2853696331","cluster_addr":"172.21.15.164:6803/2853696331","heartbeat_back_addr":"172.21.15.164:6807/2853696331","heartbeat_front_addr":"172.21.15.164:6805/2853696331","state":["exists","up"]},{"osd":6,"uuid":"367f5f5b-5919-4534-b88c-229f327e4950","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6812","nonce":131309660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6815","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6818","nonce":131309660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6831","nonce":131309660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6824","nonce":131309660}]},"public_addr":"172.21.15.164:6812/131309660","cluster_addr":"172.21.15.164:6818/131309660","heartbeat_back_addr":"172.21.15.164:6831/131309660","heartbeat_front_addr":"172.21.15.164:6824/131309660","state":["exists","up"]},{"osd":7,"uuid":"90d9e486-e1ae-4305-80a6-6f483e534643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6811","nonce":652173699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6816","nonce":652173699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6829","nonce":652173699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6823","nonce":652173699}]},"public_addr":"172.21.15.164:6811/652173699","cluster_addr":"172.21.15.164:6816/652173699","heartbeat_back_addr":"172.21.15.164:6829/652173699","heartbeat_front_addr":"172.21.15.164:6823/652173699","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.059233+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.083130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.848896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.847456+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.914135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.834284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.003715+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.033514+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:01:11.938 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T07:01:11.938 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T07:01:12.236 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:12.237 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":20,"fsid":"25bb30d1-9541-4e40-8659-18d83bb1bd8a","created":"2024-09-13T07:00:49.323883+0000","modified":"2024-09-13T07:01:11.210625+0000","last_up_change":"2024-09-13T07:01:00.146832+0000","last_in_change":"2024-09-13T07:00:51.579203+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T07:01:00.675365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T07:01:07.404263+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21736eee-00cd-4eef-9725-eca7776957b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6819","nonce":3731386737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6823","nonce":3731386737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6831","nonce":3731386737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6827","nonce":3731386737}]},"public_addr":"172.21.15.118:6819/3731386737","cluster_addr":"172.21.15.118:6823/3731386737","heartbeat_back_addr":"172.21.15.118:6831/3731386737","heartbeat_front_addr":"172.21.15.118:6827/3731386737","state":["exists","up"]},{"osd":1,"uuid":"05818db2-841d-406d-be17-8fbdab4fff69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6818","nonce":876310814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6822","nonce":876310814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6830","nonce":876310814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6826","nonce":876310814}]},"public_addr":"172.21.15.118:6818/876310814","cluster_addr":"172.21.15.118:6822/876310814","heartbeat_back_addr":"172.21.15.118:6830/876310814","heartbeat_front_addr":"172.21.15.118:6826/876310814","state":["exists","up"]},{"osd":2,"uuid":"2387761c-42d2-481b-b5b5-b16d3a943b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6809","nonce":690188762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6811","nonce":690188762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6815","nonce":690188762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6813","nonce":690188762}]},"public_addr":"172.21.15.118:6809/690188762","cluster_addr":"172.21.15.118:6811/690188762","heartbeat_back_addr":"172.21.15.118:6815/690188762","heartbeat_front_addr":"172.21.15.118:6813/690188762","state":["exists","up"]},{"osd":3,"uuid":"56500e90-f76b-47c8-a649-893400359c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6801","nonce":2198175481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6803","nonce":2198175481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6807","nonce":2198175481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6805","nonce":2198175481}]},"public_addr":"172.21.15.118:6801/2198175481","cluster_addr":"172.21.15.118:6803/2198175481","heartbeat_back_addr":"172.21.15.118:6807/2198175481","heartbeat_front_addr":"172.21.15.118:6805/2198175481","state":["exists","up"]},{"osd":4,"uuid":"f594a612-aa07-4565-a09e-08368cf397b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6813","nonce":1835601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6819","nonce":1835601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6830","nonce":1835601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6825","nonce":1835601651}]},"public_addr":"172.21.15.164:6813/1835601651","cluster_addr":"172.21.15.164:6819/1835601651","heartbeat_back_addr":"172.21.15.164:6830/1835601651","heartbeat_front_addr":"172.21.15.164:6825/1835601651","state":["exists","up"]},{"osd":5,"uuid":"1013b81f-e6b4-430b-938d-88759d43b881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2853696331},{"type":"v1","addr":"172.21.15.164:6801","nonce":2853696331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2853696331},{"type":"v1","addr":"172.21.15.164:6803","nonce":2853696331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2853696331},{"type":"v1","addr":"172.21.15.164:6807","nonce":2853696331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2853696331},{"type":"v1","addr":"172.21.15.164:6805","nonce":2853696331}]},"public_addr":"172.21.15.164:6801/2853696331","cluster_addr":"172.21.15.164:6803/2853696331","heartbeat_back_addr":"172.21.15.164:6807/2853696331","heartbeat_front_addr":"172.21.15.164:6805/2853696331","state":["exists","up"]},{"osd":6,"uuid":"367f5f5b-5919-4534-b88c-229f327e4950","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6812","nonce":131309660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6815","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6818","nonce":131309660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6831","nonce":131309660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6824","nonce":131309660}]},"public_addr":"172.21.15.164:6812/131309660","cluster_addr":"172.21.15.164:6818/131309660","heartbeat_back_addr":"172.21.15.164:6831/131309660","heartbeat_front_addr":"172.21.15.164:6824/131309660","state":["exists","up"]},{"osd":7,"uuid":"90d9e486-e1ae-4305-80a6-6f483e534643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6811","nonce":652173699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6816","nonce":652173699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6829","nonce":652173699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6823","nonce":652173699}]},"public_addr":"172.21.15.164:6811/652173699","cluster_addr":"172.21.15.164:6816/652173699","heartbeat_back_addr":"172.21.15.164:6829/652173699","heartbeat_front_addr":"172.21.15.164:6823/652173699","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.059233+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.083130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.848896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.847456+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.914135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.834284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.003715+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.033514+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:01:12.251 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T07:01:12.252 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T07:01:12.252 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T07:01:12.252 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T07:01:12.253 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T07:01:12.253 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T07:01:12.253 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T07:01:12.254 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T07:01:12.463 INFO:teuthology.orchestra.run.smithi118.stdout:55834574853 2024-09-13T07:01:12.464 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T07:01:12.465 INFO:teuthology.orchestra.run.smithi118.stdout:55834574854 2024-09-13T07:01:12.465 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T07:01:12.466 INFO:teuthology.orchestra.run.smithi118.stdout:55834574854 2024-09-13T07:01:12.467 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T07:01:12.476 INFO:teuthology.orchestra.run.smithi118.stdout:55834574853 2024-09-13T07:01:12.476 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T07:01:12.483 INFO:teuthology.orchestra.run.smithi118.stdout:55834574854 2024-09-13T07:01:12.484 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T07:01:12.484 INFO:teuthology.orchestra.run.smithi118.stdout:55834574853 2024-09-13T07:01:12.485 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T07:01:12.487 INFO:teuthology.orchestra.run.smithi118.stdout:55834574853 2024-09-13T07:01:12.488 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T07:01:12.496 INFO:teuthology.orchestra.run.smithi118.stdout:55834574853 2024-09-13T07:01:12.496 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T07:01:12.941 INFO:teuthology.orchestra.run.smithi118.stdout:55834574851 2024-09-13T07:01:12.946 INFO:teuthology.orchestra.run.smithi118.stdout:55834574852 2024-09-13T07:01:12.960 INFO:teuthology.orchestra.run.smithi118.stdout:55834574852 2024-09-13T07:01:12.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.6 2024-09-13T07:01:12.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.0 2024-09-13T07:01:12.970 INFO:teuthology.orchestra.run.smithi118.stdout:55834574851 2024-09-13T07:01:12.978 INFO:teuthology.orchestra.run.smithi118.stdout:55834574852 2024-09-13T07:01:12.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.1 2024-09-13T07:01:12.983 INFO:teuthology.orchestra.run.smithi118.stdout:55834574851 2024-09-13T07:01:12.988 INFO:teuthology.orchestra.run.smithi118.stdout:55834574851 2024-09-13T07:01:12.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.3 2024-09-13T07:01:12.994 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.7 2024-09-13T07:01:12.998 INFO:teuthology.orchestra.run.smithi118.stdout:55834574851 2024-09-13T07:01:13.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.5 2024-09-13T07:01:13.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.2 2024-09-13T07:01:13.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.4 2024-09-13T07:01:13.967 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T07:01:13.968 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T07:01:13.981 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T07:01:13.989 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T07:01:13.996 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T07:01:14.001 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T07:01:14.001 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T07:01:14.014 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T07:01:14.433 INFO:teuthology.orchestra.run.smithi118.stdout:55834574853 2024-09-13T07:01:14.436 INFO:teuthology.orchestra.run.smithi118.stdout:55834574854 2024-09-13T07:01:14.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2024-09-13T07:01:14.452 DEBUG:teuthology.parallel:result is None 2024-09-13T07:01:14.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-09-13T07:01:14.457 INFO:teuthology.orchestra.run.smithi118.stdout:55834574853 2024-09-13T07:01:14.457 DEBUG:teuthology.parallel:result is None 2024-09-13T07:01:14.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-09-13T07:01:14.473 DEBUG:teuthology.parallel:result is None 2024-09-13T07:01:14.476 INFO:teuthology.orchestra.run.smithi118.stdout:55834574854 2024-09-13T07:01:14.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-09-13T07:01:14.488 DEBUG:teuthology.parallel:result is None 2024-09-13T07:01:14.504 INFO:teuthology.orchestra.run.smithi118.stdout:55834574853 2024-09-13T07:01:14.508 INFO:teuthology.orchestra.run.smithi118.stdout:55834574853 2024-09-13T07:01:14.512 INFO:teuthology.orchestra.run.smithi118.stdout:55834574854 2024-09-13T07:01:14.522 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-09-13T07:01:14.522 DEBUG:teuthology.parallel:result is None 2024-09-13T07:01:14.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-09-13T07:01:14.527 DEBUG:teuthology.parallel:result is None 2024-09-13T07:01:14.528 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-09-13T07:01:14.528 DEBUG:teuthology.parallel:result is None 2024-09-13T07:01:14.543 INFO:teuthology.orchestra.run.smithi118.stdout:55834574853 2024-09-13T07:01:14.560 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2024-09-13T07:01:14.561 DEBUG:teuthology.parallel:result is None 2024-09-13T07:01:14.561 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T07:01:14.561 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:01:14.839 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:14.841 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:01:14.851 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T07:01:14.601311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235324,"kb_used_data":2768,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739396,"statfs":{"total":773094113280,"available":772853141504,"internally_reserved":0,"allocated":2834432,"data_stored":1731654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.399431"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.221334+0000","last_change":"2024-09-13T07:01:11.221939+0000","last_active":"2024-09-13T07:01:11.221334+0000","last_peered":"2024-09-13T07:01:11.221334+0000","last_clean":"2024-09-13T07:01:11.221334+0000","last_became_active":"2024-09-13T07:01:09.212302+0000","last_became_peered":"2024-09-13T07:01:09.212302+0000","last_unstale":"2024-09-13T07:01:11.221334+0000","last_undegraded":"2024-09-13T07:01:11.221334+0000","last_fullsized":"2024-09-13T07:01:11.221334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:49.220240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.231808+0000","last_change":"2024-09-13T07:01:11.232122+0000","last_active":"2024-09-13T07:01:11.231808+0000","last_peered":"2024-09-13T07:01:11.231808+0000","last_clean":"2024-09-13T07:01:11.231808+0000","last_became_active":"2024-09-13T07:01:09.213058+0000","last_became_peered":"2024-09-13T07:01:09.213058+0000","last_unstale":"2024-09-13T07:01:11.231808+0000","last_undegraded":"2024-09-13T07:01:11.231808+0000","last_fullsized":"2024-09-13T07:01:11.231808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:10.839167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067353900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.859538+0000","last_change":"2024-09-13T07:01:11.860064+0000","last_active":"2024-09-13T07:01:11.859538+0000","last_peered":"2024-09-13T07:01:11.859538+0000","last_clean":"2024-09-13T07:01:11.859538+0000","last_became_active":"2024-09-13T07:01:09.521457+0000","last_became_peered":"2024-09-13T07:01:09.521457+0000","last_unstale":"2024-09-13T07:01:11.859538+0000","last_undegraded":"2024-09-13T07:01:11.859538+0000","last_fullsized":"2024-09-13T07:01:11.859538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:39.434445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.231697+0000","last_change":"2024-09-13T07:01:11.232131+0000","last_active":"2024-09-13T07:01:11.231697+0000","last_peered":"2024-09-13T07:01:11.231697+0000","last_clean":"2024-09-13T07:01:11.231697+0000","last_became_active":"2024-09-13T07:01:09.523241+0000","last_became_peered":"2024-09-13T07:01:09.523241+0000","last_unstale":"2024-09-13T07:01:11.231697+0000","last_undegraded":"2024-09-13T07:01:11.231697+0000","last_fullsized":"2024-09-13T07:01:11.231697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:58:22.184544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.228726+0000","last_change":"2024-09-13T07:01:11.228726+0000","last_active":"2024-09-13T07:01:11.228726+0000","last_peered":"2024-09-13T07:01:11.228726+0000","last_clean":"2024-09-13T07:01:11.228726+0000","last_became_active":"2024-09-13T07:01:09.211518+0000","last_became_peered":"2024-09-13T07:01:09.211518+0000","last_unstale":"2024-09-13T07:01:11.228726+0000","last_undegraded":"2024-09-13T07:01:11.228726+0000","last_fullsized":"2024-09-13T07:01:11.228726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:21.453172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.222011+0000","last_change":"2024-09-13T07:01:11.222462+0000","last_active":"2024-09-13T07:01:11.222011+0000","last_peered":"2024-09-13T07:01:11.222011+0000","last_clean":"2024-09-13T07:01:11.222011+0000","last_became_active":"2024-09-13T07:01:09.523133+0000","last_became_peered":"2024-09-13T07:01:09.523133+0000","last_unstale":"2024-09-13T07:01:11.222011+0000","last_undegraded":"2024-09-13T07:01:11.222011+0000","last_fullsized":"2024-09-13T07:01:11.222011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:12:38.585984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074093200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.859629+0000","last_change":"2024-09-13T07:01:11.859918+0000","last_active":"2024-09-13T07:01:11.859629+0000","last_peered":"2024-09-13T07:01:11.859629+0000","last_clean":"2024-09-13T07:01:11.859629+0000","last_became_active":"2024-09-13T07:01:09.210971+0000","last_became_peered":"2024-09-13T07:01:09.210971+0000","last_unstale":"2024-09-13T07:01:11.859629+0000","last_undegraded":"2024-09-13T07:01:11.859629+0000","last_fullsized":"2024-09-13T07:01:11.859629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:30.775691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.227079+0000","last_change":"2024-09-13T07:01:11.227425+0000","last_active":"2024-09-13T07:01:11.227079+0000","last_peered":"2024-09-13T07:01:11.227079+0000","last_clean":"2024-09-13T07:01:11.227079+0000","last_became_active":"2024-09-13T07:01:09.207226+0000","last_became_peered":"2024-09-13T07:01:09.207226+0000","last_unstale":"2024-09-13T07:01:11.227079+0000","last_undegraded":"2024-09-13T07:01:11.227079+0000","last_fullsized":"2024-09-13T07:01:11.227079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:12.911230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T07:01:10.205141+0000","last_change":"2024-09-13T07:01:02.162365+0000","last_active":"2024-09-13T07:01:10.205141+0000","last_peered":"2024-09-13T07:01:10.205141+0000","last_clean":"2024-09-13T07:01:10.205141+0000","last_became_active":"2024-09-13T07:01:02.161407+0000","last_became_peered":"2024-09-13T07:01:02.161407+0000","last_unstale":"2024-09-13T07:01:10.205141+0000","last_undegraded":"2024-09-13T07:01:10.205141+0000","last_fullsized":"2024-09-13T07:01:10.205141+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:17.998339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":659228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3580000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31324,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340516,"statfs":{"total":96636764160,"available":96604688384,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":798720,"data_stored":659228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:01:14.853 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:01:15.139 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:15.142 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:01:15.158 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T07:01:14.601311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235324,"kb_used_data":2768,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739396,"statfs":{"total":773094113280,"available":772853141504,"internally_reserved":0,"allocated":2834432,"data_stored":1731654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.399431"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.221334+0000","last_change":"2024-09-13T07:01:11.221939+0000","last_active":"2024-09-13T07:01:11.221334+0000","last_peered":"2024-09-13T07:01:11.221334+0000","last_clean":"2024-09-13T07:01:11.221334+0000","last_became_active":"2024-09-13T07:01:09.212302+0000","last_became_peered":"2024-09-13T07:01:09.212302+0000","last_unstale":"2024-09-13T07:01:11.221334+0000","last_undegraded":"2024-09-13T07:01:11.221334+0000","last_fullsized":"2024-09-13T07:01:11.221334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:49.220240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.231808+0000","last_change":"2024-09-13T07:01:11.232122+0000","last_active":"2024-09-13T07:01:11.231808+0000","last_peered":"2024-09-13T07:01:11.231808+0000","last_clean":"2024-09-13T07:01:11.231808+0000","last_became_active":"2024-09-13T07:01:09.213058+0000","last_became_peered":"2024-09-13T07:01:09.213058+0000","last_unstale":"2024-09-13T07:01:11.231808+0000","last_undegraded":"2024-09-13T07:01:11.231808+0000","last_fullsized":"2024-09-13T07:01:11.231808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:10.839167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067353900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.859538+0000","last_change":"2024-09-13T07:01:11.860064+0000","last_active":"2024-09-13T07:01:11.859538+0000","last_peered":"2024-09-13T07:01:11.859538+0000","last_clean":"2024-09-13T07:01:11.859538+0000","last_became_active":"2024-09-13T07:01:09.521457+0000","last_became_peered":"2024-09-13T07:01:09.521457+0000","last_unstale":"2024-09-13T07:01:11.859538+0000","last_undegraded":"2024-09-13T07:01:11.859538+0000","last_fullsized":"2024-09-13T07:01:11.859538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:39.434445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.231697+0000","last_change":"2024-09-13T07:01:11.232131+0000","last_active":"2024-09-13T07:01:11.231697+0000","last_peered":"2024-09-13T07:01:11.231697+0000","last_clean":"2024-09-13T07:01:11.231697+0000","last_became_active":"2024-09-13T07:01:09.523241+0000","last_became_peered":"2024-09-13T07:01:09.523241+0000","last_unstale":"2024-09-13T07:01:11.231697+0000","last_undegraded":"2024-09-13T07:01:11.231697+0000","last_fullsized":"2024-09-13T07:01:11.231697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:58:22.184544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.228726+0000","last_change":"2024-09-13T07:01:11.228726+0000","last_active":"2024-09-13T07:01:11.228726+0000","last_peered":"2024-09-13T07:01:11.228726+0000","last_clean":"2024-09-13T07:01:11.228726+0000","last_became_active":"2024-09-13T07:01:09.211518+0000","last_became_peered":"2024-09-13T07:01:09.211518+0000","last_unstale":"2024-09-13T07:01:11.228726+0000","last_undegraded":"2024-09-13T07:01:11.228726+0000","last_fullsized":"2024-09-13T07:01:11.228726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:21.453172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.222011+0000","last_change":"2024-09-13T07:01:11.222462+0000","last_active":"2024-09-13T07:01:11.222011+0000","last_peered":"2024-09-13T07:01:11.222011+0000","last_clean":"2024-09-13T07:01:11.222011+0000","last_became_active":"2024-09-13T07:01:09.523133+0000","last_became_peered":"2024-09-13T07:01:09.523133+0000","last_unstale":"2024-09-13T07:01:11.222011+0000","last_undegraded":"2024-09-13T07:01:11.222011+0000","last_fullsized":"2024-09-13T07:01:11.222011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:12:38.585984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074093200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.859629+0000","last_change":"2024-09-13T07:01:11.859918+0000","last_active":"2024-09-13T07:01:11.859629+0000","last_peered":"2024-09-13T07:01:11.859629+0000","last_clean":"2024-09-13T07:01:11.859629+0000","last_became_active":"2024-09-13T07:01:09.210971+0000","last_became_peered":"2024-09-13T07:01:09.210971+0000","last_unstale":"2024-09-13T07:01:11.859629+0000","last_undegraded":"2024-09-13T07:01:11.859629+0000","last_fullsized":"2024-09-13T07:01:11.859629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:30.775691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.227079+0000","last_change":"2024-09-13T07:01:11.227425+0000","last_active":"2024-09-13T07:01:11.227079+0000","last_peered":"2024-09-13T07:01:11.227079+0000","last_clean":"2024-09-13T07:01:11.227079+0000","last_became_active":"2024-09-13T07:01:09.207226+0000","last_became_peered":"2024-09-13T07:01:09.207226+0000","last_unstale":"2024-09-13T07:01:11.227079+0000","last_undegraded":"2024-09-13T07:01:11.227079+0000","last_fullsized":"2024-09-13T07:01:11.227079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:12.911230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T07:01:10.205141+0000","last_change":"2024-09-13T07:01:02.162365+0000","last_active":"2024-09-13T07:01:10.205141+0000","last_peered":"2024-09-13T07:01:10.205141+0000","last_clean":"2024-09-13T07:01:10.205141+0000","last_became_active":"2024-09-13T07:01:02.161407+0000","last_became_peered":"2024-09-13T07:01:02.161407+0000","last_unstale":"2024-09-13T07:01:10.205141+0000","last_undegraded":"2024-09-13T07:01:10.205141+0000","last_fullsized":"2024-09-13T07:01:10.205141+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:17.998339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":659228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3580000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31324,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340516,"statfs":{"total":96636764160,"available":96604688384,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":798720,"data_stored":659228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:01:15.159 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T07:01:15.160 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T07:01:15.160 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T07:01:15.160 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T07:01:15.476 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:15.476 INFO:teuthology.orchestra.run.smithi118.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T07:01:15.490 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T07:01:15.490 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-13T07:01:15.502 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-13T07:01:15.502 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}} 2024-09-13T07:01:15.502 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-13T07:01:15.502 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-13T07:01:15.808 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-13T07:01:15.809 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T07:01:15.809 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:01:16.099 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:16.102 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:01:16.113 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T07:01:14.601311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235324,"kb_used_data":2768,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739396,"statfs":{"total":773094113280,"available":772853141504,"internally_reserved":0,"allocated":2834432,"data_stored":1731654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.399431"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.221334+0000","last_change":"2024-09-13T07:01:11.221939+0000","last_active":"2024-09-13T07:01:11.221334+0000","last_peered":"2024-09-13T07:01:11.221334+0000","last_clean":"2024-09-13T07:01:11.221334+0000","last_became_active":"2024-09-13T07:01:09.212302+0000","last_became_peered":"2024-09-13T07:01:09.212302+0000","last_unstale":"2024-09-13T07:01:11.221334+0000","last_undegraded":"2024-09-13T07:01:11.221334+0000","last_fullsized":"2024-09-13T07:01:11.221334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:49.220240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.231808+0000","last_change":"2024-09-13T07:01:11.232122+0000","last_active":"2024-09-13T07:01:11.231808+0000","last_peered":"2024-09-13T07:01:11.231808+0000","last_clean":"2024-09-13T07:01:11.231808+0000","last_became_active":"2024-09-13T07:01:09.213058+0000","last_became_peered":"2024-09-13T07:01:09.213058+0000","last_unstale":"2024-09-13T07:01:11.231808+0000","last_undegraded":"2024-09-13T07:01:11.231808+0000","last_fullsized":"2024-09-13T07:01:11.231808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:10.839167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067353900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.859538+0000","last_change":"2024-09-13T07:01:11.860064+0000","last_active":"2024-09-13T07:01:11.859538+0000","last_peered":"2024-09-13T07:01:11.859538+0000","last_clean":"2024-09-13T07:01:11.859538+0000","last_became_active":"2024-09-13T07:01:09.521457+0000","last_became_peered":"2024-09-13T07:01:09.521457+0000","last_unstale":"2024-09-13T07:01:11.859538+0000","last_undegraded":"2024-09-13T07:01:11.859538+0000","last_fullsized":"2024-09-13T07:01:11.859538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:39.434445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.231697+0000","last_change":"2024-09-13T07:01:11.232131+0000","last_active":"2024-09-13T07:01:11.231697+0000","last_peered":"2024-09-13T07:01:11.231697+0000","last_clean":"2024-09-13T07:01:11.231697+0000","last_became_active":"2024-09-13T07:01:09.523241+0000","last_became_peered":"2024-09-13T07:01:09.523241+0000","last_unstale":"2024-09-13T07:01:11.231697+0000","last_undegraded":"2024-09-13T07:01:11.231697+0000","last_fullsized":"2024-09-13T07:01:11.231697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:58:22.184544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.228726+0000","last_change":"2024-09-13T07:01:11.228726+0000","last_active":"2024-09-13T07:01:11.228726+0000","last_peered":"2024-09-13T07:01:11.228726+0000","last_clean":"2024-09-13T07:01:11.228726+0000","last_became_active":"2024-09-13T07:01:09.211518+0000","last_became_peered":"2024-09-13T07:01:09.211518+0000","last_unstale":"2024-09-13T07:01:11.228726+0000","last_undegraded":"2024-09-13T07:01:11.228726+0000","last_fullsized":"2024-09-13T07:01:11.228726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:21.453172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.222011+0000","last_change":"2024-09-13T07:01:11.222462+0000","last_active":"2024-09-13T07:01:11.222011+0000","last_peered":"2024-09-13T07:01:11.222011+0000","last_clean":"2024-09-13T07:01:11.222011+0000","last_became_active":"2024-09-13T07:01:09.523133+0000","last_became_peered":"2024-09-13T07:01:09.523133+0000","last_unstale":"2024-09-13T07:01:11.222011+0000","last_undegraded":"2024-09-13T07:01:11.222011+0000","last_fullsized":"2024-09-13T07:01:11.222011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:12:38.585984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074093200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.859629+0000","last_change":"2024-09-13T07:01:11.859918+0000","last_active":"2024-09-13T07:01:11.859629+0000","last_peered":"2024-09-13T07:01:11.859629+0000","last_clean":"2024-09-13T07:01:11.859629+0000","last_became_active":"2024-09-13T07:01:09.210971+0000","last_became_peered":"2024-09-13T07:01:09.210971+0000","last_unstale":"2024-09-13T07:01:11.859629+0000","last_undegraded":"2024-09-13T07:01:11.859629+0000","last_fullsized":"2024-09-13T07:01:11.859629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:30.775691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.227079+0000","last_change":"2024-09-13T07:01:11.227425+0000","last_active":"2024-09-13T07:01:11.227079+0000","last_peered":"2024-09-13T07:01:11.227079+0000","last_clean":"2024-09-13T07:01:11.227079+0000","last_became_active":"2024-09-13T07:01:09.207226+0000","last_became_peered":"2024-09-13T07:01:09.207226+0000","last_unstale":"2024-09-13T07:01:11.227079+0000","last_undegraded":"2024-09-13T07:01:11.227079+0000","last_fullsized":"2024-09-13T07:01:11.227079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:12.911230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T07:01:10.205141+0000","last_change":"2024-09-13T07:01:02.162365+0000","last_active":"2024-09-13T07:01:10.205141+0000","last_peered":"2024-09-13T07:01:10.205141+0000","last_clean":"2024-09-13T07:01:10.205141+0000","last_became_active":"2024-09-13T07:01:02.161407+0000","last_became_peered":"2024-09-13T07:01:02.161407+0000","last_unstale":"2024-09-13T07:01:10.205141+0000","last_undegraded":"2024-09-13T07:01:10.205141+0000","last_fullsized":"2024-09-13T07:01:10.205141+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:17.998339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":659228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3580000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31324,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340516,"statfs":{"total":96636764160,"available":96604688384,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":798720,"data_stored":659228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:01:16.115 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:01:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:16.420 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:01:16.431 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T07:01:14.601311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235324,"kb_used_data":2768,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739396,"statfs":{"total":773094113280,"available":772853141504,"internally_reserved":0,"allocated":2834432,"data_stored":1731654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.399431"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.221334+0000","last_change":"2024-09-13T07:01:11.221939+0000","last_active":"2024-09-13T07:01:11.221334+0000","last_peered":"2024-09-13T07:01:11.221334+0000","last_clean":"2024-09-13T07:01:11.221334+0000","last_became_active":"2024-09-13T07:01:09.212302+0000","last_became_peered":"2024-09-13T07:01:09.212302+0000","last_unstale":"2024-09-13T07:01:11.221334+0000","last_undegraded":"2024-09-13T07:01:11.221334+0000","last_fullsized":"2024-09-13T07:01:11.221334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:49.220240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.231808+0000","last_change":"2024-09-13T07:01:11.232122+0000","last_active":"2024-09-13T07:01:11.231808+0000","last_peered":"2024-09-13T07:01:11.231808+0000","last_clean":"2024-09-13T07:01:11.231808+0000","last_became_active":"2024-09-13T07:01:09.213058+0000","last_became_peered":"2024-09-13T07:01:09.213058+0000","last_unstale":"2024-09-13T07:01:11.231808+0000","last_undegraded":"2024-09-13T07:01:11.231808+0000","last_fullsized":"2024-09-13T07:01:11.231808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:10.839167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067353900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.859538+0000","last_change":"2024-09-13T07:01:11.860064+0000","last_active":"2024-09-13T07:01:11.859538+0000","last_peered":"2024-09-13T07:01:11.859538+0000","last_clean":"2024-09-13T07:01:11.859538+0000","last_became_active":"2024-09-13T07:01:09.521457+0000","last_became_peered":"2024-09-13T07:01:09.521457+0000","last_unstale":"2024-09-13T07:01:11.859538+0000","last_undegraded":"2024-09-13T07:01:11.859538+0000","last_fullsized":"2024-09-13T07:01:11.859538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:39.434445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.231697+0000","last_change":"2024-09-13T07:01:11.232131+0000","last_active":"2024-09-13T07:01:11.231697+0000","last_peered":"2024-09-13T07:01:11.231697+0000","last_clean":"2024-09-13T07:01:11.231697+0000","last_became_active":"2024-09-13T07:01:09.523241+0000","last_became_peered":"2024-09-13T07:01:09.523241+0000","last_unstale":"2024-09-13T07:01:11.231697+0000","last_undegraded":"2024-09-13T07:01:11.231697+0000","last_fullsized":"2024-09-13T07:01:11.231697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:58:22.184544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.228726+0000","last_change":"2024-09-13T07:01:11.228726+0000","last_active":"2024-09-13T07:01:11.228726+0000","last_peered":"2024-09-13T07:01:11.228726+0000","last_clean":"2024-09-13T07:01:11.228726+0000","last_became_active":"2024-09-13T07:01:09.211518+0000","last_became_peered":"2024-09-13T07:01:09.211518+0000","last_unstale":"2024-09-13T07:01:11.228726+0000","last_undegraded":"2024-09-13T07:01:11.228726+0000","last_fullsized":"2024-09-13T07:01:11.228726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:21.453172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.222011+0000","last_change":"2024-09-13T07:01:11.222462+0000","last_active":"2024-09-13T07:01:11.222011+0000","last_peered":"2024-09-13T07:01:11.222011+0000","last_clean":"2024-09-13T07:01:11.222011+0000","last_became_active":"2024-09-13T07:01:09.523133+0000","last_became_peered":"2024-09-13T07:01:09.523133+0000","last_unstale":"2024-09-13T07:01:11.222011+0000","last_undegraded":"2024-09-13T07:01:11.222011+0000","last_fullsized":"2024-09-13T07:01:11.222011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:12:38.585984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074093200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.859629+0000","last_change":"2024-09-13T07:01:11.859918+0000","last_active":"2024-09-13T07:01:11.859629+0000","last_peered":"2024-09-13T07:01:11.859629+0000","last_clean":"2024-09-13T07:01:11.859629+0000","last_became_active":"2024-09-13T07:01:09.210971+0000","last_became_peered":"2024-09-13T07:01:09.210971+0000","last_unstale":"2024-09-13T07:01:11.859629+0000","last_undegraded":"2024-09-13T07:01:11.859629+0000","last_fullsized":"2024-09-13T07:01:11.859629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:30.775691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T07:01:11.227079+0000","last_change":"2024-09-13T07:01:11.227425+0000","last_active":"2024-09-13T07:01:11.227079+0000","last_peered":"2024-09-13T07:01:11.227079+0000","last_clean":"2024-09-13T07:01:11.227079+0000","last_became_active":"2024-09-13T07:01:09.207226+0000","last_became_peered":"2024-09-13T07:01:09.207226+0000","last_unstale":"2024-09-13T07:01:11.227079+0000","last_undegraded":"2024-09-13T07:01:11.227079+0000","last_fullsized":"2024-09-13T07:01:11.227079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:12.911230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T07:01:10.205141+0000","last_change":"2024-09-13T07:01:02.162365+0000","last_active":"2024-09-13T07:01:10.205141+0000","last_peered":"2024-09-13T07:01:10.205141+0000","last_clean":"2024-09-13T07:01:10.205141+0000","last_became_active":"2024-09-13T07:01:02.161407+0000","last_became_peered":"2024-09-13T07:01:02.161407+0000","last_unstale":"2024-09-13T07:01:10.205141+0000","last_undegraded":"2024-09-13T07:01:10.205141+0000","last_fullsized":"2024-09-13T07:01:10.205141+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:17.998339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":659228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3580000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31324,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340516,"statfs":{"total":96636764160,"available":96604688384,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":798720,"data_stored":659228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:01:16.432 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T07:01:16.432 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-13T07:01:16.734 INFO:teuthology.orchestra.run.smithi118.stdout:epoch 21 2024-09-13T07:01:16.734 INFO:teuthology.orchestra.run.smithi118.stdout:fsid 25bb30d1-9541-4e40-8659-18d83bb1bd8a 2024-09-13T07:01:16.734 INFO:teuthology.orchestra.run.smithi118.stdout:created 2024-09-13T07:00:49.323883+0000 2024-09-13T07:01:16.734 INFO:teuthology.orchestra.run.smithi118.stdout:modified 2024-09-13T07:01:14.060244+0000 2024-09-13T07:01:16.734 INFO:teuthology.orchestra.run.smithi118.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-13T07:01:16.734 INFO:teuthology.orchestra.run.smithi118.stdout:crush_version 4 2024-09-13T07:01:16.734 INFO:teuthology.orchestra.run.smithi118.stdout:full_ratio 0.95 2024-09-13T07:01:16.734 INFO:teuthology.orchestra.run.smithi118.stdout:backfillfull_ratio 0.9 2024-09-13T07:01:16.734 INFO:teuthology.orchestra.run.smithi118.stdout:nearfull_ratio 0.85 2024-09-13T07:01:16.735 INFO:teuthology.orchestra.run.smithi118.stdout:require_min_compat_client luminous 2024-09-13T07:01:16.735 INFO:teuthology.orchestra.run.smithi118.stdout:min_compat_client jewel 2024-09-13T07:01:16.735 INFO:teuthology.orchestra.run.smithi118.stdout:require_osd_release reef 2024-09-13T07:01:16.735 INFO:teuthology.orchestra.run.smithi118.stdout:stretch_mode_enabled false 2024-09-13T07:01:16.735 INFO:teuthology.orchestra.run.smithi118.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-13T07:01:16.735 INFO:teuthology.orchestra.run.smithi118.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 20 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-13T07:01:16.735 INFO:teuthology.orchestra.run.smithi118.stdout:max_osd 8 2024-09-13T07:01:16.735 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6817/3731386737,v1:172.21.15.118:6819/3731386737] [v2:172.21.15.118:6821/3731386737,v1:172.21.15.118:6823/3731386737] exists,up 21736eee-00cd-4eef-9725-eca7776957b4 2024-09-13T07:01:16.735 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6816/876310814,v1:172.21.15.118:6818/876310814] [v2:172.21.15.118:6820/876310814,v1:172.21.15.118:6822/876310814] exists,up 05818db2-841d-406d-be17-8fbdab4fff69 2024-09-13T07:01:16.735 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6808/690188762,v1:172.21.15.118:6809/690188762] [v2:172.21.15.118:6810/690188762,v1:172.21.15.118:6811/690188762] exists,up 2387761c-42d2-481b-b5b5-b16d3a943b22 2024-09-13T07:01:16.735 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6800/2198175481,v1:172.21.15.118:6801/2198175481] [v2:172.21.15.118:6802/2198175481,v1:172.21.15.118:6803/2198175481] exists,up 56500e90-f76b-47c8-a649-893400359c53 2024-09-13T07:01:16.735 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6810/1835601651,v1:172.21.15.164:6813/1835601651] [v2:172.21.15.164:6817/1835601651,v1:172.21.15.164:6819/1835601651] exists,up f594a612-aa07-4565-a09e-08368cf397b6 2024-09-13T07:01:16.735 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/2853696331,v1:172.21.15.164:6801/2853696331] [v2:172.21.15.164:6802/2853696331,v1:172.21.15.164:6803/2853696331] exists,up 1013b81f-e6b4-430b-938d-88759d43b881 2024-09-13T07:01:16.736 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/131309660,v1:172.21.15.164:6812/131309660] [v2:172.21.15.164:6815/131309660,v1:172.21.15.164:6818/131309660] exists,up 367f5f5b-5919-4534-b88c-229f327e4950 2024-09-13T07:01:16.736 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6809/652173699,v1:172.21.15.164:6811/652173699] [v2:172.21.15.164:6814/652173699,v1:172.21.15.164:6816/652173699] exists,up 90d9e486-e1ae-4305-80a6-6f483e534643 2024-09-13T07:01:16.749 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6817/3731386737,v1:172.21.15.118:6819/3731386737] [v2:172.21.15.118:6821/3731386737,v1:172.21.15.118:6823/3731386737] exists,up 21736eee-00cd-4eef-9725-eca7776957b4', 'osd.1 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6816/876310814,v1:172.21.15.118:6818/876310814] [v2:172.21.15.118:6820/876310814,v1:172.21.15.118:6822/876310814] exists,up 05818db2-841d-406d-be17-8fbdab4fff69', 'osd.2 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6808/690188762,v1:172.21.15.118:6809/690188762] [v2:172.21.15.118:6810/690188762,v1:172.21.15.118:6811/690188762] exists,up 2387761c-42d2-481b-b5b5-b16d3a943b22', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6800/2198175481,v1:172.21.15.118:6801/2198175481] [v2:172.21.15.118:6802/2198175481,v1:172.21.15.118:6803/2198175481] exists,up 56500e90-f76b-47c8-a649-893400359c53', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6810/1835601651,v1:172.21.15.164:6813/1835601651] [v2:172.21.15.164:6817/1835601651,v1:172.21.15.164:6819/1835601651] exists,up f594a612-aa07-4565-a09e-08368cf397b6', 'osd.5 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/2853696331,v1:172.21.15.164:6801/2853696331] [v2:172.21.15.164:6802/2853696331,v1:172.21.15.164:6803/2853696331] exists,up 1013b81f-e6b4-430b-938d-88759d43b881', 'osd.6 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/131309660,v1:172.21.15.164:6812/131309660] [v2:172.21.15.164:6815/131309660,v1:172.21.15.164:6818/131309660] exists,up 367f5f5b-5919-4534-b88c-229f327e4950', 'osd.7 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6809/652173699,v1:172.21.15.164:6811/652173699] [v2:172.21.15.164:6814/652173699,v1:172.21.15.164:6816/652173699] exists,up 90d9e486-e1ae-4305-80a6-6f483e534643'] 2024-09-13T07:01:16.749 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-13T07:01:16.860 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "name": "mon.a", 2024-09-13T07:01:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster": "ceph", 2024-09-13T07:01:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-13T07:01:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "admin_socket_mode": "", 2024-09-13T07:01:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-13T07:01:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "auth_client_required": "cephx, none", 2024-09-13T07:01:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "auth_cluster_required": "cephx", 2024-09-13T07:01:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "auth_debug": "true", 2024-09-13T07:01:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-13T07:01:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-13T07:01:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "auth_service_required": "cephx", 2024-09-13T07:01:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-13T07:01:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "auth_supported": "cephx", 2024-09-13T07:01:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_aio": "true", 2024-09-13T07:01:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-13T07:01:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_aio_poll_ms": "250", 2024-09-13T07:01:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_aio_reap_max": "16", 2024-09-13T07:01:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_async_discard": "false", 2024-09-13T07:01:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_block_size": "4096", 2024-09-13T07:01:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_debug_aio": "false", 2024-09-13T07:01:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-13T07:01:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-13T07:01:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_debug_inflight_ios": "false", 2024-09-13T07:01:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_enable_discard": "false", 2024-09-13T07:01:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_flock_retry": "3", 2024-09-13T07:01:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-13T07:01:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_inject_crash": "0", 2024-09-13T07:01:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-13T07:01:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_ioring": "false", 2024-09-13T07:01:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_ioring_hipri": "false", 2024-09-13T07:01:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-13T07:01:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-13T07:01:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bdev_type": "", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_alloc_size": "1048576", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_allocator": "hybrid", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_buffered_io": "true", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_check_for_zeros": "false", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_compact_log_sync": "false", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-13T07:01:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_max_log_runway": "4194304", 2024-09-13T07:01:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_max_prefetch": "1048576", 2024-09-13T07:01:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_min_flush_size": "524288", 2024-09-13T07:01:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_min_log_runway": "1048576", 2024-09-13T07:01:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_replay_recovery": "false", 2024-09-13T07:01:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-13T07:01:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-13T07:01:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_sync_write": "false", 2024-09-13T07:01:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-13T07:01:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-13T07:01:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-13T07:01:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_allocation_from_file": "true", 2024-09-13T07:01:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_allocator": "hybrid", 2024-09-13T07:01:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-13T07:01:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-13T07:01:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-13T07:01:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-13T07:01:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-13T07:01:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-13T07:01:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-13T07:01:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_block_create": "true", 2024-09-13T07:01:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_block_db_create": "false", 2024-09-13T07:01:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_block_db_path": "", 2024-09-13T07:01:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_block_db_size": "0", 2024-09-13T07:01:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_block_path": "", 2024-09-13T07:01:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_block_preallocate_file": "false", 2024-09-13T07:01:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_block_size": "107374182400", 2024-09-13T07:01:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_block_wal_create": "false", 2024-09-13T07:01:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_block_wal_path": "", 2024-09-13T07:01:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_block_wal_size": "100663296", 2024-09-13T07:01:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_bluefs": "true", 2024-09-13T07:01:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-13T07:01:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-13T07:01:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-13T07:01:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-13T07:01:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T07:01:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T07:01:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-13T07:01:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T07:01:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_autotune": "true", 2024-09-13T07:01:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-13T07:01:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-13T07:01:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-13T07:01:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-13T07:01:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_size": "0", 2024-09-13T07:01:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-13T07:01:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-13T07:01:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-13T07:01:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-13T07:01:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_type": "2q", 2024-09-13T07:01:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-13T07:01:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_clone_cow": "true", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_compression_mode": "none", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_csum_type": "crc32c", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_freelist": "false", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-13T07:01:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_small_allocations": "0", 2024-09-13T07:01:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-13T07:01:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_default_buffered_read": "true", 2024-09-13T07:01:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_default_buffered_write": "false", 2024-09-13T07:01:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-13T07:01:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-13T07:01:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-13T07:01:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-13T07:01:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-13T07:01:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-13T07:01:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-13T07:01:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-13T07:01:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fail_eio": "false", 2024-09-13T07:01:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-13T07:01:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-13T07:01:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-13T07:01:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-13T07:01:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-13T07:01:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-13T07:01:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_on_mount": "false", 2024-09-13T07:01:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-13T07:01:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_on_umount": "false", 2024-09-13T07:01:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-13T07:01:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-13T07:01:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-13T07:01:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-13T07:01:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-13T07:01:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_ignore_data_csum": "false", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_log_op_age": "5.000000", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_max_alloc_size": "0", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_max_blob_size": "0", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_max_deferred_txc": "32", 2024-09-13T07:01:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_min_alloc_size": "0", 2024-09-13T07:01:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-13T07:01:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-13T07:01:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_nid_prealloc": "1024", 2024-09-13T07:01:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-13T07:01:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-13T07:01:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-13T07:01:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-13T07:01:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_retry_disk_reads": "3", 2024-09-13T07:01:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_rocksdb_cf": "true", 2024-09-13T07:01:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-13T07:01:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-13T07:01:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_spdk_mem": "512", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_tracing": "false", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-13T07:01:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-13T07:01:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-13T07:01:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-13T07:01:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-13T07:01:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-13T07:01:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-13T07:01:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-13T07:01:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-13T07:01:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_zero_block_detection": "false", 2024-09-13T07:01:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-13T07:01:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-13T07:01:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-13T07:01:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-13T07:01:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephx_cluster_require_signatures": "false", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephx_cluster_require_version": "2", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephx_require_signatures": "false", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephx_require_version": "2", 2024-09-13T07:01:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "cephx_service_require_signatures": "false", 2024-09-13T07:01:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "cephx_service_require_version": "2", 2024-09-13T07:01:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "cephx_sign_messages": "true", 2024-09-13T07:01:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "chdir": "", 2024-09-13T07:01:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_acl_type": "", 2024-09-13T07:01:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_asio_thread_count": "2", 2024-09-13T07:01:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_cache_mid": "0.750000", 2024-09-13T07:01:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_cache_size": "16384", 2024-09-13T07:01:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_caps_release_delay": "5", 2024-09-13T07:01:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_check_pool_perm": "true", 2024-09-13T07:01:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-13T07:01:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_debug_force_sync_read": "false", 2024-09-13T07:01:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_debug_getattr_caps": "false", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_debug_inject_features": "", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_debug_inject_tick_delay": "0", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_die_on_failed_remount": "false", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_dirsize_rbytes": "true", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_force_lazyio": "false", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_fs": "", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_inject_release_failure": "false", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_max_inline_size": "4096", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_mds_namespace": "", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_metadata": "", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_mount_gid": "-1", 2024-09-13T07:01:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_mount_timeout": "300", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_mount_uid": "-1", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_mountpoint": "/", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_notify_timeout": "10", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_oc": "true", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_oc_max_dirty": "104857600", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_oc_max_objects": "1000", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_oc_size": "209715200", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_oc_target_dirty": "8388608", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_permissions": "true", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_quota": "true", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_quota_df": "true", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_readahead_max_bytes": "0", 2024-09-13T07:01:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_readahead_max_periods": "4", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client_readahead_min": "131072", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client_reconnect_stale": "false", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client_shutdown_timeout": "30", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client_snapdir": ".snap", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client_tick_interval": "1", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client_trace": "", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client_try_dentry_invalidate": "false", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client_use_faked_inos": "false", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "client_use_random_mds": "false", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "clog_to_graylog": "false", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "clog_to_graylog_port": "12201", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "clog_to_monitors": "default=true", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "clog_to_syslog": "false", 2024-09-13T07:01:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "clog_to_syslog_level": "info", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "-", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_network": "", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_network_interface": "", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "compressor_zlib_isal": "false", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "compressor_zlib_level": "5", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "compressor_zlib_winsize": "-15", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "compressor_zstd_level": "1", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "crimson_alien_op_num_threads": "6", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-13T07:01:16.881 INFO:teuthology.orchestra.run.smithi118.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "crimson_seastar_smp": "1", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_location": "", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_location_hook": "", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_location_hook_timeout": "10", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "daemonize": "false", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "daos_pool": "tank", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_alienstore": "0/5", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_allow_any_pool_priority": "false", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_asok": "1/5", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_asok_assert_abort": "false", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_asserts_on_shutdown": "true", 2024-09-13T07:01:16.882 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_auth": "20/20", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_bdev": "1/3", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_bluefs": "1/5", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_bluestore": "1/5", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_buffer": "0/1", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_ceph_exporter": "1/5", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_cephfs_mirror": "0/5", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_cephsqlite": "0/5", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_client": "0/5", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_compressor": "1/5", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_context": "0/1", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_crush": "1/1", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_crypto": "1/5", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_cyanstore": "0/5", 2024-09-13T07:01:16.883 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_deliberately_leak_memory": "false", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_disable_randomized_ping": "false", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_dpdk": "1/5", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_eventtrace": "1/5", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_filer": "0/1", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_filestore": "1/3", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_finisher": "1/1", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_fuse": "1/5", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_heartbeat_testing_span": "0", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_heartbeatmap": "1/5", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_javaclient": "1/5", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_journal": "1/3", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_journaler": "0/5", 2024-09-13T07:01:16.884 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_kstore": "1/5", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_leveldb": "4/5", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_lockdep": "0/1", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_mclock": "1/5", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_mds": "1/5", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_mds_balancer": "1/5", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_mds_locker": "1/5", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_mds_log": "1/5", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_mds_log_expire": "1/5", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_mds_migrator": "1/5", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_memstore": "1/5", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_mgr": "2/5", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_mgrc": "1/5", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_mon": "20/20", 2024-09-13T07:01:16.885 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_monc": "0/10", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_ms": "1/1", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_none": "0/5", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_objclass": "0/5", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_objectcacher": "0/5", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_objecter": "0/1", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_optracker": "0/5", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_osd": "1/5", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_paxos": "20/20", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_perfcounter": "1/5", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_prioritycache": "1/5", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_rados": "0/5", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_rbd": "0/5", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_rbd_mirror": "0/5", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_rbd_pwl": "0/5", 2024-09-13T07:01:16.886 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_rbd_replay": "0/5", 2024-09-13T07:01:16.887 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_refs": "0/0", 2024-09-13T07:01:16.887 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_reserver": "1/1", 2024-09-13T07:01:16.887 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_rgw": "1/5", 2024-09-13T07:01:16.887 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_rgw_access": "1/5", 2024-09-13T07:01:16.887 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_rgw_datacache": "1/5", 2024-09-13T07:01:16.887 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_rgw_dbstore": "1/5", 2024-09-13T07:01:16.887 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_rgw_flight": "1/5", 2024-09-13T07:01:16.887 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_rgw_sync": "1/5", 2024-09-13T07:01:16.887 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_rocksdb": "4/5", 2024-09-13T07:01:16.887 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore": "0/5", 2024-09-13T07:01:16.887 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore_backref": "0/5", 2024-09-13T07:01:16.887 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore_cache": "0/5", 2024-09-13T07:01:16.887 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore_cleaner": "0/5", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore_device": "0/5", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore_epm": "0/5", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore_journal": "0/5", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore_lba": "0/5", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore_odata": "0/5", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore_omap": "0/5", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore_onode": "0/5", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore_t": "0/5", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_seastore_tm": "0/5", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_striper": "0/1", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_test": "0/5", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_throttle": "1/1", 2024-09-13T07:01:16.888 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_timer": "0/1", 2024-09-13T07:01:16.889 INFO:teuthology.orchestra.run.smithi118.stdout: "debug_tp": "0/5", 2024-09-13T07:01:16.889 INFO:teuthology.orchestra.run.smithi118.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-13T07:01:16.889 INFO:teuthology.orchestra.run.smithi118.stdout: "device_failure_prediction_mode": "none", 2024-09-13T07:01:16.889 INFO:teuthology.orchestra.run.smithi118.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-13T07:01:16.889 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-13T07:01:16.889 INFO:teuthology.orchestra.run.smithi118.stdout: "err_to_graylog": "false", 2024-09-13T07:01:16.889 INFO:teuthology.orchestra.run.smithi118.stdout: "err_to_journald": "false", 2024-09-13T07:01:16.889 INFO:teuthology.orchestra.run.smithi118.stdout: "err_to_stderr": "true", 2024-09-13T07:01:16.889 INFO:teuthology.orchestra.run.smithi118.stdout: "err_to_syslog": "false", 2024-09-13T07:01:16.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event_tracing": "false", 2024-09-13T07:01:16.889 INFO:teuthology.orchestra.run.smithi118.stdout: "exporter_addr": "0.0.0.0", 2024-09-13T07:01:16.889 INFO:teuthology.orchestra.run.smithi118.stdout: "exporter_http_port": "9926", 2024-09-13T07:01:16.889 INFO:teuthology.orchestra.run.smithi118.stdout: "exporter_prio_limit": "5", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "exporter_sort_metrics": "true", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "exporter_stats_period": "5", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "fake_statfs_for_testing": "0", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "fatal_signal_handlers": "true", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "filer_max_purge_ops": "10", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "filer_max_truncate_ops": "128", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_apply_finisher_threads": "1", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_blackhole": "false", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_btrfs_clone_range": "true", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_btrfs_snap": "true", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_caller_concurrency": "10", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_collect_device_partition_information": "true", 2024-09-13T07:01:16.890 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_commit_timeout": "600.000000", 2024-09-13T07:01:16.891 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_debug_inject_read_err": "false", 2024-09-13T07:01:16.891 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_debug_omap_check": "false", 2024-09-13T07:01:16.891 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-13T07:01:16.891 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_debug_verify_split": "false", 2024-09-13T07:01:16.891 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_dump_file": "", 2024-09-13T07:01:16.891 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-13T07:01:16.891 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-13T07:01:16.891 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_fadvise": "true", 2024-09-13T07:01:16.891 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_fail_eio": "true", 2024-09-13T07:01:16.891 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_fd_cache_shards": "16", 2024-09-13T07:01:16.891 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_fd_cache_size": "128", 2024-09-13T07:01:16.891 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_fiemap": "false", 2024-09-13T07:01:16.891 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_fiemap_threshold": "4096", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_inject_stall": "0", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_journal_parallel": "false", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_journal_trailing": "false", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_journal_writeahead": "false", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_kill_at": "0", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_inline_xattrs": "0", 2024-09-13T07:01:16.892 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-13T07:01:16.893 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-13T07:01:16.893 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-13T07:01:16.893 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-13T07:01:16.893 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_xattr_value_size": "0", 2024-09-13T07:01:16.893 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-13T07:01:16.893 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-13T07:01:16.893 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-13T07:01:16.893 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_merge_threshold": "-10", 2024-09-13T07:01:16.893 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-13T07:01:16.893 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_odsync_write": "false", 2024-09-13T07:01:16.893 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_omap_backend": "rocksdb", 2024-09-13T07:01:16.893 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_omap_backend_path": "", 2024-09-13T07:01:16.893 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_op_thread_timeout": "60", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_op_threads": "2", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_punch_hole": "false", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-13T07:01:16.894 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-13T07:01:16.895 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_queue_max_ops": "50", 2024-09-13T07:01:16.895 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-13T07:01:16.895 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_seek_data_hole": "false", 2024-09-13T07:01:16.895 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_sloppy_crc": "false", 2024-09-13T07:01:16.895 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-13T07:01:16.895 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_splice": "false", 2024-09-13T07:01:16.895 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_split_multiple": "2", 2024-09-13T07:01:16.895 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_split_rand_factor": "20", 2024-09-13T07:01:16.895 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_update_to": "1000", 2024-09-13T07:01:16.895 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-13T07:01:16.895 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-13T07:01:16.895 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-13T07:01:16.895 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_wbthrottle_enable": "true", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_xfs_extsize": "false", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "filestore_zfs_snap": "false", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "fio_dir": "/tmp/fio", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "fsid": "25bb30d1-9541-4e40-8659-18d83bb1bd8a", 2024-09-13T07:01:16.896 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_allow_other": "true", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_atomic_o_trunc": "true", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_big_writes": "true", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_debug": "false", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_default_permissions": "false", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_disable_pagecache": "false", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_max_write": "0", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_multithreaded": "true", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_require_active_mds": "true", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_set_user_groups": "true", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_splice_move": "true", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_splice_read": "true", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_splice_write": "true", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-13T07:01:16.897 INFO:teuthology.orchestra.run.smithi118.stdout: "fuse_use_invalidate_cb": "true", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "gss_target_name": "ceph", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_file": "", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_inject_failure": "0", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_interval": "5", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "host": "smithi118", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-13T07:01:16.898 INFO:teuthology.orchestra.run.smithi118.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "inject_early_sigterm": "false", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "jaeger_agent_port": "6799", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "jaeger_tracing_enable": "false", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_aio": "true", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_align_min_size": "65536", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_block_align": "true", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_block_size": "4096", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_dio": "true", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_discard": "false", 2024-09-13T07:01:16.899 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_force_aio": "false", 2024-09-13T07:01:16.900 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_ignore_corruption": "false", 2024-09-13T07:01:16.900 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_max_write_bytes": "10485760", 2024-09-13T07:01:16.900 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_max_write_entries": "100", 2024-09-13T07:01:16.900 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_replay_from": "0", 2024-09-13T07:01:16.900 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-13T07:01:16.900 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-13T07:01:16.900 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-13T07:01:16.900 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-13T07:01:16.900 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_write_header_frequency": "0", 2024-09-13T07:01:16.900 INFO:teuthology.orchestra.run.smithi118.stdout: "journal_zero_on_create": "false", 2024-09-13T07:01:16.900 INFO:teuthology.orchestra.run.smithi118.stdout: "journaler_prefetch_periods": "10", 2024-09-13T07:01:16.900 INFO:teuthology.orchestra.run.smithi118.stdout: "journaler_prezero_periods": "5", 2024-09-13T07:01:16.900 INFO:teuthology.orchestra.run.smithi118.stdout: "journaler_write_head_interval": "15", 2024-09-13T07:01:16.901 INFO:teuthology.orchestra.run.smithi118.stdout: "key": "", 2024-09-13T07:01:16.901 INFO:teuthology.orchestra.run.smithi118.stdout: "keyfile": "", 2024-09-13T07:01:16.901 INFO:teuthology.orchestra.run.smithi118.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-13T07:01:16.901 INFO:teuthology.orchestra.run.smithi118.stdout: "kstore_backend": "rocksdb", 2024-09-13T07:01:16.901 INFO:teuthology.orchestra.run.smithi118.stdout: "kstore_default_stripe_size": "65536", 2024-09-13T07:01:16.901 INFO:teuthology.orchestra.run.smithi118.stdout: "kstore_fsck_on_mount": "false", 2024-09-13T07:01:16.901 INFO:teuthology.orchestra.run.smithi118.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-13T07:01:16.901 INFO:teuthology.orchestra.run.smithi118.stdout: "kstore_max_bytes": "67108864", 2024-09-13T07:01:16.901 INFO:teuthology.orchestra.run.smithi118.stdout: "kstore_max_ops": "512", 2024-09-13T07:01:16.901 INFO:teuthology.orchestra.run.smithi118.stdout: "kstore_nid_prealloc": "1024", 2024-09-13T07:01:16.901 INFO:teuthology.orchestra.run.smithi118.stdout: "kstore_onode_map_size": "1024", 2024-09-13T07:01:16.901 INFO:teuthology.orchestra.run.smithi118.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-13T07:01:16.901 INFO:teuthology.orchestra.run.smithi118.stdout: "kstore_sync_submit_transaction": "false", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "kstore_sync_transaction": "false", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "leveldb_block_size": "65536", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "leveldb_bloom_size": "0", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "leveldb_cache_size": "536870912", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "leveldb_compact_on_mount": "false", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "leveldb_compression": "false", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "leveldb_log": "", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "leveldb_max_open_files": "0", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "leveldb_paranoid": "false", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "librados_thread_count": "2", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "lockdep": "false", 2024-09-13T07:01:16.902 INFO:teuthology.orchestra.run.smithi118.stdout: "lockdep_force_backtrace": "false", 2024-09-13T07:01:16.903 INFO:teuthology.orchestra.run.smithi118.stdout: "log_coarse_timestamps": "true", 2024-09-13T07:01:16.903 INFO:teuthology.orchestra.run.smithi118.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-13T07:01:16.903 INFO:teuthology.orchestra.run.smithi118.stdout: "log_flush_on_exit": "false", 2024-09-13T07:01:16.903 INFO:teuthology.orchestra.run.smithi118.stdout: "log_graylog_host": "127.0.0.1", 2024-09-13T07:01:16.903 INFO:teuthology.orchestra.run.smithi118.stdout: "log_graylog_port": "12201", 2024-09-13T07:01:16.903 INFO:teuthology.orchestra.run.smithi118.stdout: "log_max_new": "1000", 2024-09-13T07:01:16.903 INFO:teuthology.orchestra.run.smithi118.stdout: "log_max_recent": "10000", 2024-09-13T07:01:16.903 INFO:teuthology.orchestra.run.smithi118.stdout: "log_stderr_prefix": "", 2024-09-13T07:01:16.903 INFO:teuthology.orchestra.run.smithi118.stdout: "log_stop_at_utilization": "0.970000", 2024-09-13T07:01:16.903 INFO:teuthology.orchestra.run.smithi118.stdout: "log_to_file": "true", 2024-09-13T07:01:16.903 INFO:teuthology.orchestra.run.smithi118.stdout: "log_to_graylog": "false", 2024-09-13T07:01:16.903 INFO:teuthology.orchestra.run.smithi118.stdout: "log_to_journald": "false", 2024-09-13T07:01:16.903 INFO:teuthology.orchestra.run.smithi118.stdout: "log_to_stderr": "false", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "log_to_syslog": "false", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "max_rotating_auth_attempts": "10", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_action_on_write_error": "1", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_alternate_name_max": "8192", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_asio_thread_count": "2", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_export_pin": "true", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_fragment_dirs": "true", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_fragment_interval": "5", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_interval": "10", 2024-09-13T07:01:16.904 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_max": "-1", 2024-09-13T07:01:16.905 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_max_until": "-1", 2024-09-13T07:01:16.905 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_merge_size": "50", 2024-09-13T07:01:16.905 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_midchunk": "0.300000", 2024-09-13T07:01:16.905 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-13T07:01:16.905 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_min_start": "0.200000", 2024-09-13T07:01:16.905 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_minchunk": "0.001000", 2024-09-13T07:01:16.905 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_mode": "0", 2024-09-13T07:01:16.905 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_need_max": "1.200000", 2024-09-13T07:01:16.905 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_need_min": "0.800000", 2024-09-13T07:01:16.905 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-13T07:01:16.905 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-13T07:01:16.905 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_split_bits": "3", 2024-09-13T07:01:16.905 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_split_size": "10000", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_target_decay": "10.000000", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_beacon_grace": "15.000000", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_beacon_interval": "4.000000", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache_mid": "0.700000", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache_release_free_interval": "10", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache_reservation": "0.050000", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache_trim_interval": "1", 2024-09-13T07:01:16.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache_trim_threshold": "262144", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_client_prealloc_inos": "1000", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_connect_bootstrapping": "false", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_damage_table_max_entries": "10000", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_debug_auth_pins": "false", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_debug_frag": "false", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_debug_scatterstat": "false", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_debug_subtrees": "false", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_decay_halflife": "5.000000", 2024-09-13T07:01:16.907 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_default_dir_hash": "2", 2024-09-13T07:01:16.908 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_defer_session_stale": "true", 2024-09-13T07:01:16.908 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_deny_all_reconnect": "false", 2024-09-13T07:01:16.908 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_dir_keys_per_op": "16384", 2024-09-13T07:01:16.908 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_dir_max_commit_size": "10", 2024-09-13T07:01:16.908 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_dir_max_entries": "0", 2024-09-13T07:01:16.908 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_dir_prefetch": "true", 2024-09-13T07:01:16.908 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-13T07:01:16.908 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-13T07:01:16.908 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_dump_cache_on_map": "false", 2024-09-13T07:01:16.908 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-13T07:01:16.908 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-13T07:01:16.908 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_early_reply": "true", 2024-09-13T07:01:16.908 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_enable_op_tracker": "true", 2024-09-13T07:01:16.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_enforce_unique_name": "true", 2024-09-13T07:01:16.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-13T07:01:16.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-13T07:01:16.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_export_ephemeral_random": "true", 2024-09-13T07:01:16.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-13T07:01:16.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-13T07:01:16.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-13T07:01:16.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-13T07:01:16.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-13T07:01:16.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-13T07:01:16.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-13T07:01:16.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-13T07:01:16.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_health_summarize_threshold": "10", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_inject_health_dummy": "false", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_inject_migrator_session_race": "false", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_join_fs": "", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_journal_format": "1", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_kill_create_at": "0", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_kill_export_at": "0", 2024-09-13T07:01:16.910 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_kill_import_at": "0", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_kill_journal_at": "0", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_kill_journal_expire_at": "0", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_kill_journal_replay_at": "0", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_kill_link_at": "0", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_kill_mdstable_at": "0", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_kill_openc_at": "0", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_kill_rename_at": "0", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log_events_per_segment": "1024", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log_max_events": "-1", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log_max_segments": "128", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log_pause": "false", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log_segment_size": "0", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-13T07:01:16.911 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log_warn_factor": "2.000000", 2024-09-13T07:01:16.912 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_max_caps_per_client": "1048576", 2024-09-13T07:01:16.912 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_max_completed_flushes": "100000", 2024-09-13T07:01:16.912 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_max_completed_requests": "100000", 2024-09-13T07:01:16.912 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_max_export_size": "20971520", 2024-09-13T07:01:16.912 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_max_file_recover": "32", 2024-09-13T07:01:16.912 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_max_purge_files": "64", 2024-09-13T07:01:16.912 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_max_purge_ops": "8192", 2024-09-13T07:01:16.912 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-13T07:01:16.912 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-13T07:01:16.912 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_max_snaps_per_dir": "100", 2024-09-13T07:01:16.912 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_metrics_update_interval": "2", 2024-09-13T07:01:16.912 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_min_caps_per_client": "100", 2024-09-13T07:01:16.912 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_min_caps_working_set": "10000", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_numa_node": "-1", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_op_complaint_time": "30.000000", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_op_history_duration": "600", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_op_history_size": "20", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_op_history_slow_op_size": "20", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_op_log_threshold": "5", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_ping_grace": "15", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_ping_interval": "5", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-13T07:01:16.913 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_recall_max_caps": "30000", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_recall_warning_threshold": "262144", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_replay_interval": "1.000000", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_root_ino_gid": "0", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_root_ino_uid": "0", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-13T07:01:16.914 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_shutdown_check": "0", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_skip_ino": "0", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_snap_max_uid": "4294967294", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_snap_min_uid": "0", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_snap_rstat": "false", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_standby_replay_damaged": "false", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_symlink_recovery": "true", 2024-09-13T07:01:16.915 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_thrash_exports": "0", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_thrash_fragments": "0", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_tick_interval": "5.000000", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_valgrind_exit": "false", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_verify_backtrace": "1", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_verify_scatter": "false", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_wipe_sessions": "false", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "mempool_debug": "false", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "memstore_device_bytes": "1073741824", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "memstore_page_set": "false", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "memstore_page_size": "65536", 2024-09-13T07:01:16.916 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_client_bytes": "134217728", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_client_messages": "512", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_disabled_modules": "", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_max_pg_num_change": "128", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_mds_bytes": "134217728", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_mds_messages": "128", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_mon_bytes": "134217728", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_mon_messages": "128", 2024-09-13T07:01:16.917 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_osd_bytes": "536870912", 2024-09-13T07:01:16.918 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_osd_messages": "8192", 2024-09-13T07:01:16.918 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_pool": "true", 2024-09-13T07:01:16.918 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-13T07:01:16.918 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_standby_modules": "true", 2024-09-13T07:01:16.918 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_stats_period": "5", 2024-09-13T07:01:16.918 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_stats_threshold": "5", 2024-09-13T07:01:16.918 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_tick_period": "2", 2024-09-13T07:01:16.918 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-13T07:01:16.918 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-13T07:01:16.918 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_allow_pool_delete": "true", 2024-09-13T07:01:16.918 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_allow_pool_size_one": "true", 2024-09-13T07:01:16.918 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_auth_validate_all_caps": "true", 2024-09-13T07:01:16.918 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-13T07:01:16.919 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-13T07:01:16.919 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_client_bytes": "104857600", 2024-09-13T07:01:16.919 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_client_directed_command_retry": "5", 2024-09-13T07:01:16.919 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-13T07:01:16.919 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-13T07:01:16.919 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-13T07:01:16.919 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-13T07:01:16.919 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_client_hunt_parallel": "3", 2024-09-13T07:01:16.919 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_client_log_interval": "1.000000", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_client_ping_interval": "10.000000", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_cluster_log_file_level": "debug", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_cluster_log_to_file": "true", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_cluster_log_to_journald": "false", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-13T07:01:16.920 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-13T07:01:16.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-13T07:01:16.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-13T07:01:16.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_compact_on_bootstrap": "false", 2024-09-13T07:01:16.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_compact_on_start": "false", 2024-09-13T07:01:16.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_compact_on_trim": "true", 2024-09-13T07:01:16.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-13T07:01:16.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-13T07:01:16.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-13T07:01:16.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_cpu_threads": "4", 2024-09-13T07:01:16.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_crush_min_required_version": "hammer", 2024-09-13T07:01:16.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_daemon_bytes": "419430400", 2024-09-13T07:01:16.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-13T07:01:16.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_data_avail_crit": "5", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_data_avail_warn": "5", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_data_size_warn": "16106127360", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_debug_dump_json": "false", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_debug_dump_transactions": "false", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_debug_extra_checks": "false", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_debug_no_require_quincy": "false", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_debug_no_require_reef": "false", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-13T07:01:16.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_down_mkfs_grace": "120", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_election_default_strategy": "1", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_election_timeout": "5.000000", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_elector_ping_divisor": "2", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_enable_op_tracker": "true", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_fake_pool_delete": "false", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_force_quorum_join": "false", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_globalid_prealloc": "10000", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_health_detail_to_clog": "false", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_health_log_update_period": "5", 2024-09-13T07:01:16.923 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_health_max_detail": "50", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_health_to_clog": "true", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_health_to_clog_interval": "600", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_host": "172.21.15.118,[v2:172.21.15.118:3301,v1:172.21.15.118:6790],172.21.15.164", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_host_override": "", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_initial_members": "", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_lease": "5.000000", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-13T07:01:16.924 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-13T07:01:16.925 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_log_full_interval": "50", 2024-09-13T07:01:16.925 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_log_max": "10000", 2024-09-13T07:01:16.925 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_log_max_summary": "50", 2024-09-13T07:01:16.925 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-13T07:01:16.925 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_max_log_epochs": "500", 2024-09-13T07:01:16.925 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-13T07:01:16.925 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-13T07:01:16.925 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_max_osd": "10000", 2024-09-13T07:01:16.925 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_max_pg_per_osd": "10000", 2024-09-13T07:01:16.925 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_max_pool_pg_num": "65536", 2024-09-13T07:01:16.925 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-13T07:01:16.925 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_mds_force_trim_to": "0", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_mds_skip_sanity": "false", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_memory_autotune": "true", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_memory_target": "2147483648", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_mgr_beacon_grace": "30", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_mgr_digest_period": "5", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_mgr_inactive_grace": "60", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_min_osdmap_epochs": "500", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_op_complaint_time": "30", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_op_history_duration": "600", 2024-09-13T07:01:16.926 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_op_history_size": "20", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_op_history_slow_op_size": "20", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_op_log_threshold": "5", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_auto_mark_in": "false", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_cache_size": "500", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-13T07:01:16.927 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_down_out_interval": "600", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_force_trim_to": "0", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_min_down_reporters": "2", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-13T07:01:16.928 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_report_timeout": "900", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-13T07:01:16.929 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_pg_stuck_threshold": "60", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_probe_timeout": "2.000000", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_reweight_max_change": "0.050000", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_reweight_max_osds": "4", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-13T07:01:16.930 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_scrub_interval": "86400", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_scrub_max_keys": "100", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_scrub_timeout": "300", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_session_timeout": "300", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_smart_report_timeout": "5", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_stat_smooth_intervals": "6", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_stretch_pool_min_size": "2", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_stretch_pool_size": "4", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_sync_debug": "false", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_sync_provider_kill_at": "0", 2024-09-13T07:01:16.931 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_sync_requester_kill_at": "0", 2024-09-13T07:01:16.932 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_sync_timeout": "60.000000", 2024-09-13T07:01:16.932 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_target_pg_per_osd": "100", 2024-09-13T07:01:16.932 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_tick_interval": "5", 2024-09-13T07:01:16.932 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_timecheck_interval": "300.000000", 2024-09-13T07:01:16.932 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-13T07:01:16.932 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_use_min_delay_socket": "false", 2024-09-13T07:01:16.932 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_older_version_delay": "604800", 2024-09-13T07:01:16.932 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-13T07:01:16.932 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-13T07:01:16.932 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-13T07:01:16.932 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-13T07:01:16.932 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-13T07:01:16.932 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-13T07:01:16.933 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-13T07:01:16.933 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_misplaced": "false", 2024-09-13T07:01:16.933 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-13T07:01:16.933 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_older_version": "true", 2024-09-13T07:01:16.933 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-13T07:01:16.933 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-13T07:01:16.933 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-13T07:01:16.933 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-13T07:01:16.933 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-13T07:01:16.933 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-13T07:01:16.933 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-13T07:01:16.933 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-13T07:01:16.933 INFO:teuthology.orchestra.run.smithi118.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-13T07:01:16.934 INFO:teuthology.orchestra.run.smithi118.stdout: "monmap": "", 2024-09-13T07:01:16.934 INFO:teuthology.orchestra.run.smithi118.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T07:01:16.934 INFO:teuthology.orchestra.run.smithi118.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-13T07:01:16.934 INFO:teuthology.orchestra.run.smithi118.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-13T07:01:16.934 INFO:teuthology.orchestra.run.smithi118.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T07:01:16.934 INFO:teuthology.orchestra.run.smithi118.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-13T07:01:16.934 INFO:teuthology.orchestra.run.smithi118.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-13T07:01:16.934 INFO:teuthology.orchestra.run.smithi118.stdout: "motr_tracing_enabled": "false", 2024-09-13T07:01:16.934 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_op_threads": "3", 2024-09-13T07:01:16.934 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-13T07:01:16.934 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_cm": "false", 2024-09-13T07:01:16.934 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_device_name": "", 2024-09-13T07:01:16.934 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_dscp": "96", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_local_gid": "", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_port_num": "1", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_sl": "3", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_support_srq": "true", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_rdma_type": "ib", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_async_reap_threshold": "5", 2024-09-13T07:01:16.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_bind_before_connect": "false", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_bind_ipv4": "true", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_bind_ipv6": "false", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_bind_msgr1": "true", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_bind_msgr2": "true", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_bind_port_max": "7568", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_bind_port_min": "6800", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_bind_retry_count": "3", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_bind_retry_delay": "5", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_blackhole_client": "false", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_blackhole_mds": "false", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_blackhole_mgr": "false", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_blackhole_mon": "false", 2024-09-13T07:01:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_blackhole_osd": "false", 2024-09-13T07:01:16.937 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_client_mode": "crc secure", 2024-09-13T07:01:16.937 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_cluster_mode": "crc secure", 2024-09-13T07:01:16.937 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_cluster_type": "", 2024-09-13T07:01:16.937 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_compress_secure": "false", 2024-09-13T07:01:16.937 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_connection_idle_timeout": "900", 2024-09-13T07:01:16.937 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_connection_ready_timeout": "10", 2024-09-13T07:01:16.937 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_crc_data": "true", 2024-09-13T07:01:16.937 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_crc_header": "true", 2024-09-13T07:01:16.937 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_die_on_bad_msg": "false", 2024-09-13T07:01:16.937 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_die_on_bug": "true", 2024-09-13T07:01:16.937 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_die_on_old_message": "true", 2024-09-13T07:01:16.937 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_die_on_skipped_message": "false", 2024-09-13T07:01:16.937 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_coremask": "0xF", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_enable_tso": "true", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_hugepages": "", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_lro": "true", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_memory_channel": "4", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-13T07:01:16.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_pmd": "", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_port_id": "0", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_dump_on_send": "false", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_initial_backoff": "0.200000", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_inject_delay_max": "1.000000", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_inject_delay_type": "", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_inject_network_congestion": "0", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_inject_socket_failures": "5000", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_learn_addr_from_peer": "true", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_max_accept_failures": "4", 2024-09-13T07:01:16.939 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_max_backoff": "15.000000", 2024-09-13T07:01:16.940 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_mon_client_mode": "secure crc", 2024-09-13T07:01:16.940 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-13T07:01:16.940 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_mon_service_mode": "secure crc", 2024-09-13T07:01:16.940 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_osd_compress_min_size": "1024", 2024-09-13T07:01:16.940 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_osd_compress_mode": "none", 2024-09-13T07:01:16.940 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-13T07:01:16.940 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-13T07:01:16.940 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_pq_min_cost": "65536", 2024-09-13T07:01:16.940 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_public_type": "", 2024-09-13T07:01:16.940 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_service_mode": "crc secure", 2024-09-13T07:01:16.940 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_tcp_listen_backlog": "512", 2024-09-13T07:01:16.940 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_tcp_nodelay": "true", 2024-09-13T07:01:16.940 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_tcp_rcvbuf": "0", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "ms_type": "async+posix", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "no_config_file": "false", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter_completion_locks_per_session": "32", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter_inflight_ops": "1024", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter_tick_interval": "5.000000", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter_timeout": "10.000000", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "objectstore_blackhole": "false", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "openssl_engine_opts": "", 2024-09-13T07:01:16.941 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_agent_delay_time": "5.000000", 2024-09-13T07:01:16.942 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_agent_hist_halflife": "1000", 2024-09-13T07:01:16.942 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_agent_max_low_ops": "2", 2024-09-13T07:01:16.942 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_agent_max_ops": "4", 2024-09-13T07:01:16.942 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-13T07:01:16.942 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-13T07:01:16.942 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_agent_slop": "0.020000", 2024-09-13T07:01:16.942 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-13T07:01:16.942 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-13T07:01:16.942 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_asio_thread_count": "2", 2024-09-13T07:01:16.942 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_async_recovery_min_cost": "100", 2024-09-13T07:01:16.942 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-13T07:01:16.942 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-13T07:01:16.942 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_backfill_scan_max": "512", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_backfill_scan_min": "64", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_backoff_on_degraded": "false", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_backoff_on_peering": "false", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_backoff_on_unfound": "true", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_beacon_report_interval": "300", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_bench_duration": "30", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_bench_max_block_size": "67108864", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_blkin_trace_all": "false", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-13T07:01:16.943 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-13T07:01:16.944 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_check_for_log_corruption": "false", 2024-09-13T07:01:16.944 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-13T07:01:16.944 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T07:01:16.944 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-13T07:01:16.944 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T07:01:16.944 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_class_update_on_start": "true", 2024-09-13T07:01:16.944 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_client_message_cap": "256", 2024-09-13T07:01:16.944 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_client_message_size_cap": "524288000", 2024-09-13T07:01:16.944 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_client_op_priority": "63", 2024-09-13T07:01:16.944 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_client_watch_timeout": "30", 2024-09-13T07:01:16.944 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_command_max_records": "256", 2024-09-13T07:01:16.944 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_compact_on_start": "false", 2024-09-13T07:01:16.944 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_crush_update_on_start": "true", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_crush_update_weight_set": "true", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_feed_pullee": "-1", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-13T07:01:16.945 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_misdirected_ops": "false", 2024-09-13T07:01:16.946 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_no_acting_change": "false", 2024-09-13T07:01:16.946 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_no_purge_strays": "false", 2024-09-13T07:01:16.946 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_op_order": "false", 2024-09-13T07:01:16.946 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-13T07:01:16.946 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-13T07:01:16.946 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-13T07:01:16.946 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-13T07:01:16.946 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_shutdown": "false", 2024-09-13T07:01:16.946 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-13T07:01:16.946 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-13T07:01:16.946 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-13T07:01:16.946 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-13T07:01:16.946 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_verify_snaps": "false", 2024-09-13T07:01:16.947 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-13T07:01:16.947 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-13T07:01:16.947 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_deep_scrub_keys": "1024", 2024-09-13T07:01:16.947 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-13T07:01:16.947 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-13T07:01:16.947 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-13T07:01:16.947 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_deep_scrub_stride": "524288", 2024-09-13T07:01:16.947 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-13T07:01:16.947 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-13T07:01:16.947 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_default_notify_timeout": "30", 2024-09-13T07:01:16.947 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_delete_sleep": "0.000000", 2024-09-13T07:01:16.947 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-13T07:01:16.947 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_discard_disconnected_ops": "true", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_enable_op_tracker": "true", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_fast_info": "true", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_fast_shutdown": "true", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-13T07:01:16.948 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-13T07:01:16.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_function_tracing": "false", 2024-09-13T07:01:16.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_heartbeat_grace": "20", 2024-09-13T07:01:16.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_heartbeat_interval": "6", 2024-09-13T07:01:16.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-13T07:01:16.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_heartbeat_min_peers": "10", 2024-09-13T07:01:16.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_heartbeat_min_size": "2000", 2024-09-13T07:01:16.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_heartbeat_stale": "600", 2024-09-13T07:01:16.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-13T07:01:16.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_hit_set_max_size": "100000", 2024-09-13T07:01:16.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_hit_set_min_size": "1000", 2024-09-13T07:01:16.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-13T07:01:16.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-13T07:01:16.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_journal_size": "5120", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_kill_backfill_at": "0", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_map_cache_size": "50", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_map_dedup": "true", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_map_message_max": "40", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_map_share_max_epochs": "40", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_attr_name_len": "100", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_attr_size": "0", 2024-09-13T07:01:16.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_backfills": "1", 2024-09-13T07:01:16.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_markdown_count": "5", 2024-09-13T07:01:16.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_markdown_period": "600", 2024-09-13T07:01:16.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_object_name_len": "2048", 2024-09-13T07:01:16.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_object_namespace_len": "256", 2024-09-13T07:01:16.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_object_size": "134217728", 2024-09-13T07:01:16.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-13T07:01:16.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-13T07:01:16.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_pg_blocked_by": "16", 2024-09-13T07:01:16.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_pg_log_entries": "10000", 2024-09-13T07:01:16.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-13T07:01:16.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_pgls": "1024", 2024-09-13T07:01:16.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_push_cost": "8388608", 2024-09-13T07:01:16.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_push_objects": "10", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_scrubs": "3", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_trimming_pgs": "2", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_write_op_reply_len": "64", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_max_write_size": "90", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-13T07:01:16.952 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_profile": "balanced", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_memory_base": "805306368", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_memory_cache_min": "134217728", 2024-09-13T07:01:16.953 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_memory_target": "4294967296", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_memory_target_autotune": "false", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_min_pg_log_entries": "250", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_min_recovery_priority": "0", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mon_report_interval": "5", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_num_cache_shards": "32", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_num_op_tracker_shard": "32", 2024-09-13T07:01:16.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_numa_auto_affinity": "true", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_numa_node": "-1", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_numa_prefer_iface": "true", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_objecter_finishers": "1", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_objectstore": "bluestore", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_objectstore_fuse": "false", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_objectstore_tracing": "false", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_complaint_time": "30.000000", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_history_duration": "600", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_history_size": "20", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_history_slow_op_size": "20", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_log_threshold": "5", 2024-09-13T07:01:16.955 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_num_shards": "0", 2024-09-13T07:01:16.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_num_shards_hdd": "5", 2024-09-13T07:01:16.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_num_shards_ssd": "8", 2024-09-13T07:01:16.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-13T07:01:16.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-13T07:01:16.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-13T07:01:16.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-13T07:01:16.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_pq_min_cost": "65536", 2024-09-13T07:01:16.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-13T07:01:16.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_queue_cut_off": "high", 2024-09-13T07:01:16.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-13T07:01:16.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_op_thread_timeout": "15", 2024-09-13T07:01:16.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_open_classes_on_start": "true", 2024-09-13T07:01:16.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_os_flags": "0", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_peering_op_priority": "255", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pg_delete_cost": "1048576", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pg_delete_priority": "5", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pg_log_trim_max": "10000", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pg_log_trim_min": "100", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-13T07:01:16.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-13T07:01:16.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-13T07:01:16.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-13T07:01:16.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-13T07:01:16.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_crimson": "false", 2024-09-13T07:01:16.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-13T07:01:16.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-13T07:01:16.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-13T07:01:16.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-13T07:01:16.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-13T07:01:16.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-13T07:01:16.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-13T07:01:16.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-13T07:01:16.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_flags": "0", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_min_size": "0", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_pg_num": "32", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_pgp_num": "0", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_size": "2", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_default_type": "replicated", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_push_per_object_cost": "1000", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recover_clone_overlap": "true", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-13T07:01:16.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_cost": "20971520", 2024-09-13T07:01:16.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-13T07:01:16.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_max_active": "0", 2024-09-13T07:01:16.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-13T07:01:16.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-13T07:01:16.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-13T07:01:16.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-13T07:01:16.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_max_single_start": "1", 2024-09-13T07:01:16.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_op_priority": "3", 2024-09-13T07:01:16.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-13T07:01:16.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_priority": "5", 2024-09-13T07:01:16.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-13T07:01:16.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_sleep": "0.000000", 2024-09-13T07:01:16.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-13T07:01:16.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-13T07:01:16.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-13T07:01:16.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_repair_during_recovery": "false", 2024-09-13T07:01:16.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_requested_scrub_priority": "120", 2024-09-13T07:01:16.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-13T07:01:16.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-13T07:01:16.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_auto_repair": "false", 2024-09-13T07:01:16.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-13T07:01:16.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-13T07:01:16.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_begin_hour": "0", 2024-09-13T07:01:16.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_begin_week_day": "0", 2024-09-13T07:01:16.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_chunk_max": "25", 2024-09-13T07:01:16.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_chunk_min": "5", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_cost": "52428800", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_during_recovery": "false", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_end_hour": "0", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_end_week_day": "0", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_event_cost": "4096", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_invalid_stats": "true", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_max_preemptions": "5", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_priority": "5", 2024-09-13T07:01:16.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_sleep": "0.000000", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_skip_data_digest": "false", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_smart_report_timeout": "5", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_snap_trim_cost": "1048576", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_snap_trim_priority": "5", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-13T07:01:16.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-13T07:01:16.964 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-13T07:01:16.964 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-13T07:01:16.964 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_target_transaction_size": "30", 2024-09-13T07:01:16.964 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-13T07:01:16.964 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-13T07:01:16.964 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-13T07:01:16.964 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-13T07:01:16.964 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-13T07:01:16.964 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-13T07:01:16.964 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-13T07:01:16.964 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-13T07:01:16.964 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-13T07:01:16.964 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_tracing": "false", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_use_stale_snap": "false", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "osdc_blkin_trace_all": "false", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "paxos_kill_at": "0", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "paxos_max_join_drift": "10", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "paxos_min": "500", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "paxos_min_wait": "0.050000", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "paxos_propose_interval": "1.000000", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "paxos_service_trim_max": "500", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "paxos_service_trim_min": "250", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "paxos_stash_full_interval": "25", 2024-09-13T07:01:16.965 INFO:teuthology.orchestra.run.smithi118.stdout: "paxos_trim_max": "500", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "paxos_trim_min": "250", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "perf": "true", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "-", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrv": "", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "public_bind_addr": "-", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "public_network": "", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "public_network_interface": "", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "qat_compressor_enabled": "false", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "qat_compressor_session_max_number": "256", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "rados_mon_op_timeout": "0", 2024-09-13T07:01:16.966 INFO:teuthology.orchestra.run.smithi118.stdout: "rados_osd_op_timeout": "0", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rados_tracing": "false", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_atime_update_interval": "60", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_balance_parent_reads": "false", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_balance_snap_reads": "false", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_blkin_trace_all": "false", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_cache": "true", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-13T07:01:16.967 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_cache_policy": "writearound", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_cache_size": "33554432", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_clone_copy_on_read": "false", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_compression_hint": "none", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_concurrent_management_ops": "10", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_default_clone_format": "auto", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_default_data_pool": "", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_default_features": "61", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_default_format": "2", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_default_map_options": "", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_default_order": "22", 2024-09-13T07:01:16.968 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_default_pool": "rbd", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_default_stripe_count": "0", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_default_stripe_unit": "0", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_enable_alloc_hint": "true", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_io_scheduler": "simple", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-13T07:01:16.969 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_journal_order": "24", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_journal_pool": "", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_journal_splay_width": "4", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_localize_parent_reads": "false", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_localize_snap_reads": "false", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-13T07:01:16.970 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-13T07:01:16.971 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-13T07:01:16.971 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-13T07:01:16.971 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-13T07:01:16.971 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-13T07:01:16.971 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-13T07:01:16.971 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-13T07:01:16.971 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-13T07:01:16.971 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-13T07:01:16.971 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-13T07:01:16.971 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-13T07:01:16.971 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-13T07:01:16.971 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-13T07:01:16.971 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-13T07:01:16.972 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_mtime_update_interval": "60", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_non_blocking_aio": "true", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_op_thread_timeout": "60", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_op_threads": "1", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_parent_cache_enabled": "false", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_plugins": "", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_bps_burst": "0", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_bps_limit": "0", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_exclude_ops": "0", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_iops_burst": "0", 2024-09-13T07:01:16.973 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_iops_limit": "0", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-13T07:01:16.974 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-13T07:01:16.975 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-13T07:01:16.975 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_read_from_replica_policy": "default", 2024-09-13T07:01:16.975 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-13T07:01:16.975 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-13T07:01:16.975 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-13T07:01:16.975 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-13T07:01:16.975 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_skip_partial_discard": "true", 2024-09-13T07:01:16.975 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-13T07:01:16.975 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_tracing": "false", 2024-09-13T07:01:16.975 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_validate_names": "true", 2024-09-13T07:01:16.975 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd_validate_pool": "true", 2024-09-13T07:01:16.975 INFO:teuthology.orchestra.run.smithi118.stdout: "restapi_base_url": "", 2024-09-13T07:01:16.975 INFO:teuthology.orchestra.run.smithi118.stdout: "restapi_log_level": "", 2024-09-13T07:01:16.976 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_acl_grants_max_num": "100", 2024-09-13T07:01:16.976 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_admin_entry": "admin", 2024-09-13T07:01:16.976 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-13T07:01:16.976 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_backend_store": "rados", 2024-09-13T07:01:16.976 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_barbican_url": "", 2024-09-13T07:01:16.976 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_beast_enable_async": "true", 2024-09-13T07:01:16.976 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-13T07:01:16.976 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-13T07:01:16.976 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-13T07:01:16.976 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-13T07:01:16.976 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-13T07:01:16.976 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-13T07:01:16.976 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_cache_enabled": "true", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_cache_expiry_interval": "900", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_cache_lru_size": "10000", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_config_store": "rados", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_content_length_compat": "false", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_copy_obj_progress": "true", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_cors_rules_max_num": "100", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_cross_domain_policy": "", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_kmip_addr": "", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-13T07:01:16.977 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_kmip_password": "", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_kmip_username": "", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_require_ssl": "true", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-13T07:01:16.978 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-13T07:01:16.979 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-13T07:01:16.979 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-13T07:01:16.979 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-13T07:01:16.979 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-13T07:01:16.979 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-13T07:01:16.979 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-13T07:01:16.979 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_vault_addr": "", 2024-09-13T07:01:16.979 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_vault_auth": "token", 2024-09-13T07:01:16.979 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_vault_namespace": "", 2024-09-13T07:01:16.979 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_vault_prefix": "", 2024-09-13T07:01:16.979 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-13T07:01:16.979 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-13T07:01:16.979 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-13T07:01:16.980 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-13T07:01:16.980 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_vault_token_file": "", 2024-09-13T07:01:16.980 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-13T07:01:16.980 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_curl_buffersize": "524288", 2024-09-13T07:01:16.980 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-13T07:01:16.980 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_curl_low_speed_time": "300", 2024-09-13T07:01:16.980 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-13T07:01:16.980 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-13T07:01:16.980 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-13T07:01:16.980 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-13T07:01:16.980 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-13T07:01:16.980 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-13T07:01:16.980 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_data_log_changes_size": "1000", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_data_log_num_shards": "128", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_data_log_window": "30", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-13T07:01:16.981 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-13T07:01:16.982 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dns_name": "", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dns_s3website_name": "", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_dynamic_resharding": "true", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_enable_gc_threads": "true", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_enable_lc_threads": "true", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_enable_ops_log": "false", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_enable_quota_threads": "true", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_enable_static_website": "false", 2024-09-13T07:01:16.983 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_enable_usage_log": "false", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_enforce_swift_acls": "true", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_exit_timeout_secs": "120", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_expose_bucket": "false", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_extended_http_attrs": "", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_filter": "none", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_frontends": "beast port=7480", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_gc_max_deferred": "50", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_gc_max_objs": "32", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-13T07:01:16.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_gc_processor_period": "3600", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_init_timeout": "300", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_admin_domain": "", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_admin_password": "", 2024-09-13T07:01:16.985 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_admin_password_path": "", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_admin_project": "", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_admin_tenant": "", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_admin_token": "", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_admin_token_path": "", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_admin_user": "", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_api_version": "2", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_barbican_domain": "", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_barbican_password": "", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_barbican_project": "", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_barbican_user": "", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-13T07:01:16.986 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-13T07:01:16.987 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-13T07:01:16.987 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-13T07:01:16.987 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_url": "", 2024-09-13T07:01:16.987 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-13T07:01:16.987 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_lc_debug_interval": "-1", 2024-09-13T07:01:16.987 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_lc_lock_max_time": "90", 2024-09-13T07:01:16.987 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_lc_max_objs": "32", 2024-09-13T07:01:16.987 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_lc_max_rules": "1000", 2024-09-13T07:01:16.987 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_lc_max_worker": "3", 2024-09-13T07:01:16.987 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-13T07:01:16.987 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_lc_thread_delay": "0", 2024-09-13T07:01:16.987 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-13T07:01:16.987 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_ldap_dnattr": "uid", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_ldap_searchfilter": "", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_log_http_headers": "", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_log_object_name_utc": "false", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_max_attr_name_len": "0", 2024-09-13T07:01:16.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_max_attr_size": "0", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_max_chunk_size": "4194304", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_max_listing_results": "1000", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_max_notify_retries": "10", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_max_put_param_size": "1048576", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_max_put_size": "5368709120", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_max_slo_entries": "1000", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_md_log_max_shards": "64", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-13T07:01:16.989 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_mp_lock_max_time": "600", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_nfs_max_gc": "300", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-13T07:01:16.990 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_num_async_rados_threads": "32", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_num_control_oids": "8", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_numa_node": "-1", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_objexp_chunk_size": "100", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_objexp_gc_interval": "600", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-13T07:01:16.991 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-13T07:01:16.992 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-13T07:01:16.992 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_op_thread_timeout": "600", 2024-09-13T07:01:16.992 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_op_tracing": "false", 2024-09-13T07:01:16.992 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_opa_token": "", 2024-09-13T07:01:16.992 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_opa_url": "", 2024-09-13T07:01:16.992 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_opa_verify_ssl": "true", 2024-09-13T07:01:16.992 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-13T07:01:16.992 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-13T07:01:16.992 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_ops_log_rados": "false", 2024-09-13T07:01:16.992 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_ops_log_socket_path": "", 2024-09-13T07:01:16.992 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-13T07:01:16.992 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-13T07:01:16.992 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-13T07:01:16.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_period_push_interval": "2.000000", 2024-09-13T07:01:16.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-13T07:01:16.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-13T07:01:16.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-13T07:01:16.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_print_continue": "true", 2024-09-13T07:01:16.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-13T07:01:16.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-13T07:01:16.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-13T07:01:16.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-13T07:01:16.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-13T07:01:16.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_rados_tracing": "false", 2024-09-13T07:01:16.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_realm": "", 2024-09-13T07:01:16.993 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_realm_id": "", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_region": "", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_request_uri": "", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_reshard_batch_size": "64", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_reshard_max_aio": "128", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_reshard_num_logs": "16", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_reshard_thread_interval": "600", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_resolve_cname": "false", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-13T07:01:16.994 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_run_sync_thread": "true", 2024-09-13T07:01:16.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-13T07:01:16.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-13T07:01:16.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-13T07:01:16.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-13T07:01:16.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-13T07:01:16.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-13T07:01:16.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-13T07:01:16.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-13T07:01:16.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_scheduler_type": "throttler", 2024-09-13T07:01:16.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_script_uri": "", 2024-09-13T07:01:16.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_service_provider_name": "", 2024-09-13T07:01:16.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-13T07:01:16.995 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sts_client_id": "", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sts_client_secret": "", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sts_entry": "sts", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sts_key": "sts", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sts_min_session_duration": "900", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sts_token_introspection_url": "", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_swift_account_in_url": "false", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_swift_auth_entry": "auth", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_swift_auth_url": "", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_swift_custom_header": "", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_swift_need_stats": "true", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_swift_tenant_name": "", 2024-09-13T07:01:16.996 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_swift_token_expiration": "86400", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_swift_url": "", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_swift_url_prefix": "swift", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sync_lease_period": "120", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-13T07:01:16.997 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-13T07:01:16.998 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-13T07:01:16.998 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_thread_pool_size": "512", 2024-09-13T07:01:16.998 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_torrent_comment": "", 2024-09-13T07:01:16.998 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_torrent_createby": "", 2024-09-13T07:01:16.998 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_torrent_encoding": "", 2024-09-13T07:01:16.998 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_torrent_flag": "false", 2024-09-13T07:01:16.998 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_torrent_origin": "", 2024-09-13T07:01:16.998 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-13T07:01:16.998 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_torrent_tracker": "", 2024-09-13T07:01:16.998 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_trust_forwarded_https": "false", 2024-09-13T07:01:16.998 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-13T07:01:16.998 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-13T07:01:16.998 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_usage_max_shards": "32", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_usage_max_user_shards": "1", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_use_opa_authz": "false", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_user_max_buckets": "1000", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_user_policies_max_num": "100", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_user_unique_email": "true", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_verify_ssl": "true", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_zone": "", 2024-09-13T07:01:16.999 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_zone_id": "", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_zonegroup": "", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_zonegroup_id": "", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rgwlc_auto_session_clear": "true", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_block_size": "4096", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_cache_size": "536870912", 2024-09-13T07:01:17.000 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_index_type": "binary_search", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_partition_filters": "false", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_perf": "false", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-13T07:01:17.001 INFO:teuthology.orchestra.run.smithi118.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "run_dir": "/var/run/ceph", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_block_create": "true", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_cache_lru_size": "67108864", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_default_max_object_size": "16777216", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_device_size": "53687091200", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_journal_batch_capacity": "16", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_main_device_type": "SSD", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-13T07:01:17.002 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-13T07:01:17.003 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-13T07:01:17.003 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-13T07:01:17.003 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-13T07:01:17.003 INFO:teuthology.orchestra.run.smithi118.stdout: "seastore_segment_size": "67108864", 2024-09-13T07:01:17.003 INFO:teuthology.orchestra.run.smithi118.stdout: "set_keepcaps": "false", 2024-09-13T07:01:17.003 INFO:teuthology.orchestra.run.smithi118.stdout: "setgroup": "", 2024-09-13T07:01:17.003 INFO:teuthology.orchestra.run.smithi118.stdout: "setuser": "", 2024-09-13T07:01:17.003 INFO:teuthology.orchestra.run.smithi118.stdout: "setuser_match_path": "", 2024-09-13T07:01:17.003 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-13T07:01:17.003 INFO:teuthology.orchestra.run.smithi118.stdout: "thp": "false", 2024-09-13T07:01:17.003 INFO:teuthology.orchestra.run.smithi118.stdout: "threadpool_default_timeout": "60", 2024-09-13T07:01:17.003 INFO:teuthology.orchestra.run.smithi118.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-13T07:01:17.003 INFO:teuthology.orchestra.run.smithi118.stdout: "throttler_perf_counter": "true" 2024-09-13T07:01:17.004 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:17.006 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-13T07:01:17.145 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a: {} 2024-09-13T07:01:17.161 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b: {} 2024-09-13T07:01:17.176 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c: {} 2024-09-13T07:01:17.179 INFO:teuthology.orchestra.run.smithi118.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T07:01:17.179 INFO:teuthology.orchestra.run.smithi118.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T07:01:17.179 INFO:teuthology.orchestra.run.smithi118.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T07:01:17.197 DEBUG:teuthology.orchestra.run.smithi118:> type ceph-objectstore-tool 2024-09-13T07:01:17.201 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T07:01:17.201 DEBUG:teuthology.orchestra.run.smithi164:> type ceph-objectstore-tool 2024-09-13T07:01:17.207 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T07:01:17.208 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-13T07:01:17.218 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-13T07:01:17.218 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-13T07:01:17.218 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:01:17.218 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:01:17.218 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-13T07:01:17.218 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-13T07:01:17.219 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-13T07:01:17.219 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-13T07:01:17.219 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:01:17.219 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-13T07:01:17.220 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:01:17.220 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-13T07:01:17.220 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:01:17.221 DEBUG:teuthology.orchestra.run.smithi118:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-13T07:01:17.222 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:17.216+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:17.257 INFO:teuthology.orchestra.run.smithi118.stdout:mapped writes DISABLED 2024-09-13T07:01:17.291 INFO:teuthology.orchestra.run.smithi118.stdout:mapped reads DISABLED 2024-09-13T07:01:17.291 INFO:teuthology.orchestra.run.smithi118.stdout:Seed set to 7023 2024-09-13T07:01:17.321 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:17.316+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:17.340 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:17.340 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:17.340 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:17.340 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:17.351 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:01:17.377 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-13T07:01:17.383 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:17.406 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:17.422 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:17.417+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:17.425 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:17.443 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:17.467 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:17.485 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:17.485 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:17.496 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:01:17.496 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:17.522 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:17.523 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-13T07:01:17.523 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:17.520+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:17.549 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:17.550 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:17.551 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:17.551 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:17.551 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:17.551 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:17.551 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:17.551 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:17.551 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:17.620 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:17.620 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:17.624 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:17.617+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:17.630 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:01:17.689 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-13T07:01:17.725 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:17.721+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:17.734 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:17.734 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:17.744 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:01:17.826 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:17.820+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:17.846 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-13T07:01:17.849 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:17.849 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:17.859 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:01:17.926 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:17.924+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:17.992 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-13T07:01:18.016 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:18.016 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:18.026 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:01:18.027 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:18.024+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:18.128 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:18.128 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:18.128 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:18.121+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:18.137 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:01:18.160 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-13T07:01:18.229 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:18.224+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:18.237 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:18.238 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:18.248 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:01:18.275 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:01:18.330 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:18.324+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:18.353 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:18.353 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:18.362 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:01:18.430 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:18.425+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:18.517 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:18.517 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:18.527 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:01:18.531 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:18.525+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:18.531 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-13T07:01:18.629 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:18.629 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:18.632 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:18.629+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:18.639 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:01:18.733 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:18.728+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:18.736 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:18.736 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:18.746 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:01:18.834 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:18.829+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:18.859 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:18.860 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:18.870 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:01:18.935 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:18.929+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:18.973 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:18.973 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:18.982 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:01:19.035 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:19.029+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:19.085 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:19.085 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:19.095 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:01:19.137 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:19.132+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:19.238 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:19.233+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:19.246 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:19.246 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:19.255 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:01:19.338 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:19.333+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:19.365 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:19.365 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:19.374 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:01:19.440 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:19.436+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:19.478 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:19.479 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:19.489 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:01:19.541 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:19.537+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:19.563 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:01:19.598 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:19.598 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:19.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:01:19.642 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:19.636+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:19.713 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:19.733 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:19.743 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:19.740+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:19.752 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:19.764 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:19.765 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:19.771 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:19.775 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:01:19.795 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:19.820 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:19.841 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:19.843 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:19.837+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:19.868 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:19.870 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:19.870 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:19.870 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:19.871 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:19.871 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:19.871 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:19.871 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:19.871 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:19.879 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [] 2024-09-13T07:01:19.879 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:19.889 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:01:19.945 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:19.941+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:19.993 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:19.993 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:20.002 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:01:20.046 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:20.041+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:20.108 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:20.109 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:20.118 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:01:20.148 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:20.144+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:20.226 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:20.226 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:20.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:20.227 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:20.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.227 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.227 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.231124+0000", 2024-09-13T07:01:20.227 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.992364565999999, 2024-09-13T07:01:20.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010472848, 2024-09-13T07:01:20.227 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.227 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.227 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.227 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.227 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.227 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37 2024-09-13T07:01:20.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.228 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.231124+0000", 2024-09-13T07:01:20.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.231124+0000", 2024-09-13T07:01:20.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.231123+0000", 2024-09-13T07:01:20.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:20.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.231134+0000", 2024-09-13T07:01:20.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0159999999999999e-05 2024-09-13T07:01:20.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.231156+0000", 2024-09-13T07:01:20.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2028999999999999e-05 2024-09-13T07:01:20.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.231197+0000", 2024-09-13T07:01:20.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1229000000000001e-05 2024-09-13T07:01:20.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.239636+0000", 2024-09-13T07:01:20.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084388650000000003 2024-09-13T07:01:20.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.239805+0000", 2024-09-13T07:01:20.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016978199999999999 2024-09-13T07:01:20.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.240004+0000", 2024-09-13T07:01:20.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019858199999999999 2024-09-13T07:01:20.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.241036+0000", 2024-09-13T07:01:20.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001031606 2024-09-13T07:01:20.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.241527+0000", 2024-09-13T07:01:20.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049138100000000004 2024-09-13T07:01:20.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.241566+0000", 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8646999999999998e-05 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.241597+0000", 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1445000000000001e-05 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.233 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.512821+0000", 2024-09-13T07:01:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.710667493999999, 2024-09-13T07:01:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012539794, 2024-09-13T07:01:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-13T07:01:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.512821+0000", 2024-09-13T07:01:20.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.512821+0000", 2024-09-13T07:01:20.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.512820+0000", 2024-09-13T07:01:20.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.512972+0000", 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015206900000000001 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.512993+0000", 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0993e-05 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513022+0000", 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8656000000000001e-05 2024-09-13T07:01:20.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.519544+0000", 2024-09-13T07:01:20.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065228389999999999 2024-09-13T07:01:20.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.519691+0000", 2024-09-13T07:01:20.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014618000000000001 2024-09-13T07:01:20.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.519917+0000", 2024-09-13T07:01:20.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000226185 2024-09-13T07:01:20.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.523679+0000", 2024-09-13T07:01:20.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003762086 2024-09-13T07:01:20.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525217+0000", 2024-09-13T07:01:20.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001537846 2024-09-13T07:01:20.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525307+0000", 2024-09-13T07:01:20.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0531000000000004e-05 2024-09-13T07:01:20.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525361+0000", 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3851999999999999e-05 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513067+0000", 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.710422247, 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013413369, 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.241 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-13T07:01:20.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.241 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513067+0000", 2024-09-13T07:01:20.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513067+0000", 2024-09-13T07:01:20.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513065+0000", 2024-09-13T07:01:20.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:20.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513078+0000", 2024-09-13T07:01:20.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2819999999999999e-05 2024-09-13T07:01:20.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513105+0000", 2024-09-13T07:01:20.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7501999999999999e-05 2024-09-13T07:01:20.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513129+0000", 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4392999999999998e-05 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.521691+0000", 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0085619879999999995 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.521810+0000", 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118775 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.521988+0000", 2024-09-13T07:01:20.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017780000000000001 2024-09-13T07:01:20.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525380+0000", 2024-09-13T07:01:20.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003392445 2024-09-13T07:01:20.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526310+0000", 2024-09-13T07:01:20.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00092912300000000001 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526396+0000", 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6163999999999997e-05 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526480+0000", 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4215000000000006e-05 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.247 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.247 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513148+0000", 2024-09-13T07:01:20.247 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.710341061000001, 2024-09-13T07:01:20.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013731426999999999, 2024-09-13T07:01:20.247 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.247 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.249 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-13T07:01:20.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.249 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513148+0000", 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513148+0000", 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513147+0000", 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:20.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513157+0000", 2024-09-13T07:01:20.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0686000000000001e-05 2024-09-13T07:01:20.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513180+0000", 2024-09-13T07:01:20.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2160000000000001e-05 2024-09-13T07:01:20.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513193+0000", 2024-09-13T07:01:20.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-13T07:01:20.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.522568+0000", 2024-09-13T07:01:20.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093749149999999993 2024-09-13T07:01:20.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.522733+0000", 2024-09-13T07:01:20.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016480999999999999 2024-09-13T07:01:20.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.522992+0000", 2024-09-13T07:01:20.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000259258 2024-09-13T07:01:20.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526515+0000", 2024-09-13T07:01:20.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035229509999999999 2024-09-13T07:01:20.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526680+0000", 2024-09-13T07:01:20.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016493000000000001 2024-09-13T07:01:20.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526811+0000", 2024-09-13T07:01:20.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013147199999999999 2024-09-13T07:01:20.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526879+0000", 2024-09-13T07:01:20.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7836000000000006e-05 2024-09-13T07:01:20.254 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.254 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.254 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513206+0000", 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.710282566, 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014937498, 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.255 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513206+0000", 2024-09-13T07:01:20.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513206+0000", 2024-09-13T07:01:20.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513205+0000", 2024-09-13T07:01:20.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513444+0000", 2024-09-13T07:01:20.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000238989 2024-09-13T07:01:20.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513477+0000", 2024-09-13T07:01:20.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3118999999999999e-05 2024-09-13T07:01:20.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513508+0000", 2024-09-13T07:01:20.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1259e-05 2024-09-13T07:01:20.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.523747+0000", 2024-09-13T07:01:20.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010238858999999999 2024-09-13T07:01:20.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.523925+0000", 2024-09-13T07:01:20.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000177846 2024-09-13T07:01:20.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.524169+0000", 2024-09-13T07:01:20.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000243818 2024-09-13T07:01:20.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.527925+0000", 2024-09-13T07:01:20.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037561029999999998 2024-09-13T07:01:20.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528043+0000", 2024-09-13T07:01:20.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011843 2024-09-13T07:01:20.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528097+0000", 2024-09-13T07:01:20.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3612000000000001e-05 2024-09-13T07:01:20.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528144+0000", 2024-09-13T07:01:20.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7057e-05 2024-09-13T07:01:20.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.262 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.262 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.262 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513585+0000", 2024-09-13T07:01:20.262 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.709904303999998, 2024-09-13T07:01:20.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014919218, 2024-09-13T07:01:20.262 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.262 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.263 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-13T07:01:20.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.263 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513585+0000", 2024-09-13T07:01:20.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513585+0000", 2024-09-13T07:01:20.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513583+0000", 2024-09-13T07:01:20.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:20.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513595+0000", 2024-09-13T07:01:20.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1878999999999999e-05 2024-09-13T07:01:20.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513612+0000", 2024-09-13T07:01:20.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.721e-05 2024-09-13T07:01:20.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513639+0000", 2024-09-13T07:01:20.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6733e-05 2024-09-13T07:01:20.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525467+0000", 2024-09-13T07:01:20.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828200000000001 2024-09-13T07:01:20.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525618+0000", 2024-09-13T07:01:20.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015065 2024-09-13T07:01:20.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525783+0000", 2024-09-13T07:01:20.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016588099999999999 2024-09-13T07:01:20.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528270+0000", 2024-09-13T07:01:20.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024861330000000002 2024-09-13T07:01:20.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528292+0000", 2024-09-13T07:01:20.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2127e-05 2024-09-13T07:01:20.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528405+0000", 2024-09-13T07:01:20.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113583 2024-09-13T07:01:20.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528504+0000", 2024-09-13T07:01:20.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8492000000000005e-05 2024-09-13T07:01:20.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.268 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.268 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.269 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:20.269 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.709823906, 2024-09-13T07:01:20.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018207798000000001, 2024-09-13T07:01:20.269 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.269 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.269 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-13T07:01:20.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513664+0000", 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513674+0000", 2024-09-13T07:01:20.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8770000000000005e-06 2024-09-13T07:01:20.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513695+0000", 2024-09-13T07:01:20.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1319e-05 2024-09-13T07:01:20.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513714+0000", 2024-09-13T07:01:20.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.914e-05 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526929+0000", 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013215146000000001 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.527080+0000", 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000150824 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.527312+0000", 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023218100000000001 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530679+0000", 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003366529 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531659+0000", 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0009803940000000001 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531784+0000", 2024-09-13T07:01:20.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012519399999999999 2024-09-13T07:01:20.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531873+0000", 2024-09-13T07:01:20.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8307999999999995e-05 2024-09-13T07:01:20.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.274 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.709761637, 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018374253, 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:20.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:20.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513726+0000", 2024-09-13T07:01:20.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513933+0000", 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020722500000000001 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513984+0000", 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0719e-05 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513999+0000", 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4877e-05 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528581+0000", 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014581604999999999 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528674+0000", 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3021000000000003e-05 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528803+0000", 2024-09-13T07:01:20.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012964199999999999 2024-09-13T07:01:20.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531891+0000", 2024-09-13T07:01:20.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00308764 2024-09-13T07:01:20.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531990+0000", 2024-09-13T07:01:20.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9407999999999994e-05 2024-09-13T07:01:20.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532064+0000", 2024-09-13T07:01:20.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3793000000000003e-05 2024-09-13T07:01:20.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532101+0000", 2024-09-13T07:01:20.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7357000000000002e-05 2024-09-13T07:01:20.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.281 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.281 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.281 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:20.281 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.709472378000001, 2024-09-13T07:01:20.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018326366, 2024-09-13T07:01:20.281 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.281 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.281 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.282 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-13T07:01:20.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.282 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:20.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514015+0000", 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514026+0000", 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1053000000000001e-05 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514047+0000", 2024-09-13T07:01:20.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0687e-05 2024-09-13T07:01:20.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514065+0000", 2024-09-13T07:01:20.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8799000000000001e-05 2024-09-13T07:01:20.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529495+0000", 2024-09-13T07:01:20.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015429877 2024-09-13T07:01:20.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529642+0000", 2024-09-13T07:01:20.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014682399999999999 2024-09-13T07:01:20.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529871+0000", 2024-09-13T07:01:20.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022936399999999999 2024-09-13T07:01:20.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532183+0000", 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023112889999999998 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532199+0000", 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6082e-05 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532269+0000", 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9812000000000002e-05 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532343+0000", 2024-09-13T07:01:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4141999999999996e-05 2024-09-13T07:01:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.709410141999999, 2024-09-13T07:01:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018420999, 2024-09-13T07:01:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.287 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:20.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514078+0000", 2024-09-13T07:01:20.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:20.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514088+0000", 2024-09-13T07:01:20.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0433e-05 2024-09-13T07:01:20.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514104+0000", 2024-09-13T07:01:20.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5616000000000001e-05 2024-09-13T07:01:20.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514133+0000", 2024-09-13T07:01:20.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9453000000000001e-05 2024-09-13T07:01:20.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530751+0000", 2024-09-13T07:01:20.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016617996999999999 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530926+0000", 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017493599999999999 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531174+0000", 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024812100000000002 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532358+0000", 2024-09-13T07:01:20.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00118396 2024-09-13T07:01:20.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532421+0000", 2024-09-13T07:01:20.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2749999999999994e-05 2024-09-13T07:01:20.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532467+0000", 2024-09-13T07:01:20.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6427000000000001e-05 2024-09-13T07:01:20.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532500+0000", 2024-09-13T07:01:20.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2329000000000002e-05 2024-09-13T07:01:20.293 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.293 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.293 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.293 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.293 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.070435+0000", 2024-09-13T07:01:20.293 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.153054013999999, 2024-09-13T07:01:20.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010104053, 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 132 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.070435+0000", 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.070435+0000", 2024-09-13T07:01:20.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.070434+0000", 2024-09-13T07:01:20.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:20.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.070441+0000", 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.793e-06 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.070447+0000", 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1659999999999999e-06 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.070452+0000", 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6280000000000004e-06 2024-09-13T07:01:20.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.072964+0000", 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002512313 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.073032+0000", 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7399999999999998e-05 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.073142+0000", 2024-09-13T07:01:20.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110297 2024-09-13T07:01:20.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.075835+0000", 2024-09-13T07:01:20.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026933130000000001 2024-09-13T07:01:20.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.080391+0000", 2024-09-13T07:01:20.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00455559 2024-09-13T07:01:20.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.080459+0000", 2024-09-13T07:01:20.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8373000000000004e-05 2024-09-13T07:01:20.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.080539+0000", 2024-09-13T07:01:20.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9781000000000001e-05 2024-09-13T07:01:20.299 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.299 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.299 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.071013+0000", 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.152476065999998, 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010087227000000001, 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 135 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071013+0000", 2024-09-13T07:01:20.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071013+0000", 2024-09-13T07:01:20.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071012+0000", 2024-09-13T07:01:20.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:20.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071016+0000", 2024-09-13T07:01:20.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3510000000000002e-06 2024-09-13T07:01:20.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071020+0000", 2024-09-13T07:01:20.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7650000000000001e-06 2024-09-13T07:01:20.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071024+0000", 2024-09-13T07:01:20.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7060000000000002e-06 2024-09-13T07:01:20.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.075271+0000", 2024-09-13T07:01:20.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042465599999999999 2024-09-13T07:01:20.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.075332+0000", 2024-09-13T07:01:20.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1242999999999996e-05 2024-09-13T07:01:20.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.075427+0000", 2024-09-13T07:01:20.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4701e-05 2024-09-13T07:01:20.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.077983+0000", 2024-09-13T07:01:20.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002556811 2024-09-13T07:01:20.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.081013+0000", 2024-09-13T07:01:20.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030294219999999999 2024-09-13T07:01:20.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.081050+0000", 2024-09-13T07:01:20.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7187999999999997e-05 2024-09-13T07:01:20.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.081100+0000", 2024-09-13T07:01:20.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9925000000000001e-05 2024-09-13T07:01:20.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.305 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.071028+0000", 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.152460901000001, 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011230551, 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 136 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071028+0000", 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071028+0000", 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071028+0000", 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:20.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071031+0000", 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1810000000000001e-06 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071035+0000", 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5129999999999998e-06 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071044+0000", 2024-09-13T07:01:20.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2679999999999994e-06 2024-09-13T07:01:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.075902+0000", 2024-09-13T07:01:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048583050000000003 2024-09-13T07:01:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.075969+0000", 2024-09-13T07:01:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7552000000000006e-05 2024-09-13T07:01:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.076084+0000", 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114164 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.080975+0000", 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048912460000000001 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082155+0000", 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001180277 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082201+0000", 2024-09-13T07:01:20.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5958999999999998e-05 2024-09-13T07:01:20.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082258+0000", 2024-09-13T07:01:20.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7410000000000001e-05 2024-09-13T07:01:20.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.311 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.312 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.312 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.071106+0000", 2024-09-13T07:01:20.312 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.152382638999999, 2024-09-13T07:01:20.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011381179999999999, 2024-09-13T07:01:20.312 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.312 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.312 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 137 2024-09-13T07:01:20.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.312 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071106+0000", 2024-09-13T07:01:20.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071106+0000", 2024-09-13T07:01:20.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071106+0000", 2024-09-13T07:01:20.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:20.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071197+0000", 2024-09-13T07:01:20.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1367999999999997e-05 2024-09-13T07:01:20.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071205+0000", 2024-09-13T07:01:20.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6769999999999995e-06 2024-09-13T07:01:20.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071210+0000", 2024-09-13T07:01:20.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.135e-06 2024-09-13T07:01:20.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.076472+0000", 2024-09-13T07:01:20.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052622579999999997 2024-09-13T07:01:20.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.076578+0000", 2024-09-13T07:01:20.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010632999999999999 2024-09-13T07:01:20.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.076721+0000", 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014302100000000001 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082014+0000", 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052930720000000002 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082342+0000", 2024-09-13T07:01:20.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000327208 2024-09-13T07:01:20.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082408+0000", 2024-09-13T07:01:20.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5889000000000006e-05 2024-09-13T07:01:20.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082487+0000", 2024-09-13T07:01:20.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9857999999999993e-05 2024-09-13T07:01:20.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.317 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.318 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.318 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.071216+0000", 2024-09-13T07:01:20.318 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.152273145999999, 2024-09-13T07:01:20.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011448228, 2024-09-13T07:01:20.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.318 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-13T07:01:20.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071216+0000", 2024-09-13T07:01:20.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071216+0000", 2024-09-13T07:01:20.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071215+0000", 2024-09-13T07:01:20.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:20.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071219+0000", 2024-09-13T07:01:20.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4570000000000001e-06 2024-09-13T07:01:20.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071223+0000", 2024-09-13T07:01:20.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5120000000000002e-06 2024-09-13T07:01:20.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071227+0000", 2024-09-13T07:01:20.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1509999999999997e-06 2024-09-13T07:01:20.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.077362+0000", 2024-09-13T07:01:20.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061349619999999999 2024-09-13T07:01:20.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.077445+0000", 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2330000000000006e-05 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.077576+0000", 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013119100000000001 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082509+0000", 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049333729999999996 2024-09-13T07:01:20.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082575+0000", 2024-09-13T07:01:20.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5507999999999997e-05 2024-09-13T07:01:20.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082612+0000", 2024-09-13T07:01:20.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7058000000000003e-05 2024-09-13T07:01:20.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082664+0000", 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2188000000000002e-05 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.071232+0000", 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.152257228, 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011600325999999999, 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.325 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-13T07:01:20.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.325 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071232+0000", 2024-09-13T07:01:20.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071232+0000", 2024-09-13T07:01:20.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071231+0000", 2024-09-13T07:01:20.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:20.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071234+0000", 2024-09-13T07:01:20.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9160000000000001e-06 2024-09-13T07:01:20.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071239+0000", 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4830000000000001e-06 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071243+0000", 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9079999999999997e-06 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.078327+0000", 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070842199999999996 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.078438+0000", 2024-09-13T07:01:20.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111397 2024-09-13T07:01:20.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.078591+0000", 2024-09-13T07:01:20.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000153034 2024-09-13T07:01:20.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082518+0000", 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039266989999999996 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082730+0000", 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021233999999999999 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082788+0000", 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7271000000000001e-05 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082832+0000", 2024-09-13T07:01:20.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4384000000000001e-05 2024-09-13T07:01:20.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.330 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.330 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.330 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.071306+0000", 2024-09-13T07:01:20.330 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.152183136000001, 2024-09-13T07:01:20.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011684793000000001, 2024-09-13T07:01:20.331 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.331 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.331 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-13T07:01:20.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.331 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071306+0000", 2024-09-13T07:01:20.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071306+0000", 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071305+0000", 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071400+0000", 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4734999999999993e-05 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071408+0000", 2024-09-13T07:01:20.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9789999999999993e-06 2024-09-13T07:01:20.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071413+0000", 2024-09-13T07:01:20.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1170000000000003e-06 2024-09-13T07:01:20.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.079114+0000", 2024-09-13T07:01:20.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077009069999999999 2024-09-13T07:01:20.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.079215+0000", 2024-09-13T07:01:20.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101072 2024-09-13T07:01:20.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.079361+0000", 2024-09-13T07:01:20.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146002 2024-09-13T07:01:20.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082530+0000", 2024-09-13T07:01:20.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031688290000000002 2024-09-13T07:01:20.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082884+0000", 2024-09-13T07:01:20.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000354662 2024-09-13T07:01:20.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082939+0000", 2024-09-13T07:01:20.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4768000000000001e-05 2024-09-13T07:01:20.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082990+0000", 2024-09-13T07:01:20.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1348999999999999e-05 2024-09-13T07:01:20.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.337 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.337 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:20.337 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.071419+0000", 2024-09-13T07:01:20.337 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.152070239, 2024-09-13T07:01:20.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012118689, 2024-09-13T07:01:20.338 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.338 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.338 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-13T07:01:20.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.338 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071419+0000", 2024-09-13T07:01:20.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071419+0000", 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071418+0000", 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071422+0000", 2024-09-13T07:01:20.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7740000000000002e-06 2024-09-13T07:01:20.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071427+0000", 2024-09-13T07:01:20.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6480000000000002e-06 2024-09-13T07:01:20.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071431+0000", 2024-09-13T07:01:20.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0980000000000004e-06 2024-09-13T07:01:20.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.081143+0000", 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097121129999999996 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.081240+0000", 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6935000000000006e-05 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.081445+0000", 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020548300000000001 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.083059+0000", 2024-09-13T07:01:20.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016134840000000001 2024-09-13T07:01:20.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.083408+0000", 2024-09-13T07:01:20.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00034926199999999998 2024-09-13T07:01:20.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.083491+0000", 2024-09-13T07:01:20.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3140999999999996e-05 2024-09-13T07:01:20.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.083537+0000", 2024-09-13T07:01:20.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6193999999999999e-05 2024-09-13T07:01:20.343 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.343 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.343 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.343 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:01:20.343 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.850389+0000", 2024-09-13T07:01:20.343 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.373099824000001, 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01082141, 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850389+0000", 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850389+0000", 2024-09-13T07:01:20.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850388+0000", 2024-09-13T07:01:20.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:20.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850570+0000", 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000182299 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850596+0000", 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5582999999999999e-05 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850610+0000", 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4209999999999999e-05 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.857529+0000", 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069189580000000002 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.857676+0000", 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146797 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.857926+0000", 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024991199999999997 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.860954+0000", 2024-09-13T07:01:20.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030283889999999998 2024-09-13T07:01:20.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.861011+0000", 2024-09-13T07:01:20.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6783999999999997e-05 2024-09-13T07:01:20.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.861129+0000", 2024-09-13T07:01:20.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117738 2024-09-13T07:01:20.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.861210+0000", 2024-09-13T07:01:20.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1760000000000003e-05 2024-09-13T07:01:20.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.349 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.349 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:01:20.349 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.850631+0000", 2024-09-13T07:01:20.349 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.372858226000002, 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011927366, 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850631+0000", 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:20.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850631+0000", 2024-09-13T07:01:20.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:20.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:20.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850630+0000", 2024-09-13T07:01:20.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:20.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:20.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850640+0000", 2024-09-13T07:01:20.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0193e-05 2024-09-13T07:01:20.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:20.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850680+0000", 2024-09-13T07:01:20.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9818999999999999e-05 2024-09-13T07:01:20.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850694+0000", 2024-09-13T07:01:20.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4752999999999999e-05 2024-09-13T07:01:20.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:20.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.860129+0000", 2024-09-13T07:01:20.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094350860000000005 2024-09-13T07:01:20.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:20.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.860260+0000", 2024-09-13T07:01:20.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013028900000000001 2024-09-13T07:01:20.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:20.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.860400+0000", 2024-09-13T07:01:20.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014016300000000001 2024-09-13T07:01:20.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:20.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.861852+0000", 2024-09-13T07:01:20.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001452231 2024-09-13T07:01:20.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:20.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.862448+0000", 2024-09-13T07:01:20.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00059627400000000004 2024-09-13T07:01:20.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.862494+0000", 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5785000000000003e-05 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.862558+0000", 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3806000000000005e-05 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:20.355 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:20.356 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:01:20.357 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:20.352+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:20.358 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:01:20.458 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:20.452+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:20.471 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:20.471 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:20.471 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:20.471 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:20.481 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:01:20.559 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:20.557+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:20.580 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:20.580 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:20.580 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:20.580 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:20.580 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:20.589 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:01:20.659 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:20.656+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:20.701 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:20.701 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:20.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:20.701 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:20.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.701 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:20 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'10, mlcod=15'8)", 2024-09-13T07:01:20.701 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:02.215529+0000", 2024-09-13T07:01:20.701 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.483967961000001, 2024-09-13T07:01:20.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028636009999999999, 2024-09-13T07:01:20.701 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.701 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.215529+0000", 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.215529+0000", 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.215527+0000", 2024-09-13T07:01:20.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:20.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.215733+0000", 2024-09-13T07:01:20.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020635200000000001 2024-09-13T07:01:20.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.215763+0000", 2024-09-13T07:01:20.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9393000000000001e-05 2024-09-13T07:01:20.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.215775+0000", 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1950000000000001e-05 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.215884+0000", 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109281 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.215920+0000", 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6015e-05 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.218295+0000", 2024-09-13T07:01:20.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023754029999999999 2024-09-13T07:01:20.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.218392+0000", 2024-09-13T07:01:20.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7060999999999998e-05 2024-09-13T07:01:20.705 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.705 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.705 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:32 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'18, mlcod=15'14)", 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:02.234332+0000", 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.465164467000001, 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025821339999999998, 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.234332+0000", 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.234332+0000", 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.234331+0000", 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.234511+0000", 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018037199999999999 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.234537+0000", 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6086000000000001e-05 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.234549+0000", 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1637000000000001e-05 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.234624+0000", 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5315000000000007e-05 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.234664+0000", 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9928999999999997e-05 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.236757+0000", 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002093503 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.236915+0000", 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000157069 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.709 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'16)", 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:02.237242+0000", 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.462254553000001, 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026978319999999998, 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.237242+0000", 2024-09-13T07:01:20.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.237242+0000", 2024-09-13T07:01:20.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.237241+0000", 2024-09-13T07:01:20.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.237254+0000", 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3179999999999999e-05 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.237284+0000", 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0372e-05 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.237306+0000", 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2158999999999999e-05 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.237390+0000", 2024-09-13T07:01:20.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3292000000000001e-05 2024-09-13T07:01:20.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.237431+0000", 2024-09-13T07:01:20.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1009e-05 2024-09-13T07:01:20.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.239860+0000", 2024-09-13T07:01:20.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024293119999999999 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.239940+0000", 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0154999999999994e-05 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'16)", 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:02.238448+0000", 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.461048754, 2024-09-13T07:01:20.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026117599999999999, 2024-09-13T07:01:20.715 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.715 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.715 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.238448+0000", 2024-09-13T07:01:20.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.238448+0000", 2024-09-13T07:01:20.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.238447+0000", 2024-09-13T07:01:20.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.238542+0000", 2024-09-13T07:01:20.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5713999999999999e-05 2024-09-13T07:01:20.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.238569+0000", 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6933000000000001e-05 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.238584+0000", 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4293e-05 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.238671+0000", 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6921000000000003e-05 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.238727+0000", 2024-09-13T07:01:20.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6609e-05 2024-09-13T07:01:20.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.240982+0000", 2024-09-13T07:01:20.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022546319999999999 2024-09-13T07:01:20.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.241060+0000", 2024-09-13T07:01:20.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8051000000000002e-05 2024-09-13T07:01:20.718 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:54 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'32, mlcod=15'29)", 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:02.266904+0000", 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.432593255, 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025414740000000002, 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.266904+0000", 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.266904+0000", 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.266902+0000", 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.266925+0000", 2024-09-13T07:01:20.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2578999999999998e-05 2024-09-13T07:01:20.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.266945+0000", 2024-09-13T07:01:20.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0854000000000001e-05 2024-09-13T07:01:20.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.266955+0000", 2024-09-13T07:01:20.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0063e-05 2024-09-13T07:01:20.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.267057+0000", 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101574 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.267097+0000", 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9677e-05 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.269280+0000", 2024-09-13T07:01:20.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021829280000000002 2024-09-13T07:01:20.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.269445+0000", 2024-09-13T07:01:20.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016545899999999999 2024-09-13T07:01:20.723 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.723 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.723 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.723 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:55 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-13T07:01:20.723 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:02.267982+0000", 2024-09-13T07:01:20.723 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.431515046000001, 2024-09-13T07:01:20.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028725449999999998, 2024-09-13T07:01:20.724 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.724 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.724 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.267982+0000", 2024-09-13T07:01:20.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.267982+0000", 2024-09-13T07:01:20.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.267980+0000", 2024-09-13T07:01:20.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.268109+0000", 2024-09-13T07:01:20.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012825400000000001 2024-09-13T07:01:20.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.268128+0000", 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9247999999999999e-05 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.268168+0000", 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0167999999999999e-05 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.268308+0000", 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000140166 2024-09-13T07:01:20.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.268372+0000", 2024-09-13T07:01:20.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3322000000000001e-05 2024-09-13T07:01:20.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.270749+0000", 2024-09-13T07:01:20.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023773000000000002 2024-09-13T07:01:20.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.270854+0000", 2024-09-13T07:01:20.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105432 2024-09-13T07:01:20.728 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.728 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.728 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.728 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:84 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'54, mlcod=15'50)", 2024-09-13T07:01:20.728 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:02.321216+0000", 2024-09-13T07:01:20.728 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.378280977999999, 2024-09-13T07:01:20.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031257720000000002, 2024-09-13T07:01:20.728 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.321216+0000", 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.321216+0000", 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.321214+0000", 2024-09-13T07:01:20.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:20.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.321297+0000", 2024-09-13T07:01:20.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2705000000000002e-05 2024-09-13T07:01:20.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.321323+0000", 2024-09-13T07:01:20.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5942e-05 2024-09-13T07:01:20.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.321339+0000", 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6367000000000001e-05 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.321410+0000", 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0963000000000005e-05 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.321447+0000", 2024-09-13T07:01:20.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6939e-05 2024-09-13T07:01:20.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.324264+0000", 2024-09-13T07:01:20.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002816781 2024-09-13T07:01:20.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.324342+0000", 2024-09-13T07:01:20.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7872000000000003e-05 2024-09-13T07:01:20.732 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.732 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.732 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.733 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:87 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'51)", 2024-09-13T07:01:20.733 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:02.324260+0000", 2024-09-13T07:01:20.733 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.375237164000001, 2024-09-13T07:01:20.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026402700000000001, 2024-09-13T07:01:20.733 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.733 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.733 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.324260+0000", 2024-09-13T07:01:20.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.324260+0000", 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.324258+0000", 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.324364+0000", 2024-09-13T07:01:20.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105612 2024-09-13T07:01:20.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.324387+0000", 2024-09-13T07:01:20.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3567999999999999e-05 2024-09-13T07:01:20.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.324397+0000", 2024-09-13T07:01:20.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7429999999999997e-06 2024-09-13T07:01:20.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.324490+0000", 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3165999999999995e-05 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.324526+0000", 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5382000000000001e-05 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.326765+0000", 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022394210000000001 2024-09-13T07:01:20.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:02.326900+0000", 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013484700000000001 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:130 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'88, mlcod=15'85)", 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:03.072724+0000", 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.626773268000001, 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055021599999999999, 2024-09-13T07:01:20.737 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.738 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.738 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.072724+0000", 2024-09-13T07:01:20.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.072724+0000", 2024-09-13T07:01:20.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.072722+0000", 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.072734+0000", 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1673e-05 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.072759+0000", 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5009000000000001e-05 2024-09-13T07:01:20.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.072774+0000", 2024-09-13T07:01:20.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4953000000000001e-05 2024-09-13T07:01:20.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.072914+0000", 2024-09-13T07:01:20.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014013099999999999 2024-09-13T07:01:20.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.072970+0000", 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5775e-05 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078152+0000", 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0051826870000000001 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078226+0000", 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3318000000000005e-05 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.741 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:131 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'89, mlcod=15'85)", 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:03.073514+0000", 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.625982788999998, 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047443720000000002, 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.073514+0000", 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.073514+0000", 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.073512+0000", 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.073730+0000", 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021731499999999999 2024-09-13T07:01:20.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.073761+0000", 2024-09-13T07:01:20.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1816000000000002e-05 2024-09-13T07:01:20.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.073779+0000", 2024-09-13T07:01:20.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7306e-05 2024-09-13T07:01:20.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.073998+0000", 2024-09-13T07:01:20.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219158 2024-09-13T07:01:20.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.074051+0000", 2024-09-13T07:01:20.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2618999999999999e-05 2024-09-13T07:01:20.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078233+0000", 2024-09-13T07:01:20.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0041824310000000003 2024-09-13T07:01:20.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078258+0000", 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5420000000000001e-05 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:132 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'90, mlcod=15'85)", 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:03.074487+0000", 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.625010314000001, 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005197821, 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.746 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.074487+0000", 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.074487+0000", 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.074486+0000", 2024-09-13T07:01:20.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:20.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.074502+0000", 2024-09-13T07:01:20.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6463000000000001e-05 2024-09-13T07:01:20.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.074516+0000", 2024-09-13T07:01:20.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3842999999999999e-05 2024-09-13T07:01:20.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.074527+0000", 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0726e-05 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.075388+0000", 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00086130099999999997 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.075446+0000", 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8075000000000002e-05 2024-09-13T07:01:20.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079637+0000", 2024-09-13T07:01:20.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0041912240000000003 2024-09-13T07:01:20.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079684+0000", 2024-09-13T07:01:20.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7055999999999998e-05 2024-09-13T07:01:20.750 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.750 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.750 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.751 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:133 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'91, mlcod=15'86)", 2024-09-13T07:01:20.751 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:03.075369+0000", 2024-09-13T07:01:20.751 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.624127523999999, 2024-09-13T07:01:20.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043449159999999999, 2024-09-13T07:01:20.751 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.751 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.751 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.075369+0000", 2024-09-13T07:01:20.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.075369+0000", 2024-09-13T07:01:20.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.075368+0000", 2024-09-13T07:01:20.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:20.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.075381+0000", 2024-09-13T07:01:20.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.288e-05 2024-09-13T07:01:20.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.075393+0000", 2024-09-13T07:01:20.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1826e-05 2024-09-13T07:01:20.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.075401+0000", 2024-09-13T07:01:20.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2130000000000008e-06 2024-09-13T07:01:20.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.076555+0000", 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011537240000000001 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.076616+0000", 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1155000000000007e-05 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079691+0000", 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030753529999999999 2024-09-13T07:01:20.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079714+0000", 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2884999999999999e-05 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:134 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'92, mlcod=15'86)", 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:03.075986+0000", 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.623510866, 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0053847959999999999, 2024-09-13T07:01:20.755 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.756 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.756 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.075986+0000", 2024-09-13T07:01:20.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.075986+0000", 2024-09-13T07:01:20.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.075985+0000", 2024-09-13T07:01:20.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.076071+0000", 2024-09-13T07:01:20.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6712999999999995e-05 2024-09-13T07:01:20.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.076131+0000", 2024-09-13T07:01:20.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0124000000000002e-05 2024-09-13T07:01:20.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.076146+0000", 2024-09-13T07:01:20.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4535999999999999e-05 2024-09-13T07:01:20.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.077321+0000", 2024-09-13T07:01:20.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011745060000000001 2024-09-13T07:01:20.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.077347+0000", 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6953000000000001e-05 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.081310+0000", 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039627489999999998 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.081371+0000", 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0560999999999998e-05 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.759 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.760 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.760 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:135 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'93, mlcod=15'86)", 2024-09-13T07:01:20.760 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:03.076787+0000", 2024-09-13T07:01:20.760 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.62270955, 2024-09-13T07:01:20.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0046446710000000004, 2024-09-13T07:01:20.760 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.760 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.760 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.076787+0000", 2024-09-13T07:01:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.076787+0000", 2024-09-13T07:01:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.076786+0000", 2024-09-13T07:01:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.076800+0000", 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4663e-05 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.076822+0000", 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1168000000000001e-05 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.076848+0000", 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6517e-05 2024-09-13T07:01:20.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078332+0000", 2024-09-13T07:01:20.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014835130000000001 2024-09-13T07:01:20.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078379+0000", 2024-09-13T07:01:20.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7828999999999999e-05 2024-09-13T07:01:20.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.081381+0000", 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030017300000000002 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.081432+0000", 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0736999999999999e-05 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:136 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'94, mlcod=15'86)", 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:03.077453+0000", 2024-09-13T07:01:20.764 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.622043541, 2024-09-13T07:01:20.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0051052520000000002, 2024-09-13T07:01:20.765 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.765 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.765 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.077453+0000", 2024-09-13T07:01:20.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.077453+0000", 2024-09-13T07:01:20.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.077452+0000", 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.077469+0000", 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7159999999999998e-05 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.077485+0000", 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6178999999999999e-05 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.077495+0000", 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079101+0000", 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001605333 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079126+0000", 2024-09-13T07:01:20.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4956e-05 2024-09-13T07:01:20.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.082520+0000", 2024-09-13T07:01:20.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033946290000000001 2024-09-13T07:01:20.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.082559+0000", 2024-09-13T07:01:20.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8325000000000003e-05 2024-09-13T07:01:20.768 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:137 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'95, mlcod=15'86)", 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:03.078177+0000", 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.621320136000001, 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043947049999999996, 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078177+0000", 2024-09-13T07:01:20.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078177+0000", 2024-09-13T07:01:20.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078175+0000", 2024-09-13T07:01:20.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078319+0000", 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000144043 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078345+0000", 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5982e-05 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078363+0000", 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7433999999999999e-05 2024-09-13T07:01:20.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079774+0000", 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014117629999999999 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079811+0000", 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6384000000000003e-05 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.082560+0000", 2024-09-13T07:01:20.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002749427 2024-09-13T07:01:20.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.082571+0000", 2024-09-13T07:01:20.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1151999999999999e-05 2024-09-13T07:01:20.773 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.773 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.773 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.773 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:138 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'96, mlcod=15'86)", 2024-09-13T07:01:20.773 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:03.078936+0000", 2024-09-13T07:01:20.773 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.620560666999999, 2024-09-13T07:01:20.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036492030000000002, 2024-09-13T07:01:20.774 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.774 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.774 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078936+0000", 2024-09-13T07:01:20.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078936+0000", 2024-09-13T07:01:20.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078934+0000", 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078949+0000", 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4786e-05 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078978+0000", 2024-09-13T07:01:20.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8962000000000002e-05 2024-09-13T07:01:20.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.078989+0000", 2024-09-13T07:01:20.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0791000000000001e-05 2024-09-13T07:01:20.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.080815+0000", 2024-09-13T07:01:20.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001826076 2024-09-13T07:01:20.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.080887+0000", 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2674000000000002e-05 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.082576+0000", 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00168801 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.082585+0000", 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8670000000000006e-06 2024-09-13T07:01:20.777 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:139 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'97, mlcod=15'88)", 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:03.079930+0000", 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.619567011000001, 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030235409999999998, 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079930+0000", 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079930+0000", 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079929+0000", 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079939+0000", 2024-09-13T07:01:20.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.01e-05 2024-09-13T07:01:20.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079956+0000", 2024-09-13T07:01:20.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7136999999999999e-05 2024-09-13T07:01:20.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.079968+0000", 2024-09-13T07:01:20.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1702000000000001e-05 2024-09-13T07:01:20.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.081499+0000", 2024-09-13T07:01:20.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001530986 2024-09-13T07:01:20.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.081532+0000", 2024-09-13T07:01:20.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3562000000000002e-05 2024-09-13T07:01:20.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.082888+0000", 2024-09-13T07:01:20.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001355694 2024-09-13T07:01:20.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.082953+0000", 2024-09-13T07:01:20.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5507999999999997e-05 2024-09-13T07:01:20.782 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.782 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.782 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.782 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:140 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'98, mlcod=15'88)", 2024-09-13T07:01:20.782 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:03.080746+0000", 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.618750671000001, 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028177660000000002, 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.080746+0000", 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.080746+0000", 2024-09-13T07:01:20.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.080745+0000", 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.080831+0000", 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6335000000000006e-05 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.080888+0000", 2024-09-13T07:01:20.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6963999999999999e-05 2024-09-13T07:01:20.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.080898+0000", 2024-09-13T07:01:20.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0557000000000001e-05 2024-09-13T07:01:20.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.082092+0000", 2024-09-13T07:01:20.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011940099999999999 2024-09-13T07:01:20.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.082106+0000", 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3023000000000001e-05 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.083523+0000", 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014175609999999999 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.083564+0000", 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.087e-05 2024-09-13T07:01:20.786 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4166.0:160 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'114, mlcod=16'112)", 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:03.865927+0000", 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 16.833570192, 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025118889999999998, 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:20.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.865927+0000", 2024-09-13T07:01:20.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:20.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.865927+0000", 2024-09-13T07:01:20.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:20.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:20.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.865925+0000", 2024-09-13T07:01:20.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:20.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.866797+0000", 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000872224 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.866825+0000", 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7430999999999999e-05 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.866854+0000", 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.932e-05 2024-09-13T07:01:20.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.866906+0000", 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1591999999999997e-05 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.866936+0000", 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0213e-05 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.868353+0000", 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014175959999999999 2024-09-13T07:01:20.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:20.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:20.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:20.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:03.868439+0000", 2024-09-13T07:01:20.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5061000000000004e-05 2024-09-13T07:01:20.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.791 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:20.791 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:20.791 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:20.793 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:01:20.793 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:20.788+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:20.894 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:20.889+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:20.902 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:20.902 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:20.902 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:20.902 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:20.913 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:01:20.994 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:20.992+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:21.011 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:21.012 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:21.021 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:01:21.096 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:21.092+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:21.131 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:21.131 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:21.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:21.132 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:21.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.132 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4226.0:5 2.2 e19/17 2:5cea7035:::rbd_info:head v 19'1, mlcod=0'0)", 2024-09-13T07:01:21.132 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:10.210182+0000", 2024-09-13T07:01:21.132 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.921007585, 2024-09-13T07:01:21.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002068102, 2024-09-13T07:01:21.132 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.132 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:21.132 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:10.210182+0000", 2024-09-13T07:01:21.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:10.210182+0000", 2024-09-13T07:01:21.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:10.210181+0000", 2024-09-13T07:01:21.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:21.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:10.210196+0000", 2024-09-13T07:01:21.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4392999999999999e-05 2024-09-13T07:01:21.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:10.210225+0000", 2024-09-13T07:01:21.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9620000000000001e-05 2024-09-13T07:01:21.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:10.210253+0000", 2024-09-13T07:01:21.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7512e-05 2024-09-13T07:01:21.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:10.210434+0000", 2024-09-13T07:01:21.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000181373 2024-09-13T07:01:21.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:10.210499+0000", 2024-09-13T07:01:21.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4892000000000001e-05 2024-09-13T07:01:21.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:21.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:10.212173+0000", 2024-09-13T07:01:21.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016736629999999999 2024-09-13T07:01:21.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:10.212250+0000", 2024-09-13T07:01:21.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7442999999999997e-05 2024-09-13T07:01:21.137 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.137 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.137 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.137 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4226.0:7 2.2 e20/17 2:5cea7035:::rbd_info:head v 20'2, mlcod=0'0)", 2024-09-13T07:01:21.137 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:11.229017+0000", 2024-09-13T07:01:21.137 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.9021727449999997, 2024-09-13T07:01:21.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016163670000000001, 2024-09-13T07:01:21.138 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.138 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:21.138 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:11.229017+0000", 2024-09-13T07:01:21.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:11.229017+0000", 2024-09-13T07:01:21.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:11.229016+0000", 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:11.229027+0000", 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0747000000000001e-05 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:11.229066+0000", 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8649999999999998e-05 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:11.229083+0000", 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7569999999999999e-05 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:11.229177+0000", 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3466000000000003e-05 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:11.229227+0000", 2024-09-13T07:01:21.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0351e-05 2024-09-13T07:01:21.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:21.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:11.230569+0000", 2024-09-13T07:01:21.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013418 2024-09-13T07:01:21.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:11.230633+0000", 2024-09-13T07:01:21.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4265999999999998e-05 2024-09-13T07:01:21.141 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.141 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.142 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.142 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:7 3.1a e24/22 3:5cea7035:::rbd_info:head v 24'1, mlcod=0'0)", 2024-09-13T07:01:21.142 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:20.279382+0000", 2024-09-13T07:01:21.142 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.85180813799999999, 2024-09-13T07:01:21.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010938149999999999, 2024-09-13T07:01:21.142 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.142 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:21.142 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:20.279382+0000", 2024-09-13T07:01:21.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:20.279382+0000", 2024-09-13T07:01:21.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:20.279381+0000", 2024-09-13T07:01:21.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:20.279390+0000", 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2560000000000008e-06 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:20.279413+0000", 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2957999999999999e-05 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:20.279424+0000", 2024-09-13T07:01:21.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0917999999999999e-05 2024-09-13T07:01:21.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:20.279506+0000", 2024-09-13T07:01:21.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1841999999999993e-05 2024-09-13T07:01:21.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:20.279545+0000", 2024-09-13T07:01:21.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9016e-05 2024-09-13T07:01:21.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:21.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:20.280431+0000", 2024-09-13T07:01:21.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00088574300000000003 2024-09-13T07:01:21.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:20.280475+0000", 2024-09-13T07:01:21.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4487999999999998e-05 2024-09-13T07:01:21.147 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.147 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.147 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.147 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.147 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.147 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:21.148 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:01:21.197 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:21.193+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:21.296 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:01:21.297 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:21.292+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:21.304 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:21.304 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:21.305 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:21.305 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:21.315 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:01:21.398 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:21.396+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:21.423 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:21.423 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:21.423 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:21.423 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:21.423 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:21.432 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:01:21.499 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:21.493+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:21.587 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:21.587 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:21.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:21.587 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:21.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.587 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4226.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-13T07:01:21.587 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:09.211451+0000", 2024-09-13T07:01:21.587 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.375728709000001, 2024-09-13T07:01:21.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00076862199999999995, 2024-09-13T07:01:21.587 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.587 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:21.588 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211451+0000", 2024-09-13T07:01:21.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211451+0000", 2024-09-13T07:01:21.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211450+0000", 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211455+0000", 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9230000000000001e-06 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211468+0000", 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2152e-05 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211481+0000", 2024-09-13T07:01:21.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3988999999999999e-05 2024-09-13T07:01:21.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211552+0000", 2024-09-13T07:01:21.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0549999999999994e-05 2024-09-13T07:01:21.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211577+0000", 2024-09-13T07:01:21.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5338e-05 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.212173+0000", 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059592900000000003 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.212219+0000", 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5948999999999997e-05 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.591 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.592 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.592 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:21.596 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:01:21.599 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:21.593+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:21.700 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:21.697+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:21.704 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:21.705 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:21.705 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:21.705 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:21.714 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:01:21.801 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:21.797+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:21.811 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:21.811 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:21.811 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:21.811 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:21.811 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:21.820 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:01:21.882 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:01:21.901 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:21.897+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:21.947 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:21.947 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:21.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:21.947 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:21.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.947 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12 3.2 3:4785b378:::rbd_header.119438ec2c70:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:01:21.948 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.307897+0000", 2024-09-13T07:01:21.948 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.63767009299999999, 2024-09-13T07:01:21.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019479619999999999, 2024-09-13T07:01:21.948 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.948 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:21.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:21.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:21.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:21.948 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12 2024-09-13T07:01:21.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.948 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.307897+0000", 2024-09-13T07:01:21.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.307897+0000", 2024-09-13T07:01:21.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.307895+0000", 2024-09-13T07:01:21.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:21.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.307905+0000", 2024-09-13T07:01:21.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0061000000000001e-05 2024-09-13T07:01:21.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.307918+0000", 2024-09-13T07:01:21.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2797e-05 2024-09-13T07:01:21.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.307933+0000", 2024-09-13T07:01:21.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4902999999999999e-05 2024-09-13T07:01:21.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.308012+0000", 2024-09-13T07:01:21.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9233000000000005e-05 2024-09-13T07:01:21.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.308228+0000", 2024-09-13T07:01:21.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000216307 2024-09-13T07:01:21.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:21.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.308503+0000", 2024-09-13T07:01:21.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00027465600000000002 2024-09-13T07:01:21.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:21.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309439+0000", 2024-09-13T07:01:21.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00093679199999999998 2024-09-13T07:01:21.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:21.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309729+0000", 2024-09-13T07:01:21.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028964699999999997 2024-09-13T07:01:21.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:21.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309795+0000", 2024-09-13T07:01:21.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6104000000000003e-05 2024-09-13T07:01:21.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309845+0000", 2024-09-13T07:01:21.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9654000000000003e-05 2024-09-13T07:01:21.956 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.956 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.956 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.957 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12 3.2 e25/22 ondisk, result = 0)", 2024-09-13T07:01:21.957 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.309638+0000", 2024-09-13T07:01:21.957 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.63592873500000002, 2024-09-13T07:01:21.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000216212, 2024-09-13T07:01:21.957 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.957 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:21.957 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309638+0000", 2024-09-13T07:01:21.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309638+0000", 2024-09-13T07:01:21.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309637+0000", 2024-09-13T07:01:21.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:21.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309648+0000", 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0360000000000001e-05 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309663+0000", 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5461e-05 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309671+0000", 2024-09-13T07:01:21.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.994e-06 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309710+0000", 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8559000000000001e-05 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309721+0000", 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1061e-05 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309855+0000", 2024-09-13T07:01:21.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000133862 2024-09-13T07:01:21.961 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.961 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.961 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.961 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:16 3.2 3:4785b378:::rbd_header.119438ec2c70:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:01:21.961 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.321778+0000", 2024-09-13T07:01:21.961 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.62378853199999995, 2024-09-13T07:01:21.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00046924100000000002, 2024-09-13T07:01:21.961 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.961 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:21.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:21.962 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:21.962 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:21.962 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 16 2024-09-13T07:01:21.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.962 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.321778+0000", 2024-09-13T07:01:21.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.321778+0000", 2024-09-13T07:01:21.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.321776+0000", 2024-09-13T07:01:21.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:21.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.321788+0000", 2024-09-13T07:01:21.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1440000000000001e-05 2024-09-13T07:01:21.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.321805+0000", 2024-09-13T07:01:21.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7833999999999999e-05 2024-09-13T07:01:21.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.321814+0000", 2024-09-13T07:01:21.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1969999999999996e-06 2024-09-13T07:01:21.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.321872+0000", 2024-09-13T07:01:21.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8848999999999998e-05 2024-09-13T07:01:21.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.321964+0000", 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1261999999999995e-05 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322248+0000", 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028406999999999998 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.966 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:17 3.2 3:4785b378:::rbd_header.119438ec2c70:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:01:21.967 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.322390+0000", 2024-09-13T07:01:21.967 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.62317652499999998, 2024-09-13T07:01:21.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000311253, 2024-09-13T07:01:21.967 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.967 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:21.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:21.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:21.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:21.967 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 17 2024-09-13T07:01:21.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.967 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322390+0000", 2024-09-13T07:01:21.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322390+0000", 2024-09-13T07:01:21.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322389+0000", 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322398+0000", 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3920000000000003e-06 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322415+0000", 2024-09-13T07:01:21.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7258999999999999e-05 2024-09-13T07:01:21.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322426+0000", 2024-09-13T07:01:21.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1097999999999999e-05 2024-09-13T07:01:21.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322481+0000", 2024-09-13T07:01:21.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4509000000000003e-05 2024-09-13T07:01:21.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322572+0000", 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1793999999999996e-05 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322702+0000", 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000129443 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.971 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:18 3.2 3:4785b378:::rbd_header.119438ec2c70:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.322838+0000", 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.62272935500000004, 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020544899999999999, 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 18 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322838+0000", 2024-09-13T07:01:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322838+0000", 2024-09-13T07:01:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322837+0000", 2024-09-13T07:01:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322845+0000", 2024-09-13T07:01:21.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3680000000000002e-06 2024-09-13T07:01:21.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322859+0000", 2024-09-13T07:01:21.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3935e-05 2024-09-13T07:01:21.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322866+0000", 2024-09-13T07:01:21.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4279999999999999e-06 2024-09-13T07:01:21.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322906+0000", 2024-09-13T07:01:21.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9920999999999999e-05 2024-09-13T07:01:21.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.322953+0000", 2024-09-13T07:01:21.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6771000000000003e-05 2024-09-13T07:01:21.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323043+0000", 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0173999999999994e-05 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:19 3.2 3:4785b378:::rbd_header.119438ec2c70:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.323162+0000", 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.62240489700000001, 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017207, 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.976 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 19 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323162+0000", 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323162+0000", 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323161+0000", 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323167+0000", 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3519999999999997e-06 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323177+0000", 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0892e-05 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323183+0000", 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3739999999999997e-06 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323222+0000", 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9128000000000002e-05 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323272+0000", 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0399999999999999e-05 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323334+0000", 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1802000000000002e-05 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:20 3.2 3:4785b378:::rbd_header.119438ec2c70:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.323916+0000", 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.62165057400000001, 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049244600000000001, 2024-09-13T07:01:21.980 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 20 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323916+0000", 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323916+0000", 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323915+0000", 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323922+0000", 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1319999999999996e-06 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323938+0000", 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5834000000000001e-05 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.323950+0000", 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2126999999999999e-05 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324007+0000", 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7129000000000001e-05 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324087+0000", 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9485000000000002e-05 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324409+0000", 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000321971 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.984 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:21 3.2 3:4785b378:::rbd_header.119438ec2c70:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:01:21.985 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.324559+0000", 2024-09-13T07:01:21.985 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.62100785800000002, 2024-09-13T07:01:21.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00027780399999999998, 2024-09-13T07:01:21.985 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.985 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:21.985 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:21.985 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:21.985 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:21.985 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 21 2024-09-13T07:01:21.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.985 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324559+0000", 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324559+0000", 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324558+0000", 2024-09-13T07:01:21.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324565+0000", 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7649999999999996e-06 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324577+0000", 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.234e-05 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324588+0000", 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0694e-05 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324634+0000", 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6248000000000002e-05 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324679+0000", 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4168000000000001e-05 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324837+0000", 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015846499999999999 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:22 3.2 3:4785b378:::rbd_header.119438ec2c70:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.324972+0000", 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.62059542999999995, 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026347799999999999, 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:21.989 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:21.990 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:21.990 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:21.990 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 22 2024-09-13T07:01:21.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.990 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324972+0000", 2024-09-13T07:01:21.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324972+0000", 2024-09-13T07:01:21.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324970+0000", 2024-09-13T07:01:21.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:21.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324977+0000", 2024-09-13T07:01:21.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7959999999999998e-06 2024-09-13T07:01:21.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.324990+0000", 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3154e-05 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.325009+0000", 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8923000000000002e-05 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.325056+0000", 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7039999999999997e-05 2024-09-13T07:01:21.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.325119+0000", 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2269999999999998e-05 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.325235+0000", 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000116459 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.993 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:24 3.2 3:4785b378:::rbd_header.119438ec2c70:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.326072+0000", 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.61949458300000004, 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000177231, 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 24 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326072+0000", 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326072+0000", 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326071+0000", 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326079+0000", 2024-09-13T07:01:21.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1159999999999999e-06 2024-09-13T07:01:21.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:21.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326096+0000", 2024-09-13T07:01:21.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7132000000000001e-05 2024-09-13T07:01:21.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:21.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326109+0000", 2024-09-13T07:01:21.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2071e-05 2024-09-13T07:01:21.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326145+0000", 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5973999999999998e-05 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326184+0000", 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9477000000000002e-05 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326250+0000", 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5683000000000001e-05 2024-09-13T07:01:21.997 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.998 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:21.998 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:21.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.998 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:25 3.2 3:4785b378:::rbd_header.119438ec2c70:head [watch watch cookie 140335573381872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:01:21.998 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.326433+0000", 2024-09-13T07:01:21.998 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.61913362699999996, 2024-09-13T07:01:21.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018244999999999999, 2024-09-13T07:01:21.998 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:21.998 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:21.998 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:21.998 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:21.998 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 25 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326433+0000", 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326433+0000", 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:21.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326432+0000", 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326440+0000", 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2779999999999996e-06 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326453+0000", 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3399e-05 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:22.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326464+0000", 2024-09-13T07:01:22.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0876000000000001e-05 2024-09-13T07:01:22.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:22.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326515+0000", 2024-09-13T07:01:22.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1479e-05 2024-09-13T07:01:22.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:22.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326578+0000", 2024-09-13T07:01:22.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2842000000000006e-05 2024-09-13T07:01:22.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326701+0000", 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012261100000000001 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.327334+0000", 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00063288500000000002 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328139+0000", 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00080557899999999995 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328194+0000", 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5075999999999999e-05 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328258+0000", 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3563000000000001e-05 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:25 3.2 e25/22 ondisk, result = 0)", 2024-09-13T07:01:22.003 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.328059+0000", 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.61750820500000003, 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00020640799999999999, 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328059+0000", 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328059+0000", 2024-09-13T07:01:22.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:22.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:22.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328057+0000", 2024-09-13T07:01:22.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:22.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:22.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328066+0000", 2024-09-13T07:01:22.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3799999999999994e-06 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328080+0000", 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4484000000000001e-05 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328091+0000", 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0664e-05 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:22.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328126+0000", 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5042000000000001e-05 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328134+0000", 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6969999999999993e-06 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328265+0000", 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013148399999999999 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:22.007 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:22.008 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:22.009 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:01:22.010 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:22.004+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:22.032 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:22.050 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:22.072 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:22.097 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:22.110 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:22.104+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:22.117 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:22.121 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:22.121 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:22.131 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:01:22.139 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:22.161 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:22.182 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:22.184 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:22.184 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:22.184 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:22.185 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:22.185 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:22.185 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:22.185 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:22.185 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:22.211 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:22.205+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:22.240 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:22.240 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:22.250 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:01:22.312 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:22.308+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:22.353 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:22.353 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:22.363 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:01:22.413 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:22.408+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:22.468 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:22.468 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:22.478 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:01:22.515 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:22.512+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:22.585 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:22.585 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:22.595 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:01:22.615 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:22.612+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:22.698 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:22.698 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:22.707 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:01:22.716 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:22.712+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:22.817 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:22.812+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:22.817 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:22.818 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:22.827 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:01:22.917 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:22.912+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:22.932 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:22.932 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:22.942 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:01:23.018 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:23.013+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:23.052 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:23.052 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:23.061 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:01:23.121 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:23.116+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:23.165 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:23.165 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:23.176 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:01:23.222 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:23.217+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:23.266 INFO:teuthology.orchestra.run.smithi118.stdout:2 write 0xa320693 thru 0xa32a788 (0xa0f6 bytes) 2024-09-13T07:01:23.275 INFO:teuthology.orchestra.run.smithi118.stdout:truncating to largest ever: 0xc763f55 2024-09-13T07:01:23.279 INFO:teuthology.orchestra.run.smithi118.stdout:3 trunc from 0xa32a789 to 0xc763f55 2024-09-13T07:01:23.284 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:23.284 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:23.294 INFO:teuthology.orchestra.run.smithi118.stdout:4 read 0x1b1b8dc thru 0x1b25b74 (0xa299 bytes) 2024-09-13T07:01:23.295 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:01:23.295 INFO:teuthology.orchestra.run.smithi118.stdout:5 writesame 0x1e42600 thru 0x1e4fb90 data_size 0x1(0xd591 bytes) 2024-09-13T07:01:23.311 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:01:23.337 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:23.333+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:23.337 INFO:teuthology.orchestra.run.smithi118.stdout:6 write 0xbf14da8 thru 0xbf1f290 (0xa4e9 bytes) 2024-09-13T07:01:23.345 INFO:teuthology.orchestra.run.smithi118.stdout:8 read 0xbf36fba thru 0xbf45ee1 (0xef28 bytes) 2024-09-13T07:01:23.347 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:23.347 INFO:teuthology.orchestra.run.smithi118.stdout:12 read 0x1a57cf5 thru 0x1a5a0d9 (0x23e5 bytes) 2024-09-13T07:01:23.347 INFO:teuthology.orchestra.run.smithi118.stdout:15 read 0x8922741 thru 0x8925b93 (0x3453 bytes) 2024-09-13T07:01:23.348 INFO:teuthology.orchestra.run.smithi118.stdout:16 punch from 0x5cffb00 to 0x5d0b61d, (0xbb1d bytes) 2024-09-13T07:01:23.348 INFO:teuthology.orchestra.run.smithi118.stdout:17 punch from 0x6026d6e to 0x6031621, (0xa8b3 bytes) 2024-09-13T07:01:23.348 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:23.348 INFO:teuthology.orchestra.run.smithi118.stdout:19 read 0x6edad61 thru 0x6ee371d (0x89bd bytes) 2024-09-13T07:01:23.353 INFO:teuthology.orchestra.run.smithi118.stdout:21 trunc from 0xc763f55 to 0x89b26ec 2024-09-13T07:01:23.375 INFO:teuthology.orchestra.run.smithi118.stdout:truncating to largest ever: 0xe33b133 2024-09-13T07:01:23.384 INFO:teuthology.orchestra.run.smithi118.stdout:22 trunc from 0x89b26ec to 0xe33b133 2024-09-13T07:01:23.393 INFO:teuthology.orchestra.run.smithi118.stdout:23 write 0x7db18c thru 0x7e4fef (0x9e64 bytes) 2024-09-13T07:01:23.402 INFO:teuthology.orchestra.run.smithi118.stdout:25 trunc from 0xe33b133 to 0xd05260b 2024-09-13T07:01:23.411 INFO:teuthology.orchestra.run.smithi118.stdout:26 writesame 0x91aa8c1 thru 0x91b464e data_size 0x56(0x9d8e bytes) 2024-09-13T07:01:23.417 INFO:teuthology.orchestra.run.smithi118.stdout:27 write 0x980d95c thru 0x980dbcc (0x271 bytes) 2024-09-13T07:01:23.422 INFO:teuthology.orchestra.run.smithi118.stdout:28 read 0x2fee371 thru 0x2ffe1d7 (0xfe67 bytes) 2024-09-13T07:01:23.423 INFO:teuthology.orchestra.run.smithi118.stdout:29 read 0x32dace2 thru 0x32ddf48 (0x3267 bytes) 2024-09-13T07:01:23.423 INFO:teuthology.orchestra.run.smithi118.stdout:32 write 0xa1e45fb thru 0xa1f3c5c (0xf662 bytes) 2024-09-13T07:01:23.425 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:23.420+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:23.429 INFO:teuthology.orchestra.run.smithi118.stdout:33 write 0xae70bfa thru 0xae79b9c (0x8fa3 bytes) 2024-09-13T07:01:23.434 INFO:teuthology.orchestra.run.smithi118.stdout:34 write 0x1c57554 thru 0x1c61eae (0xa95b bytes) 2024-09-13T07:01:23.436 INFO:teuthology.orchestra.run.smithi118.stdout:36 read 0x3e30b5d thru 0x3e3cdbb (0xc25f bytes) 2024-09-13T07:01:23.436 INFO:teuthology.orchestra.run.smithi118.stdout:37 read 0x3838dc1 thru 0x3843ab9 (0xacf9 bytes) 2024-09-13T07:01:23.436 INFO:teuthology.orchestra.run.smithi118.stdout:39 write 0x6c02999 thru 0x6c02e40 (0x4a8 bytes) 2024-09-13T07:01:23.450 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:23.451 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:23.451 INFO:teuthology.orchestra.run.smithi118.stdout:41 trunc from 0xd05260b to 0x6702fd9 2024-09-13T07:01:23.462 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:01:23.479 INFO:teuthology.orchestra.run.smithi118.stdout:42 trunc from 0x6702fd9 to 0x813136 2024-09-13T07:01:23.497 INFO:teuthology.orchestra.run.smithi118.stdout:43 trunc from 0x813136 to 0x62bb7a 2024-09-13T07:01:23.508 INFO:teuthology.orchestra.run.smithi118.stdout:44 trunc from 0x62bb7a to 0x56a90d 2024-09-13T07:01:23.527 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:23.521+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:23.536 INFO:teuthology.orchestra.run.smithi118.stdout:47 write 0x2f5b876 thru 0x2f5d484 (0x1c0f bytes) 2024-09-13T07:01:23.543 INFO:teuthology.orchestra.run.smithi118.stdout:48 write 0x199d221 thru 0x19a36c7 (0x64a7 bytes) 2024-09-13T07:01:23.549 INFO:teuthology.orchestra.run.smithi118.stdout:49 punch from 0x1ad72a0 to 0x1adb59e, (0x42fe bytes) 2024-09-13T07:01:23.549 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:23.549 INFO:teuthology.orchestra.run.smithi118.stdout:51 punch from 0x2b1aa44 to 0x2b201d5, (0x5791 bytes) 2024-09-13T07:01:23.562 INFO:teuthology.orchestra.run.smithi118.stdout:55 write 0x5b1bf3b thru 0x5b2a462 (0xe528 bytes) 2024-09-13T07:01:23.570 INFO:teuthology.orchestra.run.smithi118.stdout:56 write 0x37244c4 thru 0x372a9e7 (0x6524 bytes) 2024-09-13T07:01:23.571 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:23.571 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:23.577 INFO:teuthology.orchestra.run.smithi118.stdout:57 punch from 0x57c3eb6 to 0x57d3975, (0xfabf bytes) 2024-09-13T07:01:23.577 INFO:teuthology.orchestra.run.smithi118.stdout:58 trunc from 0x5b2a463 to 0x63e6202 2024-09-13T07:01:23.581 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:01:23.586 INFO:teuthology.orchestra.run.smithi118.stdout:59 punch from 0x2253e31 to 0x2261c93, (0xde62 bytes) 2024-09-13T07:01:23.586 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:23.586 INFO:teuthology.orchestra.run.smithi118.stdout:61 write 0x4a9ef01 thru 0x4aae690 (0xf790 bytes) 2024-09-13T07:01:23.605 INFO:teuthology.orchestra.run.smithi118.stdout:62 write 0x81b29f2 thru 0x81c05ec (0xdbfb bytes) 2024-09-13T07:01:23.613 INFO:teuthology.orchestra.run.smithi118.stdout:64 read 0x2dca784 thru 0x2dcb2d0 (0xb4d bytes) 2024-09-13T07:01:23.614 INFO:teuthology.orchestra.run.smithi118.stdout:65 write 0x41b9de5 thru 0x41be569 (0x4785 bytes) 2024-09-13T07:01:23.620 INFO:teuthology.orchestra.run.smithi118.stdout:66 writesame 0xb2220b thru 0xb2432e data_size 0x2a(0x2124 bytes) 2024-09-13T07:01:23.626 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:23.628 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:23.621+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:23.640 INFO:teuthology.orchestra.run.smithi118.stdout:68 write 0xb0054b6 thru 0xb00bf71 (0x6abc bytes) 2024-09-13T07:01:23.648 INFO:teuthology.orchestra.run.smithi118.stdout:69 write 0x90ee02c thru 0x90fc2c6 (0xe29b bytes) 2024-09-13T07:01:23.656 INFO:teuthology.orchestra.run.smithi118.stdout:72 writesame 0x42925ab thru 0x429be07 data_size 0x1d(0x985d bytes) 2024-09-13T07:01:23.673 INFO:teuthology.orchestra.run.smithi118.stdout:75 trunc from 0xb00bf72 to 0x35c2f15 2024-09-13T07:01:23.687 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:01:23.688 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T07:01:23.690 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:23.690 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:23.699 INFO:teuthology.orchestra.run.smithi118.stdout:76 write 0x1bf99b7 thru 0x1c02eaa (0x94f4 bytes) 2024-09-13T07:01:23.700 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:01:23.705 INFO:teuthology.orchestra.run.smithi118.stdout:77 write 0x2dd8a83 thru 0x2ddc29c (0x381a bytes) 2024-09-13T07:01:23.729 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:23.725+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:23.729 INFO:teuthology.orchestra.run.smithi118.stdout:79 writesame 0xa9dc6b4 thru 0xa9eb0f0 data_size 0x43(0xea3d bytes) 2024-09-13T07:01:23.729 INFO:teuthology.orchestra.run.smithi118.stdout:80 writesame 0x9b94d9e thru 0x9b9d0c5 data_size 0xc(0x8328 bytes) 2024-09-13T07:01:23.749 INFO:teuthology.orchestra.run.smithi118.stdout:81 write 0xaae0713 thru 0xaae7a7f (0x736d bytes) 2024-09-13T07:01:23.755 INFO:teuthology.orchestra.run.smithi118.stdout:82 punch from 0x8abda45 to 0x8abf51b, (0x1ad6 bytes) 2024-09-13T07:01:23.756 INFO:teuthology.orchestra.run.smithi118.stdout:83 read 0x5474344 thru 0x5481969 (0xd626 bytes) 2024-09-13T07:01:23.756 INFO:teuthology.orchestra.run.smithi118.stdout:86 write 0x29674f1 thru 0x296d281 (0x5d91 bytes) 2024-09-13T07:01:23.762 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:23.762 INFO:teuthology.orchestra.run.smithi118.stdout:88 read 0x4a04a18 thru 0x4a10c98 (0xc281 bytes) 2024-09-13T07:01:23.762 INFO:teuthology.orchestra.run.smithi118.stdout:89 punch from 0x962b575 to 0x9633462, (0x7eed bytes) 2024-09-13T07:01:23.762 INFO:teuthology.orchestra.run.smithi118.stdout:90 writesame 0x2e0aa08 thru 0x2e133e5 data_size 0x7(0x89de bytes) 2024-09-13T07:01:23.787 INFO:teuthology.orchestra.run.smithi118.stdout:92 trunc from 0xaae7a80 to 0x50d587 2024-09-13T07:01:23.814 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:23.814 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:23.830 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:23.825+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:23.831 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:01:23.836 INFO:teuthology.orchestra.run.smithi118.stdout:93 writesame 0x6435113 thru 0x6441207 data_size 0x1(0xc0f5 bytes) 2024-09-13T07:01:23.836 INFO:teuthology.orchestra.run.smithi118.stdout:94 clone 1 order 20 su 16384 sc 5 2024-09-13T07:01:23.931 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:23.925+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:23.933 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:23.933 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:23.942 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:01:24.031 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:24.025+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:24.045 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:24.046 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:24.055 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:01:24.133 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:24.129+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:24.157 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:24.157 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:24.165 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:01:24.195 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:01:24.239 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:24.236+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:24.283 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:24.283 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:24.292 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:01:24.304 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:01:24.306 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T07:01:24.318 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T07:01:24.334 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:24.329+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:24.342 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:24.377 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:24.396 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:24.396 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:24.406 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:01:24.409 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:24.435 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:24.429+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:24.443 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:24.470 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:24.496 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:24.515 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:24.516 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:24.516 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:24.516 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:24.516 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.516 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4226.0:1 2.3 e18/17 2:c4c92e5a:::rbd_trash:head v 18'1, mlcod=0'0)", 2024-09-13T07:01:24.516 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:09.211451+0000", 2024-09-13T07:01:24.516 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.304542691, 2024-09-13T07:01:24.516 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00076862199999999995, 2024-09-13T07:01:24.516 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.516 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:24.516 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.516 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.517 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.517 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211451+0000", 2024-09-13T07:01:24.517 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.517 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.517 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.517 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211451+0000", 2024-09-13T07:01:24.517 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.517 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.517 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.517 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211450+0000", 2024-09-13T07:01:24.517 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211455+0000", 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9230000000000001e-06 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211468+0000", 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2152e-05 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211481+0000", 2024-09-13T07:01:24.518 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3988999999999999e-05 2024-09-13T07:01:24.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.519 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.519 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211552+0000", 2024-09-13T07:01:24.519 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0549999999999994e-05 2024-09-13T07:01:24.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.519 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.519 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.211577+0000", 2024-09-13T07:01:24.519 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5338e-05 2024-09-13T07:01:24.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.519 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:24.520 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.212173+0000", 2024-09-13T07:01:24.520 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00059592900000000003 2024-09-13T07:01:24.520 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.520 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.520 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:24.520 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:09.212219+0000", 2024-09-13T07:01:24.520 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5948999999999997e-05 2024-09-13T07:01:24.520 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.520 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.520 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.520 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.520 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.520 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:24.522 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:24.526 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:01:24.534 INFO:teuthology.orchestra.run.smithi118.stdout:95 writesame 0xed0ec8b thru 0xed19060 data_size 0x2(0xa3d6 bytes) 2024-09-13T07:01:24.534 INFO:teuthology.orchestra.run.smithi118.stdout:96 writesame 0x36a83db thru 0x36b5fab data_size 0x1(0xdbd1 bytes) 2024-09-13T07:01:24.535 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:24.532+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:24.557 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:24.559 INFO:teuthology.orchestra.run.smithi118.stdout:97 writesame 0x5dbb75b thru 0x5dc122e data_size 0x2(0x5ad4 bytes) 2024-09-13T07:01:24.560 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:24.560 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:24.560 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:24.560 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:24.560 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:24.560 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:24.560 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:24.561 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:24.571 INFO:teuthology.orchestra.run.smithi118.stdout:98 write 0x226bd18 thru 0x227b999 (0xfc82 bytes) 2024-09-13T07:01:24.581 INFO:teuthology.orchestra.run.smithi118.stdout:100 write 0x3395280 thru 0x339e1f4 (0x8f75 bytes) 2024-09-13T07:01:24.585 INFO:teuthology.orchestra.run.smithi118.stdout:101 writesame 0xe59f1cc thru 0xe5a4221 data_size 0x5b(0x5056 bytes) 2024-09-13T07:01:24.595 INFO:teuthology.orchestra.run.smithi118.stdout:103 trunc from 0xed19061 to 0xc9b7e9c 2024-09-13T07:01:24.612 INFO:teuthology.orchestra.run.smithi118.stdout:104 write 0x674c56e thru 0x67560f2 (0x9b85 bytes) 2024-09-13T07:01:24.622 INFO:teuthology.orchestra.run.smithi118.stdout:105 write 0x8cd09fb thru 0x8cdb3a7 (0xa9ad bytes) 2024-09-13T07:01:24.635 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:24.635 INFO:teuthology.orchestra.run.smithi118.stdout:107 writesame 0x98725f0 thru 0x9877aff data_size 0x8(0x5510 bytes) 2024-09-13T07:01:24.636 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:24.629+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:24.642 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:24.643 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:24.643 INFO:teuthology.orchestra.run.smithi118.stdout:111 writesame 0x24e5cff thru 0x24f38a6 data_size 0x48(0xdba8 bytes) 2024-09-13T07:01:24.651 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:24.651 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:24.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.651 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:502 3.2 3:46f58397:::rbd_data.1194d22ad098.0000000000000024:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 1048576 write_size 1048576,write 400639~8961 in=8961b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:24.651 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.648777+0000", 2024-09-13T07:01:24.651 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.0019764100000000001, 2024-09-13T07:01:24.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002026384, 2024-09-13T07:01:24.651 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.651 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "waiting for sub ops", 2024-09-13T07:01:24.651 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:24.651 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:24.652 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:24.652 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 502 2024-09-13T07:01:24.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.652 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648777+0000", 2024-09-13T07:01:24.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648777+0000", 2024-09-13T07:01:24.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648775+0000", 2024-09-13T07:01:24.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:24.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648801+0000", 2024-09-13T07:01:24.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6542000000000001e-05 2024-09-13T07:01:24.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648835+0000", 2024-09-13T07:01:24.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3722e-05 2024-09-13T07:01:24.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648852+0000", 2024-09-13T07:01:24.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7139999999999999e-05 2024-09-13T07:01:24.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648929+0000", 2024-09-13T07:01:24.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6556000000000004e-05 2024-09-13T07:01:24.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649090+0000", 2024-09-13T07:01:24.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016141500000000001 2024-09-13T07:01:24.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:24.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649324+0000", 2024-09-13T07:01:24.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023435400000000001 2024-09-13T07:01:24.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:24.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650477+0000", 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011529820000000001 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:503 3.7 3:e6bfb8ee:::rbd_data.1194d22ad098.0000000000000025:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 1048576 write_size 1048576,write 393216~16384 in=16384b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.648972+0000", 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.0017821969999999999, 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020202390000000001, 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "waiting for sub ops", 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:24.656 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:24.657 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:24.657 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 503 2024-09-13T07:01:24.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.657 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648972+0000", 2024-09-13T07:01:24.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648972+0000", 2024-09-13T07:01:24.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648969+0000", 2024-09-13T07:01:24.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:24.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648988+0000", 2024-09-13T07:01:24.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9293999999999999e-05 2024-09-13T07:01:24.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649002+0000", 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3332e-05 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649015+0000", 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3219e-05 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649097+0000", 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2281999999999996e-05 2024-09-13T07:01:24.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649391+0000", 2024-09-13T07:01:24.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029331000000000002 2024-09-13T07:01:24.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:24.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649660+0000", 2024-09-13T07:01:24.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026905400000000002 2024-09-13T07:01:24.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:24.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650862+0000", 2024-09-13T07:01:24.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0012022090000000001 2024-09-13T07:01:24.661 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.661 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.661 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.661 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:505 3.18 3:1b19bc2d:::rbd_data.1194d22ad098.0000000000000027:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 1048576 write_size 1048576,write 393216~14503 in=14503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:24.661 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.650008+0000", 2024-09-13T07:01:24.661 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.00074537599999999996, 2024-09-13T07:01:24.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011584919999999999, 2024-09-13T07:01:24.661 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.661 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "waiting for sub ops", 2024-09-13T07:01:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 505 2024-09-13T07:01:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650008+0000", 2024-09-13T07:01:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650008+0000", 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650007+0000", 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650023+0000", 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6215999999999999e-05 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650039+0000", 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5234e-05 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650051+0000", 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2729e-05 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650111+0000", 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9420999999999999e-05 2024-09-13T07:01:24.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650445+0000", 2024-09-13T07:01:24.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00033427899999999999 2024-09-13T07:01:24.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:24.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650768+0000", 2024-09-13T07:01:24.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00032304799999999999 2024-09-13T07:01:24.665 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.665 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.665 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.665 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.666 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:01:24.666 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 3 2024-09-13T07:01:24.666 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:24.666 INFO:teuthology.orchestra.run.smithi118.stdout:112 write 0x6b8a0dc thru 0x6b91423 (0x7348 bytes) 2024-09-13T07:01:24.666 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:24.667 INFO:teuthology.orchestra.run.smithi118.stdout:114 punch from 0xa9f6763 to 0xa9f9c25, (0x34c2 bytes) 2024-09-13T07:01:24.667 INFO:teuthology.orchestra.run.smithi118.stdout:115 punch from 0x28702d4 to 0x2879a12, (0x973e bytes) 2024-09-13T07:01:24.667 INFO:teuthology.orchestra.run.smithi118.stdout:116 clone 2 order 24 su 2097152 sc 13 2024-09-13T07:01:24.668 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:01:24.736 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:24.729+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:24.819 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:24.819 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:24.819 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:24.819 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:24.819 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:24.829 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:01:24.837 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:24.832+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:24.938 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:24.932+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:24.941 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:24.941 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:24.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:24.941 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:24.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.941 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12 3.2 3:4785b378:::rbd_header.119438ec2c70:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:01:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.307897+0000", 2024-09-13T07:01:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.6302947149999998, 2024-09-13T07:01:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019479619999999999, 2024-09-13T07:01:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12 2024-09-13T07:01:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.307897+0000", 2024-09-13T07:01:24.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.307897+0000", 2024-09-13T07:01:24.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.307895+0000", 2024-09-13T07:01:24.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:24.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.307905+0000", 2024-09-13T07:01:24.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0061000000000001e-05 2024-09-13T07:01:24.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.307918+0000", 2024-09-13T07:01:24.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2797e-05 2024-09-13T07:01:24.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.307933+0000", 2024-09-13T07:01:24.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4902999999999999e-05 2024-09-13T07:01:24.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.308012+0000", 2024-09-13T07:01:24.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9233000000000005e-05 2024-09-13T07:01:24.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.308228+0000", 2024-09-13T07:01:24.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000216307 2024-09-13T07:01:24.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.308503+0000", 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00027465600000000002 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309439+0000", 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00093679199999999998 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309729+0000", 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028964699999999997 2024-09-13T07:01:24.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309795+0000", 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6104000000000003e-05 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.309845+0000", 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9654000000000003e-05 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.948 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:25 3.2 3:4785b378:::rbd_header.119438ec2c70:head [watch watch cookie 140335573381872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:01:24.948 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:21.326433+0000", 2024-09-13T07:01:24.948 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.6117582490000002, 2024-09-13T07:01:24.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018244999999999999, 2024-09-13T07:01:24.948 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.948 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:24.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:24.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:24.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:24.948 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 25 2024-09-13T07:01:24.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.948 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326433+0000", 2024-09-13T07:01:24.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326433+0000", 2024-09-13T07:01:24.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326432+0000", 2024-09-13T07:01:24.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326440+0000", 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2779999999999996e-06 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326453+0000", 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3399e-05 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326464+0000", 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0876000000000001e-05 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326515+0000", 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1479e-05 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326578+0000", 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2842000000000006e-05 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:24.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.326701+0000", 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012261100000000001 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.327334+0000", 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00063288500000000002 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328139+0000", 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00080557899999999995 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:24.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328194+0000", 2024-09-13T07:01:24.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5075999999999999e-05 2024-09-13T07:01:24.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:24.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:21.328258+0000", 2024-09-13T07:01:24.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3563000000000001e-05 2024-09-13T07:01:24.953 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.953 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.953 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.953 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:390 3.2 3:4785b378:::rbd_header.119438ec2c70:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:24.954 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.474319+0000", 2024-09-13T07:01:24.954 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.46387292200000002, 2024-09-13T07:01:24.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022737230000000001, 2024-09-13T07:01:24.954 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.954 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:24.954 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:24.954 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:24.954 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:24.954 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 390 2024-09-13T07:01:24.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.954 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.474319+0000", 2024-09-13T07:01:24.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.474319+0000", 2024-09-13T07:01:24.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.474316+0000", 2024-09-13T07:01:24.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:24.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.474334+0000", 2024-09-13T07:01:24.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7408e-05 2024-09-13T07:01:24.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.474370+0000", 2024-09-13T07:01:24.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6531e-05 2024-09-13T07:01:24.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.474410+0000", 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9784999999999999e-05 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.474503+0000", 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3462999999999996e-05 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.474636+0000", 2024-09-13T07:01:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013238400000000001 2024-09-13T07:01:24.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:24.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.474898+0000", 2024-09-13T07:01:24.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000262005 2024-09-13T07:01:24.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:24.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.475943+0000", 2024-09-13T07:01:24.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001045086 2024-09-13T07:01:24.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.476488+0000", 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00054502399999999996 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.476545+0000", 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7154999999999997e-05 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.476592+0000", 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7499e-05 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.960 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.961 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.961 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:392 3.2 3:4785b378:::rbd_header.119438ec2c70:head [watch unwatch cookie 140335573381872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:24.961 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.476934+0000", 2024-09-13T07:01:24.961 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.46125745400000001, 2024-09-13T07:01:24.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023329029999999999, 2024-09-13T07:01:24.961 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.961 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:24.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:24.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:24.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:24.961 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 392 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.476934+0000", 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.476934+0000", 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.476933+0000", 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.476941+0000", 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5799999999999992e-06 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.476959+0000", 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7614000000000001e-05 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.476970+0000", 2024-09-13T07:01:24.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1034e-05 2024-09-13T07:01:24.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.477205+0000", 2024-09-13T07:01:24.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023557999999999999 2024-09-13T07:01:24.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.477285+0000", 2024-09-13T07:01:24.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9605999999999996e-05 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.477435+0000", 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015014999999999999 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.478207+0000", 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00077173599999999997 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:24.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.479107+0000", 2024-09-13T07:01:24.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00090047800000000002 2024-09-13T07:01:24.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:24.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.479157+0000", 2024-09-13T07:01:24.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9718000000000001e-05 2024-09-13T07:01:24.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:24.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.479267+0000", 2024-09-13T07:01:24.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109913 2024-09-13T07:01:24.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.966 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:393 3.2 3:4785b378:::rbd_header.119438ec2c70:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.477320+0000", 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.460871894, 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021492680000000002, 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 393 2024-09-13T07:01:24.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.968 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.477320+0000", 2024-09-13T07:01:24.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.477320+0000", 2024-09-13T07:01:24.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.477318+0000", 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.477330+0000", 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1949e-05 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.477346+0000", 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6235999999999998e-05 2024-09-13T07:01:24.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.477355+0000", 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5429999999999999e-06 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.477749+0000", 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00039326699999999999 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for rw locks", 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.477819+0000", 2024-09-13T07:01:24.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0538999999999997e-05 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.479314+0000", 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001495295 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.479387+0000", 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2903000000000003e-05 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:24.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.479469+0000", 2024-09-13T07:01:24.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1681999999999995e-05 2024-09-13T07:01:24.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.972 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.972 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:411 3.2 3:4785b378:::rbd_header.119438ec2c70:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:24.972 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.498718+0000", 2024-09-13T07:01:24.972 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.43947351600000001, 2024-09-13T07:01:24.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00055779199999999999, 2024-09-13T07:01:24.972 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.972 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:24.972 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:24.973 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:24.973 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:24.973 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 411 2024-09-13T07:01:24.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.973 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.498718+0000", 2024-09-13T07:01:24.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.498718+0000", 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.498717+0000", 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.498735+0000", 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8142999999999999e-05 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.498749+0000", 2024-09-13T07:01:24.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.349e-05 2024-09-13T07:01:24.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.498757+0000", 2024-09-13T07:01:24.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7389999999999999e-06 2024-09-13T07:01:24.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.498800+0000", 2024-09-13T07:01:24.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2849999999999998e-05 2024-09-13T07:01:24.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.498880+0000", 2024-09-13T07:01:24.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0560000000000001e-05 2024-09-13T07:01:24.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:24.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.499276+0000", 2024-09-13T07:01:24.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000395857 2024-09-13T07:01:24.976 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.976 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.976 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.977 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:441 3.b 3:d465e8df:::rbd_data.1194d22ad098.0000000000000036:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:24.977 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:24.977 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.39706672599999998, 2024-09-13T07:01:24.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012043093, 2024-09-13T07:01:24.977 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.977 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:24.977 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:24.977 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:24.977 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:24.977 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 441 2024-09-13T07:01:24.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541122+0000", 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541136+0000", 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3899e-05 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541163+0000", 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.694e-05 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541176+0000", 2024-09-13T07:01:24.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2802e-05 2024-09-13T07:01:24.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541257+0000", 2024-09-13T07:01:24.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1018000000000003e-05 2024-09-13T07:01:24.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541414+0000", 2024-09-13T07:01:24.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015745000000000001 2024-09-13T07:01:24.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:24.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.544450+0000", 2024-09-13T07:01:24.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030357209999999999 2024-09-13T07:01:24.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:24.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553002+0000", 2024-09-13T07:01:24.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085521020000000007 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553022+0000", 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9694000000000002e-05 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553103+0000", 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1008999999999996e-05 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553168+0000", 2024-09-13T07:01:24.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5010000000000003e-05 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:441 3.b e26/22 ondisk, result = 0)", 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.549086+0000", 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.38910581100000002, 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003922643, 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.549086+0000", 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.549086+0000", 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.549085+0000", 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.549091+0000", 2024-09-13T07:01:24.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2570000000000004e-06 2024-09-13T07:01:24.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.549102+0000", 2024-09-13T07:01:24.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1341e-05 2024-09-13T07:01:24.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.549109+0000", 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4980000000000003e-06 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.552978+0000", 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0038688339999999998 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.552991+0000", 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3667e-05 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553008+0000", 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7159e-05 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:445 3.5 e26/22 3:a20826fb:::rbd_data.1194d22ad098.000000000000005a:head v 26'1, mlcod=0'0)", 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.568272+0000", 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.36992005900000002, 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001150809, 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:24.987 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.568272+0000", 2024-09-13T07:01:24.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.568272+0000", 2024-09-13T07:01:24.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.568270+0000", 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.568286+0000", 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6385999999999999e-05 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.568306+0000", 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9911e-05 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.568315+0000", 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6789999999999994e-06 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.568384+0000", 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9511999999999994e-05 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.568410+0000", 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5718999999999999e-05 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.569377+0000", 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00096693899999999999 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.569422+0000", 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5247000000000002e-05 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.991 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:453 3.5 e26/22 3:a1088ec2:::rbd_data.1194d22ad098.000000000000001e:head v 26'2, mlcod=0'0)", 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.577675+0000", 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.36051673400000001, 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010202080000000001, 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.577675+0000", 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.577675+0000", 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.577673+0000", 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.577760+0000", 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6879999999999995e-05 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.577787+0000", 2024-09-13T07:01:24.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6684000000000001e-05 2024-09-13T07:01:24.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.577800+0000", 2024-09-13T07:01:24.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.314e-05 2024-09-13T07:01:24.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.577854+0000", 2024-09-13T07:01:24.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4370000000000003e-05 2024-09-13T07:01:24.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.577881+0000", 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6271e-05 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578653+0000", 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000772275 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578695+0000", 2024-09-13T07:01:24.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2329000000000001e-05 2024-09-13T07:01:24.996 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.996 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:24.996 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:24.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.996 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:456 3.8 3:1612121e:::rbd_data.1194d22ad098.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 917504~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:24.996 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.578426+0000", 2024-09-13T07:01:24.996 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.359765797, 2024-09-13T07:01:24.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001584593, 2024-09-13T07:01:24.996 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:24.996 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:24.996 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:24.996 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:24.997 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:24.997 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 456 2024-09-13T07:01:24.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.997 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:24.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:24.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578426+0000", 2024-09-13T07:01:24.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:24.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578426+0000", 2024-09-13T07:01:24.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578424+0000", 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578442+0000", 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7456e-05 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:24.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578457+0000", 2024-09-13T07:01:24.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5287000000000001e-05 2024-09-13T07:01:24.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:24.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578466+0000", 2024-09-13T07:01:24.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:01:24.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:24.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:24.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578539+0000", 2024-09-13T07:01:24.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2947999999999993e-05 2024-09-13T07:01:24.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:24.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578740+0000", 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000201218 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578911+0000", 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017111700000000001 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.579704+0000", 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00079278699999999998 2024-09-13T07:01:25.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.579918+0000", 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021394599999999999 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.579969+0000", 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0763000000000002e-05 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.580010+0000", 2024-09-13T07:01:25.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1563000000000002e-05 2024-09-13T07:01:25.002 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.002 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:25.002 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.002 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:455 3.15 e26/22 3:ad3c8f45:::rbd_data.1194d22ad098.0000000000000020:head v 26'3, mlcod=25'1)", 2024-09-13T07:01:25.002 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.578996+0000", 2024-09-13T07:01:25.002 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.35919522599999998, 2024-09-13T07:01:25.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097081900000000004, 2024-09-13T07:01:25.002 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:25.002 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.002 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:25.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578996+0000", 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578996+0000", 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.578996+0000", 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:25.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:25.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.579002+0000", 2024-09-13T07:01:25.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.049e-06 2024-09-13T07:01:25.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:25.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.579060+0000", 2024-09-13T07:01:25.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7544000000000003e-05 2024-09-13T07:01:25.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.579068+0000", 2024-09-13T07:01:25.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9959999999999993e-06 2024-09-13T07:01:25.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:25.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.579131+0000", 2024-09-13T07:01:25.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3831000000000006e-05 2024-09-13T07:01:25.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:25.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.579151+0000", 2024-09-13T07:01:25.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9470000000000002e-05 2024-09-13T07:01:25.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:25.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.579940+0000", 2024-09-13T07:01:25.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00078883600000000001 2024-09-13T07:01:25.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:25.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.579967+0000", 2024-09-13T07:01:25.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7472000000000001e-05 2024-09-13T07:01:25.006 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.007 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:25.007 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.007 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:465 3.c e26/22 3:36f8031a:::rbd_data.1194d22ad098.00000000000000e1:head v 26'1, mlcod=0'0)", 2024-09-13T07:01:25.007 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.590307+0000", 2024-09-13T07:01:25.007 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.34788512799999999, 2024-09-13T07:01:25.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001227857, 2024-09-13T07:01:25.007 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:25.007 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.007 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:25.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.590307+0000", 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.590307+0000", 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.590305+0000", 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:25.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.590320+0000", 2024-09-13T07:01:25.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4698000000000001e-05 2024-09-13T07:01:25.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:25.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.590337+0000", 2024-09-13T07:01:25.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7560000000000001e-05 2024-09-13T07:01:25.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.590346+0000", 2024-09-13T07:01:25.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9400000000000008e-06 2024-09-13T07:01:25.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:25.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.590417+0000", 2024-09-13T07:01:25.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1438000000000003e-05 2024-09-13T07:01:25.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:25.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.590451+0000", 2024-09-13T07:01:25.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3293000000000001e-05 2024-09-13T07:01:25.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.591486+0000", 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010349529999999999 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.591534+0000", 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8671999999999997e-05 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:466 3.c e26/22 3:369d6e11:::rbd_data.1194d22ad098.00000000000000e2:head v 26'2, mlcod=0'0)", 2024-09-13T07:01:25.011 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.591589+0000", 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.34660276800000001, 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00090512299999999997, 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.591589+0000", 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.591589+0000", 2024-09-13T07:01:25.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.591588+0000", 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.591609+0000", 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1202000000000001e-05 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:25.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.591624+0000", 2024-09-13T07:01:25.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4877e-05 2024-09-13T07:01:25.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.591635+0000", 2024-09-13T07:01:25.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0808000000000001e-05 2024-09-13T07:01:25.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:25.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.591679+0000", 2024-09-13T07:01:25.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3621000000000001e-05 2024-09-13T07:01:25.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.591703+0000", 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4766999999999999e-05 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.592461+0000", 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00075775500000000004 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.592494+0000", 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2891000000000001e-05 2024-09-13T07:01:25.015 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.016 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:25.016 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.016 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:469 3.c e26/22 3:36f8031a:::rbd_data.1194d22ad098.00000000000000e1:head v 26'3, mlcod=26'1)", 2024-09-13T07:01:25.016 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.600005+0000", 2024-09-13T07:01:25.016 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.338186722, 2024-09-13T07:01:25.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010069440000000001, 2024-09-13T07:01:25.016 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:25.016 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.016 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:25.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.600005+0000", 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.600005+0000", 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.600003+0000", 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:25.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.600015+0000", 2024-09-13T07:01:25.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.221e-05 2024-09-13T07:01:25.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:25.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.600037+0000", 2024-09-13T07:01:25.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1248000000000001e-05 2024-09-13T07:01:25.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.600047+0000", 2024-09-13T07:01:25.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0059e-05 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.600134+0000", 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7294999999999997e-05 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.600166+0000", 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.239e-05 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:25.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.600980+0000", 2024-09-13T07:01:25.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081316500000000002 2024-09-13T07:01:25.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:25.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.601012+0000", 2024-09-13T07:01:25.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2245e-05 2024-09-13T07:01:25.020 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.020 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:25.020 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.020 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:486 3.7 3:e0369888:::rbd_data.1194d22ad098.0000000000000066:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 688128~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:25.020 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.617668+0000", 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.32052332700000002, 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0054131769999999999, 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 486 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617668+0000", 2024-09-13T07:01:25.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:25.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617668+0000", 2024-09-13T07:01:25.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:25.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617666+0000", 2024-09-13T07:01:25.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:25.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617687+0000", 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0984000000000001e-05 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617709+0000", 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2177000000000001e-05 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617720+0000", 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0587e-05 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:25.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617806+0000", 2024-09-13T07:01:25.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5916999999999997e-05 2024-09-13T07:01:25.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:25.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.618026+0000", 2024-09-13T07:01:25.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219986 2024-09-13T07:01:25.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:25.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.618253+0000", 2024-09-13T07:01:25.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022708099999999999 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.619424+0000", 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011709859999999999 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.622945+0000", 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035214090000000001 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:25.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.623023+0000", 2024-09-13T07:01:25.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7564999999999993e-05 2024-09-13T07:01:25.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:25.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.623081+0000", 2024-09-13T07:01:25.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8819e-05 2024-09-13T07:01:25.026 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.026 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:25.026 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.026 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:487 3.b 3:d10c740a:::rbd_data.1194d22ad098.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 688128~8435] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.618111+0000", 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.320080372, 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022849250000000002, 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 487 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.618111+0000", 2024-09-13T07:01:25.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:25.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.618111+0000", 2024-09-13T07:01:25.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:25.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.618110+0000", 2024-09-13T07:01:25.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:25.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:25.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.618128+0000", 2024-09-13T07:01:25.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8255000000000001e-05 2024-09-13T07:01:25.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:25.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.618142+0000", 2024-09-13T07:01:25.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3954000000000001e-05 2024-09-13T07:01:25.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.618154+0000", 2024-09-13T07:01:25.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1576999999999999e-05 2024-09-13T07:01:25.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:25.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.618244+0000", 2024-09-13T07:01:25.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0400000000000002e-05 2024-09-13T07:01:25.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:25.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.618472+0000", 2024-09-13T07:01:25.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000227393 2024-09-13T07:01:25.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:25.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.618817+0000", 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034503499999999999 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.620237+0000", 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014205209999999999 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.620263+0000", 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5799000000000002e-05 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:25.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.620340+0000", 2024-09-13T07:01:25.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6882e-05 2024-09-13T07:01:25.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:25.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.620396+0000", 2024-09-13T07:01:25.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6459999999999998e-05 2024-09-13T07:01:25.032 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.032 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:25.032 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.032 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:502 3.2 3:46f58397:::rbd_data.1194d22ad098.0000000000000024:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 1048576 write_size 1048576,write 400639~8961 in=8961b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:25.033 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.648777+0000", 2024-09-13T07:01:25.033 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.289414328, 2024-09-13T07:01:25.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031621000000000002, 2024-09-13T07:01:25.033 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:25.033 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.033 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:25.033 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:25.033 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:25.033 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 502 2024-09-13T07:01:25.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.033 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:25.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648777+0000", 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648777+0000", 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648775+0000", 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:25.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648801+0000", 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6542000000000001e-05 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648835+0000", 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3722e-05 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648852+0000", 2024-09-13T07:01:25.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7139999999999999e-05 2024-09-13T07:01:25.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:25.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648929+0000", 2024-09-13T07:01:25.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6556000000000004e-05 2024-09-13T07:01:25.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:25.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649090+0000", 2024-09-13T07:01:25.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016141500000000001 2024-09-13T07:01:25.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649324+0000", 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023435400000000001 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650477+0000", 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0011529820000000001 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.651863+0000", 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001385861 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.651906+0000", 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2823e-05 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.651939+0000", 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3312999999999998e-05 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.038 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:503 3.7 3:e6bfb8ee:::rbd_data.1194d22ad098.0000000000000025:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 1048576 write_size 1048576,write 393216~16384 in=16384b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.648972+0000", 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.289220115, 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029352179999999999, 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 503 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:25.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648972+0000", 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648972+0000", 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648969+0000", 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:25.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.648988+0000", 2024-09-13T07:01:25.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9293999999999999e-05 2024-09-13T07:01:25.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:25.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649002+0000", 2024-09-13T07:01:25.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3332e-05 2024-09-13T07:01:25.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649015+0000", 2024-09-13T07:01:25.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3219e-05 2024-09-13T07:01:25.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:25.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649097+0000", 2024-09-13T07:01:25.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2281999999999996e-05 2024-09-13T07:01:25.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:25.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649391+0000", 2024-09-13T07:01:25.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00029331000000000002 2024-09-13T07:01:25.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:25.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.649660+0000", 2024-09-13T07:01:25.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026905400000000002 2024-09-13T07:01:25.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:25.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650862+0000", 2024-09-13T07:01:25.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0012022090000000001 2024-09-13T07:01:25.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:25.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.651771+0000", 2024-09-13T07:01:25.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000909621 2024-09-13T07:01:25.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:25.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.651853+0000", 2024-09-13T07:01:25.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1836999999999996e-05 2024-09-13T07:01:25.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:25.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.651907+0000", 2024-09-13T07:01:25.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3431999999999999e-05 2024-09-13T07:01:25.044 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.044 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:25.044 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.045 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:505 3.18 3:1b19bc2d:::rbd_data.1194d22ad098.0000000000000027:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 1048576 write_size 1048576,write 393216~14503 in=14503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:25.045 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.650008+0000", 2024-09-13T07:01:25.045 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.28818329399999998, 2024-09-13T07:01:25.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029802769999999999, 2024-09-13T07:01:25.045 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:25.045 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.045 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:25.045 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:25.045 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:25.045 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 505 2024-09-13T07:01:25.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.045 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:25.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650008+0000", 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650008+0000", 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650007+0000", 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:25.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650023+0000", 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6215999999999999e-05 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650039+0000", 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5234e-05 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650051+0000", 2024-09-13T07:01:25.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2729e-05 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650111+0000", 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9420999999999999e-05 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650445+0000", 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00033427899999999999 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:25.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.650768+0000", 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00032304799999999999 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.652032+0000", 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0012640360000000001 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.652910+0000", 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00087796300000000005 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:25.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.652959+0000", 2024-09-13T07:01:25.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9052999999999999e-05 2024-09-13T07:01:25.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:25.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:25.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:25.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.652989+0000", 2024-09-13T07:01:25.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9547e-05 2024-09-13T07:01:25.050 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.050 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:25.050 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.050 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:25.050 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:25.050 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:25.052 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:01:25.052 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:25.048+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:25.153 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:25.149+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:25.170 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:25.170 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:25.170 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:25.170 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:25.180 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:01:25.254 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:25.249+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:25.289 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:25.289 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:25.289 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:25.289 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:25.289 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:25.298 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:01:25.317 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T07:01:25.330 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T07:01:25.354 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:25.349+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:25.440 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone1 from 0xc9b7e9c (overlap 0x6441208) to 0x0 2024-09-13T07:01:25.455 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:25.449+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:25.461 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:25.461 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:20.273670+0000", 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.1841773489999996, 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065273279999999998, 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7 2024-09-13T07:01:25.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.273670+0000", 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.273670+0000", 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.273668+0000", 2024-09-13T07:01:25.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.273683+0000", 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5386000000000001e-05 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.273718+0000", 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4885999999999998e-05 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.273743+0000", 2024-09-13T07:01:25.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4919999999999999e-05 2024-09-13T07:01:25.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.273874+0000", 2024-09-13T07:01:25.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131044 2024-09-13T07:01:25.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:25.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.273885+0000", 2024-09-13T07:01:25.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0832999999999999e-05 2024-09-13T07:01:25.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.277312+0000", 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034274900000000001 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.277605+0000", 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029266699999999998 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.277827+0000", 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000222582 2024-09-13T07:01:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.279452+0000", 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016249470000000001 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.279914+0000", 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000462106 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.280078+0000", 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016359600000000001 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:20.280198+0000", 2024-09-13T07:01:25.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119658 2024-09-13T07:01:25.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.468 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.468 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T07:01:25.468 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:21.292268+0000", 2024-09-13T07:01:25.468 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1655799550000001, 2024-09-13T07:01:25.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062857959999999997, 2024-09-13T07:01:25.468 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.469 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:25.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:25.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:25.469 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 9 2024-09-13T07:01:25.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.469 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.292268+0000", 2024-09-13T07:01:25.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.292268+0000", 2024-09-13T07:01:25.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.292264+0000", 2024-09-13T07:01:25.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999962 2024-09-13T07:01:25.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.292286+0000", 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2042999999999999e-05 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.292334+0000", 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7821999999999997e-05 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.292359+0000", 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5123e-05 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.292493+0000", 2024-09-13T07:01:25.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133767 2024-09-13T07:01:25.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:25.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.292509+0000", 2024-09-13T07:01:25.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-13T07:01:25.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.296147+0000", 2024-09-13T07:01:25.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003638147 2024-09-13T07:01:25.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.296318+0000", 2024-09-13T07:01:25.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017111599999999999 2024-09-13T07:01:25.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-13T07:01:25.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.296478+0000", 2024-09-13T07:01:25.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015971799999999999 2024-09-13T07:01:25.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.298051+0000", 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0015728669999999999 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.298455+0000", 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00040374200000000001 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.298509+0000", 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3885999999999999e-05 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.298554+0000", 2024-09-13T07:01:25.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5009000000000002e-05 2024-09-13T07:01:25.475 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.475 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.475 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.475 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:10 3.1c e25/22 3:3831950c:::rbd_directory:head v 25'1, mlcod=0'0)", 2024-09-13T07:01:25.475 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:21.300495+0000", 2024-09-13T07:01:25.475 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.157352532, 2024-09-13T07:01:25.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00088617600000000002, 2024-09-13T07:01:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.300495+0000", 2024-09-13T07:01:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.300495+0000", 2024-09-13T07:01:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.300495+0000", 2024-09-13T07:01:25.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:25.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.300501+0000", 2024-09-13T07:01:25.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6259999999999997e-06 2024-09-13T07:01:25.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.300512+0000", 2024-09-13T07:01:25.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:01:25.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.300519+0000", 2024-09-13T07:01:25.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:01:25.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.300569+0000", 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0000000000000002e-05 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.300584+0000", 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.416e-05 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.301345+0000", 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076192499999999999 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.301381+0000", 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5914000000000002e-05 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:13 3.13 e25/22 3:cc772670:::rbd_object_map.119438ec2c70:head v 25'1, mlcod=0'0)", 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:21.312432+0000", 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.145415828, 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021640980000000001, 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.480 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.312432+0000", 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.312432+0000", 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.312431+0000", 2024-09-13T07:01:25.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.312441+0000", 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0155e-05 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.312462+0000", 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0780000000000001e-05 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.312472+0000", 2024-09-13T07:01:25.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6730000000000004e-06 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.312528+0000", 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5770999999999998e-05 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.312547+0000", 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9052e-05 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.314517+0000", 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001970617 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:21.314596+0000", 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8835e-05 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:389 3.13 e26/22 3:cc772670:::rbd_object_map.119438ec2c70:head v 26'58, mlcod=25'56)", 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.470752+0000", 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.98709587300000001, 2024-09-13T07:01:25.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014997299999999999, 2024-09-13T07:01:25.485 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.485 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.485 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.470752+0000", 2024-09-13T07:01:25.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.470752+0000", 2024-09-13T07:01:25.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.470750+0000", 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.470766+0000", 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6133e-05 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.470798+0000", 2024-09-13T07:01:25.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1696000000000003e-05 2024-09-13T07:01:25.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.470811+0000", 2024-09-13T07:01:25.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3413e-05 2024-09-13T07:01:25.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.470928+0000", 2024-09-13T07:01:25.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116461 2024-09-13T07:01:25.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.470987+0000", 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9172000000000002e-05 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.472191+0000", 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012044860000000001 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.472252+0000", 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0426e-05 2024-09-13T07:01:25.488 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.489 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.489 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.489 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:404 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:25.489 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.489731+0000", 2024-09-13T07:01:25.489 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.96811691499999997, 2024-09-13T07:01:25.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048434500000000001, 2024-09-13T07:01:25.489 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.489 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:25.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:25.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:25.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 404 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.489731+0000", 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.489731+0000", 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.489729+0000", 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.489742+0000", 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3409e-05 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.489775+0000", 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3436000000000003e-05 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.489790+0000", 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4713e-05 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.489904+0000", 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011406800000000001 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.490048+0000", 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144263 2024-09-13T07:01:25.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.490215+0000", 2024-09-13T07:01:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016668600000000001 2024-09-13T07:01:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:432 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.531648+0000", 2024-09-13T07:01:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.92620004, 2024-09-13T07:01:25.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047588099999999999, 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 432 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.531648+0000", 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.531648+0000", 2024-09-13T07:01:25.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.531646+0000", 2024-09-13T07:01:25.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:25.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.531657+0000", 2024-09-13T07:01:25.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1800000000000001e-05 2024-09-13T07:01:25.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.531683+0000", 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5412e-05 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.531695+0000", 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2067e-05 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.531779+0000", 2024-09-13T07:01:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4351000000000006e-05 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.531929+0000", 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000149677 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.532124+0000", 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019473400000000001 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.497 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:438 3.1a 3:5a222e70:::rbd_data.1194d22ad098.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:25.498 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.539069+0000", 2024-09-13T07:01:25.498 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.91877835399999996, 2024-09-13T07:01:25.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016496785999999999, 2024-09-13T07:01:25.498 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.498 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.498 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:25.498 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:25.498 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:25.498 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 438 2024-09-13T07:01:25.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.498 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539069+0000", 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539069+0000", 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539067+0000", 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539081+0000", 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3776e-05 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539112+0000", 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1170000000000001e-05 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539127+0000", 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4959e-05 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539228+0000", 2024-09-13T07:01:25.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100892 2024-09-13T07:01:25.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539502+0000", 2024-09-13T07:01:25.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027425000000000003 2024-09-13T07:01:25.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.542903+0000", 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034009330000000001 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.555422+0000", 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012518995 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.555439+0000", 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6820000000000002e-05 2024-09-13T07:01:25.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.555504+0000", 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4899000000000003e-05 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.555566+0000", 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2292000000000006e-05 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(client.4500.0:438 3.1a e26/22 ondisk, result = 0)", 2024-09-13T07:01:25.503 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.550663+0000", 2024-09-13T07:01:25.504 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.90718443900000001, 2024-09-13T07:01:25.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047646750000000003, 2024-09-13T07:01:25.504 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.504 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:25.504 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.550663+0000", 2024-09-13T07:01:25.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.550663+0000", 2024-09-13T07:01:25.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.550661+0000", 2024-09-13T07:01:25.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:25.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.550673+0000", 2024-09-13T07:01:25.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2442e-05 2024-09-13T07:01:25.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.550710+0000", 2024-09-13T07:01:25.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6180000000000003e-05 2024-09-13T07:01:25.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.550728+0000", 2024-09-13T07:01:25.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8039000000000001e-05 2024-09-13T07:01:25.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.555407+0000", 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046793870000000001 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.555417+0000", 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0189e-05 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.555428+0000", 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0746e-05 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.507 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:440 3.1c e26/22 3:3b975cee:::rbd_data.1194d22ad098.0000000000000035:head v 26'3, mlcod=25'1)", 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.552701+0000", 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.90514643500000003, 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011695659999999999, 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.552701+0000", 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.552701+0000", 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.552700+0000", 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.552735+0000", 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5580999999999997e-05 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.552754+0000", 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8688000000000001e-05 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.552765+0000", 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0601999999999999e-05 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.552839+0000", 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4474999999999994e-05 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.552867+0000", 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8064000000000001e-05 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.553829+0000", 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096184999999999999 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.553871+0000", 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1705999999999997e-05 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.511 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:454 3.a e26/22 3:526b957d:::rbd_data.1194d22ad098.000000000000001f:head v 26'13, mlcod=25'11)", 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.577451+0000", 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.88039695399999995, 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0015942339999999999, 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.577451+0000", 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.577451+0000", 2024-09-13T07:01:25.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.577449+0000", 2024-09-13T07:01:25.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:25.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.577478+0000", 2024-09-13T07:01:25.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9020999999999999e-05 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.577505+0000", 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6597999999999998e-05 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.577519+0000", 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4795e-05 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.577626+0000", 2024-09-13T07:01:25.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010697200000000001 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.577675+0000", 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8158999999999997e-05 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.578976+0000", 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013019609999999999 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.579045+0000", 2024-09-13T07:01:25.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8528000000000005e-05 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:459 3.1c e26/22 3:3b975cee:::rbd_data.1194d22ad098.0000000000000035:head v 26'4, mlcod=26'2)", 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.581727+0000", 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.87612118299999997, 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014950079999999999, 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.581727+0000", 2024-09-13T07:01:25.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.581727+0000", 2024-09-13T07:01:25.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.581725+0000", 2024-09-13T07:01:25.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:25.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.581747+0000", 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1730999999999999e-05 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.581772+0000", 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5208e-05 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.581781+0000", 2024-09-13T07:01:25.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3870000000000001e-06 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.581873+0000", 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1668000000000004e-05 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.581928+0000", 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5399000000000003e-05 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.583131+0000", 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001202203 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.583222+0000", 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0879999999999997e-05 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:480 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.609908+0000", 2024-09-13T07:01:25.520 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.84793948299999999, 2024-09-13T07:01:25.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00040692899999999999, 2024-09-13T07:01:25.521 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.521 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:25.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:25.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:25.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:25.521 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 480 2024-09-13T07:01:25.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.521 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.609908+0000", 2024-09-13T07:01:25.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.609908+0000", 2024-09-13T07:01:25.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.609906+0000", 2024-09-13T07:01:25.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:25.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.609919+0000", 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.291e-05 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.609945+0000", 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6812999999999999e-05 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.609958+0000", 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2136e-05 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.610052+0000", 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4964999999999996e-05 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.610179+0000", 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012621999999999999 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.610315+0000", 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000136477 2024-09-13T07:01:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.525 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.525 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:485 3.11 e26/22 3:8f598c4d:::rbd_data.1194d22ad098.0000000000000065:head v 26'3, mlcod=25'1)", 2024-09-13T07:01:25.525 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.617350+0000", 2024-09-13T07:01:25.525 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.84049822900000004, 2024-09-13T07:01:25.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016747369999999999, 2024-09-13T07:01:25.525 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.525 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.525 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.617350+0000", 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.617350+0000", 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.617347+0000", 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.617416+0000", 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8379000000000004e-05 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.617453+0000", 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.697e-05 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.617470+0000", 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7170999999999999e-05 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.617600+0000", 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012999999999999999 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.617644+0000", 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4674e-05 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.618950+0000", 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001305531 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.619024+0000", 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4357999999999995e-05 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:497 3.3 3:c4a910e1:::rbd_data.1194d22ad098.0000000000000098:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.638159+0000", 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.81968865099999999, 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003992003, 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.529 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 497 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.638159+0000", 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.638159+0000", 2024-09-13T07:01:25.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.638156+0000", 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.638174+0000", 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7305000000000001e-05 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.638209+0000", 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5324999999999998e-05 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.638229+0000", 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0017000000000002e-05 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.638370+0000", 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000141144 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.638597+0000", 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000227125 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.639249+0000", 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00065127900000000005 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.641708+0000", 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024594460000000001 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.642032+0000", 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032425299999999999 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.642101+0000", 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8891000000000002e-05 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.642151+0000", 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9826e-05 2024-09-13T07:01:25.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:496 3.f e26/22 3:f3f96f88:::rbd_data.1194d22ad098.0000000000000097:head v 26'3, mlcod=25'1)", 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.638917+0000", 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.81893108400000003, 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023156090000000002, 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.638917+0000", 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.638917+0000", 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.638915+0000", 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.638976+0000", 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0992000000000002e-05 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.639000+0000", 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4566000000000002e-05 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.639017+0000", 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6413e-05 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.639140+0000", 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012281400000000001 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.639194+0000", 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4055000000000003e-05 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.641080+0000", 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001886408 2024-09-13T07:01:25.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.641232+0000", 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015212400000000001 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(client.4500.0:497 3.3 e26/22 ondisk, result = 0)", 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.641207+0000", 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.81664121599999995, 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051035399999999997, 2024-09-13T07:01:25.539 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.641207+0000", 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.641207+0000", 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.641205+0000", 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.641216+0000", 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0791000000000001e-05 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.641234+0000", 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7924999999999999e-05 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.641244+0000", 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0108999999999999e-05 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.641662+0000", 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041835 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.641694+0000", 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1951e-05 2024-09-13T07:01:25.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.641717+0000", 2024-09-13T07:01:25.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2659000000000001e-05 2024-09-13T07:01:25.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.543 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.543 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:504 3.14 3:296fe06b:::rbd_data.1194d22ad098.0000000000000026:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 393216~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:25.543 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.648303+0000", 2024-09-13T07:01:25.543 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.80954427500000004, 2024-09-13T07:01:25.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048173759999999999, 2024-09-13T07:01:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 504 2024-09-13T07:01:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.648303+0000", 2024-09-13T07:01:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.648303+0000", 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.648302+0000", 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.648377+0000", 2024-09-13T07:01:25.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5567000000000003e-05 2024-09-13T07:01:25.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.648407+0000", 2024-09-13T07:01:25.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9348000000000001e-05 2024-09-13T07:01:25.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.648421+0000", 2024-09-13T07:01:25.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3961999999999999e-05 2024-09-13T07:01:25.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.648562+0000", 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000141316 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.648847+0000", 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028503399999999998 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.649105+0000", 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025837700000000001 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:25.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.652571+0000", 2024-09-13T07:01:25.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034654360000000001 2024-09-13T07:01:25.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:25.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.653010+0000", 2024-09-13T07:01:25.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043964999999999998 2024-09-13T07:01:25.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.653075+0000", 2024-09-13T07:01:25.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4363999999999996e-05 2024-09-13T07:01:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.653121+0000", 2024-09-13T07:01:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6041999999999997e-05 2024-09-13T07:01:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:510 3.1b e26/22 3:d88c5702:::rbd_data.1194d22ad098.000000000000006c:head v 26'2, mlcod=0'0)", 2024-09-13T07:01:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.657659+0000", 2024-09-13T07:01:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.80018914399999996, 2024-09-13T07:01:25.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001494848, 2024-09-13T07:01:25.550 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.550 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.550 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657659+0000", 2024-09-13T07:01:25.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657659+0000", 2024-09-13T07:01:25.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657657+0000", 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657694+0000", 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7342000000000003e-05 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657720+0000", 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.597e-05 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657738+0000", 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7716000000000001e-05 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657870+0000", 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013224600000000001 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657894+0000", 2024-09-13T07:01:25.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3754e-05 2024-09-13T07:01:25.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.659096+0000", 2024-09-13T07:01:25.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012025499999999999 2024-09-13T07:01:25.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.659153+0000", 2024-09-13T07:01:25.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7077000000000003e-05 2024-09-13T07:01:25.553 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:509 3.e e26/22 3:71248949:::rbd_data.1194d22ad098.000000000000006b:head v 26'5, mlcod=25'3)", 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.657832+0000", 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.80001598799999996, 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021224619999999999, 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657832+0000", 2024-09-13T07:01:25.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657832+0000", 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657830+0000", 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:25.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657862+0000", 2024-09-13T07:01:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1992000000000001e-05 2024-09-13T07:01:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657887+0000", 2024-09-13T07:01:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5304e-05 2024-09-13T07:01:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.657905+0000", 2024-09-13T07:01:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:01:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.658062+0000", 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015763999999999999 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.658124+0000", 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1791999999999994e-05 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.659859+0000", 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001734818 2024-09-13T07:01:25.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:25.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.659954+0000", 2024-09-13T07:01:25.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5410999999999998e-05 2024-09-13T07:01:25.558 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.558 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.558 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.558 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:25.558 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:25.558 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:25.560 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:01:25.560 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:25.553+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:25.629 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-13T07:01:25.660 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:25.656+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:25.718 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:25.718 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:25.718 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:25.718 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:25.728 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:01:25.761 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:25.757+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:25.813 INFO:teuthology.orchestra.run.smithi118.stdout:117 writesame 0x49a5906 thru 0x49ab554 data_size 0x1(0x5c4f bytes) 2024-09-13T07:01:25.829 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:25.829 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:25.839 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:01:25.866 INFO:teuthology.orchestra.run.smithi118.stdout:119 write 0xdc3ac3b thru 0xdc48833 (0xdbf9 bytes) 2024-09-13T07:01:25.874 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:25.874 INFO:teuthology.orchestra.run.smithi118.stdout:121 write 0x5c1f390 thru 0x5c26084 (0x6cf5 bytes) 2024-09-13T07:01:25.881 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:25.877+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:25.926 INFO:teuthology.orchestra.run.smithi118.stdout:122 punch from 0x51498a0 to 0x514ca07, (0x3167 bytes) 2024-09-13T07:01:25.927 INFO:teuthology.orchestra.run.smithi118.stdout:123 write 0x267af1a thru 0x2688703 (0xd7ea bytes) 2024-09-13T07:01:25.938 INFO:teuthology.orchestra.run.smithi118.stdout:124 writesame 0x1b3ed0d thru 0x1b485ed data_size 0x1(0x98e1 bytes) 2024-09-13T07:01:25.963 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:25.960+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:25.980 INFO:teuthology.orchestra.run.smithi118.stdout:125 writesame 0x7c212c9 thru 0x7c27ac2 data_size 0x1(0x67fa bytes) 2024-09-13T07:01:25.997 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:25.998 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4226.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-13T07:01:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:11.223460+0000", 2024-09-13T07:01:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.77166205, 2024-09-13T07:01:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064502090000000002, 2024-09-13T07:01:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4226", 2024-09-13T07:01:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/1083678545", 2024-09-13T07:01:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7 2024-09-13T07:01:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223460+0000", 2024-09-13T07:01:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223460+0000", 2024-09-13T07:01:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223458+0000", 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223471+0000", 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2933e-05 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223501+0000", 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0647e-05 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223534+0000", 2024-09-13T07:01:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2478999999999999e-05 2024-09-13T07:01:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223675+0000", 2024-09-13T07:01:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014109999999999999 2024-09-13T07:01:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223697+0000", 2024-09-13T07:01:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1577e-05 2024-09-13T07:01:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.227045+0000", 2024-09-13T07:01:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033489090000000002 2024-09-13T07:01:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.227172+0000", 2024-09-13T07:01:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012647 2024-09-13T07:01:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-13T07:01:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.227430+0000", 2024-09-13T07:01:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025785499999999998 2024-09-13T07:01:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.229137+0000", 2024-09-13T07:01:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017073780000000001 2024-09-13T07:01:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.229729+0000", 2024-09-13T07:01:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00059170999999999996 2024-09-13T07:01:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.229841+0000", 2024-09-13T07:01:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112571 2024-09-13T07:01:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.229910+0000", 2024-09-13T07:01:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8965000000000001e-05 2024-09-13T07:01:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:439 3.16 3:6cdf2866:::rbd_data.1194d22ad098.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.539100+0000", 2024-09-13T07:01:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.456022226, 2024-09-13T07:01:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01241133, 2024-09-13T07:01:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 439 2024-09-13T07:01:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539100+0000", 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539100+0000", 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539098+0000", 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539113+0000", 2024-09-13T07:01:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5653000000000001e-05 2024-09-13T07:01:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539148+0000", 2024-09-13T07:01:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4489000000000001e-05 2024-09-13T07:01:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539162+0000", 2024-09-13T07:01:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4533e-05 2024-09-13T07:01:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539297+0000", 2024-09-13T07:01:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013447399999999999 2024-09-13T07:01:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539701+0000", 2024-09-13T07:01:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00040402400000000001 2024-09-13T07:01:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.542876+0000", 2024-09-13T07:01:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031755820000000001 2024-09-13T07:01:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551351+0000", 2024-09-13T07:01:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084747239999999995 2024-09-13T07:01:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551392+0000", 2024-09-13T07:01:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0921999999999999e-05 2024-09-13T07:01:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551457+0000", 2024-09-13T07:01:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4925999999999995e-05 2024-09-13T07:01:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551511+0000", 2024-09-13T07:01:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4446e-05 2024-09-13T07:01:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:440 3.1c 3:3b975cee:::rbd_data.1194d22ad098.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.539301+0000", 2024-09-13T07:01:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4558216669999999, 2024-09-13T07:01:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019168998999999999, 2024-09-13T07:01:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 440 2024-09-13T07:01:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539301+0000", 2024-09-13T07:01:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539301+0000", 2024-09-13T07:01:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539300+0000", 2024-09-13T07:01:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539308+0000", 2024-09-13T07:01:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-06 2024-09-13T07:01:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539347+0000", 2024-09-13T07:01:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9063999999999997e-05 2024-09-13T07:01:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539353+0000", 2024-09-13T07:01:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3459999999999999e-06 2024-09-13T07:01:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539438+0000", 2024-09-13T07:01:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4757000000000002e-05 2024-09-13T07:01:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539706+0000", 2024-09-13T07:01:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026819000000000002 2024-09-13T07:01:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.543844+0000", 2024-09-13T07:01:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004137338 2024-09-13T07:01:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558385+0000", 2024-09-13T07:01:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014541304999999999 2024-09-13T07:01:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558397+0000", 2024-09-13T07:01:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2269999999999999e-05 2024-09-13T07:01:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558437+0000", 2024-09-13T07:01:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0096999999999999e-05 2024-09-13T07:01:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558470+0000", 2024-09-13T07:01:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2242000000000001e-05 2024-09-13T07:01:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:491 3.d e26/22 3:b4666c7f:::rbd_data.1194d22ad098.000000000000008e:head v 26'11, mlcod=26'9)", 2024-09-13T07:01:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.626884+0000", 2024-09-13T07:01:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.368237741, 2024-09-13T07:01:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065664440000000003, 2024-09-13T07:01:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.626884+0000", 2024-09-13T07:01:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.626884+0000", 2024-09-13T07:01:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.626883+0000", 2024-09-13T07:01:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.626990+0000", 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010761500000000001 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.627034+0000", 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3782000000000002e-05 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.627052+0000", 2024-09-13T07:01:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8408000000000001e-05 2024-09-13T07:01:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.627171+0000", 2024-09-13T07:01:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118946 2024-09-13T07:01:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.627203+0000", 2024-09-13T07:01:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1136000000000001e-05 2024-09-13T07:01:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.633358+0000", 2024-09-13T07:01:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061555459999999996 2024-09-13T07:01:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.633451+0000", 2024-09-13T07:01:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2807999999999996e-05 2024-09-13T07:01:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:492 3.d e26/22 3:b3db72cd:::rbd_data.1194d22ad098.000000000000008f:head v 26'12, mlcod=26'9)", 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.628584+0000", 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.3665380919999999, 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054548449999999998, 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628584+0000", 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628584+0000", 2024-09-13T07:01:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628582+0000", 2024-09-13T07:01:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628617+0000", 2024-09-13T07:01:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4672999999999998e-05 2024-09-13T07:01:26.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628669+0000", 2024-09-13T07:01:26.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2404999999999997e-05 2024-09-13T07:01:26.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628695+0000", 2024-09-13T07:01:26.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5863e-05 2024-09-13T07:01:26.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628808+0000", 2024-09-13T07:01:26.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011226399999999999 2024-09-13T07:01:26.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628861+0000", 2024-09-13T07:01:26.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3436000000000001e-05 2024-09-13T07:01:26.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.633985+0000", 2024-09-13T07:01:26.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051237619999999996 2024-09-13T07:01:26.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.634039+0000", 2024-09-13T07:01:26.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4172000000000002e-05 2024-09-13T07:01:26.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.027 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.027 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:928 3.1d 3:b9c62a31:::rbd_data.1194d22ad098.000000000000002d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.027 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.486942+0000", 2024-09-13T07:01:26.027 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.50818063199999997, 2024-09-13T07:01:26.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005032992, 2024-09-13T07:01:26.028 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.028 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.028 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 928 2024-09-13T07:01:26.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.028 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.486942+0000", 2024-09-13T07:01:26.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.486942+0000", 2024-09-13T07:01:26.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.486941+0000", 2024-09-13T07:01:26.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.486947+0000", 2024-09-13T07:01:26.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1510000000000001e-06 2024-09-13T07:01:26.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.486958+0000", 2024-09-13T07:01:26.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1095e-05 2024-09-13T07:01:26.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.486967+0000", 2024-09-13T07:01:26.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5400000000000001e-06 2024-09-13T07:01:26.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487017+0000", 2024-09-13T07:01:26.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.918e-05 2024-09-13T07:01:26.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487133+0000", 2024-09-13T07:01:26.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116629 2024-09-13T07:01:26.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.491975+0000", 2024-09-13T07:01:26.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048414199999999999 2024-09-13T07:01:26.032 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.032 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.032 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.032 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:934 3.17 e27/22 3:ef63110b:::rbd_data.1194d22ad098.000000000000002a:head v 27'11, mlcod=27'9)", 2024-09-13T07:01:26.032 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.490330+0000", 2024-09-13T07:01:26.032 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.50479251199999997, 2024-09-13T07:01:26.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043918170000000001, 2024-09-13T07:01:26.032 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.032 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.032 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490330+0000", 2024-09-13T07:01:26.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490330+0000", 2024-09-13T07:01:26.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490329+0000", 2024-09-13T07:01:26.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:26.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490335+0000", 2024-09-13T07:01:26.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0809999999999999e-06 2024-09-13T07:01:26.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490346+0000", 2024-09-13T07:01:26.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0985000000000001e-05 2024-09-13T07:01:26.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490352+0000", 2024-09-13T07:01:26.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0669999999999997e-06 2024-09-13T07:01:26.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490382+0000", 2024-09-13T07:01:26.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0312e-05 2024-09-13T07:01:26.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490399+0000", 2024-09-13T07:01:26.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6257e-05 2024-09-13T07:01:26.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494678+0000", 2024-09-13T07:01:26.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004279695 2024-09-13T07:01:26.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494722+0000", 2024-09-13T07:01:26.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3089e-05 2024-09-13T07:01:26.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.036 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.037 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:940 3.1c 3:3c4f52df:::rbd_data.1194d22ad098.0000000000000031:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.037 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.492246+0000", 2024-09-13T07:01:26.037 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.50287578799999999, 2024-09-13T07:01:26.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054344950000000001, 2024-09-13T07:01:26.037 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.037 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.037 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 940 2024-09-13T07:01:26.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492246+0000", 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492246+0000", 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492246+0000", 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492252+0000", 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492264+0000", 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1742e-05 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492272+0000", 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8059999999999995e-06 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492329+0000", 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7017e-05 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492419+0000", 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0506000000000004e-05 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497681+0000", 2024-09-13T07:01:26.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005261712 2024-09-13T07:01:26.041 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.041 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.041 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.041 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:942 3.16 3:6cdf2866:::rbd_data.1194d22ad098.0000000000000033:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.041 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.492619+0000", 2024-09-13T07:01:26.041 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.50250355300000005, 2024-09-13T07:01:26.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048826800000000004, 2024-09-13T07:01:26.041 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.042 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.042 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.042 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.042 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.042 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 942 2024-09-13T07:01:26.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.042 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492619+0000", 2024-09-13T07:01:26.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492619+0000", 2024-09-13T07:01:26.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492618+0000", 2024-09-13T07:01:26.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:26.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492626+0000", 2024-09-13T07:01:26.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.576e-06 2024-09-13T07:01:26.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492638+0000", 2024-09-13T07:01:26.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2163e-05 2024-09-13T07:01:26.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492645+0000", 2024-09-13T07:01:26.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8229999999999997e-06 2024-09-13T07:01:26.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492700+0000", 2024-09-13T07:01:26.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5526000000000003e-05 2024-09-13T07:01:26.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492798+0000", 2024-09-13T07:01:26.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7669000000000003e-05 2024-09-13T07:01:26.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:26.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.493010+0000", 2024-09-13T07:01:26.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021179599999999999 2024-09-13T07:01:26.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495164+0000", 2024-09-13T07:01:26.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021548890000000001 2024-09-13T07:01:26.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497417+0000", 2024-09-13T07:01:26.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022523059999999999 2024-09-13T07:01:26.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497463+0000", 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6375000000000002e-05 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497501+0000", 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8256000000000001e-05 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:943 3.1d 3:b9c62a31:::rbd_data.1194d22ad098.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.047 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.492659+0000", 2024-09-13T07:01:26.048 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.50246298199999995, 2024-09-13T07:01:26.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050088149999999998, 2024-09-13T07:01:26.048 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.048 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.048 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 943 2024-09-13T07:01:26.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.048 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492659+0000", 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492659+0000", 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492659+0000", 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492663+0000", 2024-09-13T07:01:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.822e-06 2024-09-13T07:01:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492669+0000", 2024-09-13T07:01:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2709999999999998e-06 2024-09-13T07:01:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492676+0000", 2024-09-13T07:01:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4660000000000002e-06 2024-09-13T07:01:26.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492820+0000", 2024-09-13T07:01:26.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014342599999999999 2024-09-13T07:01:26.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492897+0000", 2024-09-13T07:01:26.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7744000000000006e-05 2024-09-13T07:01:26.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:26.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.493093+0000", 2024-09-13T07:01:26.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019583099999999999 2024-09-13T07:01:26.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497452+0000", 2024-09-13T07:01:26.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043583500000000004 2024-09-13T07:01:26.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497603+0000", 2024-09-13T07:01:26.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000151268 2024-09-13T07:01:26.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497641+0000", 2024-09-13T07:01:26.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8374000000000002e-05 2024-09-13T07:01:26.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497668+0000", 2024-09-13T07:01:26.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6787999999999998e-05 2024-09-13T07:01:26.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.053 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.054 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:945 3.1c 3:3b975cee:::rbd_data.1194d22ad098.0000000000000035:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.054 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.494952+0000", 2024-09-13T07:01:26.054 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.50016987599999996, 2024-09-13T07:01:26.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058335799999999997, 2024-09-13T07:01:26.054 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.054 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.054 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 945 2024-09-13T07:01:26.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.054 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494952+0000", 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494952+0000", 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494951+0000", 2024-09-13T07:01:26.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494958+0000", 2024-09-13T07:01:26.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7870000000000004e-06 2024-09-13T07:01:26.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494972+0000", 2024-09-13T07:01:26.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.437e-05 2024-09-13T07:01:26.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494981+0000", 2024-09-13T07:01:26.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6070000000000008e-06 2024-09-13T07:01:26.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495034+0000", 2024-09-13T07:01:26.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3504999999999997e-05 2024-09-13T07:01:26.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495132+0000", 2024-09-13T07:01:26.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7428000000000003e-05 2024-09-13T07:01:26.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:26.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495319+0000", 2024-09-13T07:01:26.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018667099999999999 2024-09-13T07:01:26.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500372+0000", 2024-09-13T07:01:26.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050530929999999998 2024-09-13T07:01:26.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500681+0000", 2024-09-13T07:01:26.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030899299999999997 2024-09-13T07:01:26.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500740+0000", 2024-09-13T07:01:26.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9713000000000002e-05 2024-09-13T07:01:26.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500786+0000", 2024-09-13T07:01:26.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5566999999999999e-05 2024-09-13T07:01:26.059 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.059 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.060 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.060 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:947 3.1f e27/22 3:fb633d24:::rbd_data.1194d22ad098.0000000000000030:head v 27'9, mlcod=26'7)", 2024-09-13T07:01:26.060 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.495855+0000", 2024-09-13T07:01:26.060 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.49926753499999998, 2024-09-13T07:01:26.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045237200000000002, 2024-09-13T07:01:26.060 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.060 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.060 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495855+0000", 2024-09-13T07:01:26.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495855+0000", 2024-09-13T07:01:26.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495854+0000", 2024-09-13T07:01:26.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495862+0000", 2024-09-13T07:01:26.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6839999999999996e-06 2024-09-13T07:01:26.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495871+0000", 2024-09-13T07:01:26.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7159999999999998e-06 2024-09-13T07:01:26.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495879+0000", 2024-09-13T07:01:26.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.435e-06 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495928+0000", 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8838000000000002e-05 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495958+0000", 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9921999999999999e-05 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500341+0000", 2024-09-13T07:01:26.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043834690000000001 2024-09-13T07:01:26.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500378+0000", 2024-09-13T07:01:26.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7395999999999999e-05 2024-09-13T07:01:26.064 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.064 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.064 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.064 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:969 3.d e27/22 3:b06a3fa5:::rbd_data.1194d22ad098.0000000000000041:head v 27'18, mlcod=27'16)", 2024-09-13T07:01:26.064 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.504201+0000", 2024-09-13T07:01:26.065 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.49092122199999999, 2024-09-13T07:01:26.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077030220000000003, 2024-09-13T07:01:26.065 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.065 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.065 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504201+0000", 2024-09-13T07:01:26.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504201+0000", 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504200+0000", 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504209+0000", 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4800000000000001e-06 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504223+0000", 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4207e-05 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504230+0000", 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3819999999999998e-06 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504270+0000", 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9545999999999997e-05 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504290+0000", 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0636999999999999e-05 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.511866+0000", 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0075757680000000001 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.511904+0000", 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7821000000000003e-05 2024-09-13T07:01:26.068 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:978 3.1d 3:bb753f8d:::rbd_data.1194d22ad098.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.504609+0000", 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.490512964, 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077548790000000001, 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.069 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 978 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504609+0000", 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504609+0000", 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504608+0000", 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504614+0000", 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6799999999999998e-06 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504626+0000", 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1858999999999999e-05 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504632+0000", 2024-09-13T07:01:26.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0460000000000002e-06 2024-09-13T07:01:26.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504685+0000", 2024-09-13T07:01:26.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3542000000000003e-05 2024-09-13T07:01:26.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504781+0000", 2024-09-13T07:01:26.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5704000000000004e-05 2024-09-13T07:01:26.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.505000+0000", 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021894699999999999 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.511911+0000", 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069104489999999999 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.512277+0000", 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000366275 2024-09-13T07:01:26.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.512314+0000", 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7571999999999999e-05 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.512364+0000", 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9727e-05 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:999 3.16 3:6cc0310a:::rbd_data.1194d22ad098.0000000000000051:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.516960+0000", 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.47816176700000002, 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044275360000000001, 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 999 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.516960+0000", 2024-09-13T07:01:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.516960+0000", 2024-09-13T07:01:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.516960+0000", 2024-09-13T07:01:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.516964+0000", 2024-09-13T07:01:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:01:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.516978+0000", 2024-09-13T07:01:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3385e-05 2024-09-13T07:01:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.516983+0000", 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.502e-06 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.517040+0000", 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6740999999999998e-05 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.517153+0000", 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113471 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.521388+0000", 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042346969999999999 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1008 3.16 3:6cc0310a:::rbd_data.1194d22ad098.0000000000000051:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.522237+0000", 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.47288497499999999, 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050442450000000002, 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.079 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.080 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.080 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.080 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1008 2024-09-13T07:01:26.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.080 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522237+0000", 2024-09-13T07:01:26.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522237+0000", 2024-09-13T07:01:26.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522236+0000", 2024-09-13T07:01:26.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:26.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522247+0000", 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1511999999999999e-05 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522264+0000", 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6796999999999999e-05 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522272+0000", 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3920000000000003e-06 2024-09-13T07:01:26.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522343+0000", 2024-09-13T07:01:26.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0909999999999997e-05 2024-09-13T07:01:26.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522423+0000", 2024-09-13T07:01:26.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9233000000000005e-05 2024-09-13T07:01:26.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:26.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522600+0000", 2024-09-13T07:01:26.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017731299999999999 2024-09-13T07:01:26.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.526882+0000", 2024-09-13T07:01:26.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042823760000000001 2024-09-13T07:01:26.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527160+0000", 2024-09-13T07:01:26.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027766800000000001 2024-09-13T07:01:26.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527231+0000", 2024-09-13T07:01:26.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.161e-05 2024-09-13T07:01:26.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527282+0000", 2024-09-13T07:01:26.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.003e-05 2024-09-13T07:01:26.085 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.085 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.085 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.086 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1015 3.17 e27/22 3:ecdfb694:::rbd_data.1194d22ad098.0000000000000056:head v 27'13, mlcod=27'11)", 2024-09-13T07:01:26.086 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.525303+0000", 2024-09-13T07:01:26.086 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.46981918900000003, 2024-09-13T07:01:26.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052399150000000004, 2024-09-13T07:01:26.086 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.086 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.086 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525303+0000", 2024-09-13T07:01:26.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525303+0000", 2024-09-13T07:01:26.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525302+0000", 2024-09-13T07:01:26.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525312+0000", 2024-09-13T07:01:26.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7419999999999992e-06 2024-09-13T07:01:26.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525331+0000", 2024-09-13T07:01:26.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9176000000000001e-05 2024-09-13T07:01:26.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525339+0000", 2024-09-13T07:01:26.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5399999999999996e-06 2024-09-13T07:01:26.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525394+0000", 2024-09-13T07:01:26.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4941999999999997e-05 2024-09-13T07:01:26.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525412+0000", 2024-09-13T07:01:26.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7887999999999999e-05 2024-09-13T07:01:26.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.530495+0000", 2024-09-13T07:01:26.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050831640000000003 2024-09-13T07:01:26.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.530543+0000", 2024-09-13T07:01:26.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7518999999999997e-05 2024-09-13T07:01:26.090 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.090 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.090 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.090 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1020 3.5 3:a20826fb:::rbd_data.1194d22ad098.000000000000005a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.527436+0000", 2024-09-13T07:01:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.4676864, 2024-09-13T07:01:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052420119999999999, 2024-09-13T07:01:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1020 2024-09-13T07:01:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527436+0000", 2024-09-13T07:01:26.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527436+0000", 2024-09-13T07:01:26.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527435+0000", 2024-09-13T07:01:26.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:26.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527441+0000", 2024-09-13T07:01:26.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7549999999999997e-06 2024-09-13T07:01:26.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527457+0000", 2024-09-13T07:01:26.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5957999999999999e-05 2024-09-13T07:01:26.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527471+0000", 2024-09-13T07:01:26.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3773e-05 2024-09-13T07:01:26.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527540+0000", 2024-09-13T07:01:26.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9375000000000006e-05 2024-09-13T07:01:26.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527677+0000", 2024-09-13T07:01:26.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000136995 2024-09-13T07:01:26.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:26.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527951+0000", 2024-09-13T07:01:26.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027340300000000002 2024-09-13T07:01:26.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.529334+0000", 2024-09-13T07:01:26.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013827399999999999 2024-09-13T07:01:26.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.532580+0000", 2024-09-13T07:01:26.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003246302 2024-09-13T07:01:26.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.532628+0000", 2024-09-13T07:01:26.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8622000000000003e-05 2024-09-13T07:01:26.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.532678+0000", 2024-09-13T07:01:26.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359e-05 2024-09-13T07:01:26.097 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.097 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.097 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.097 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1023 3.17 e27/22 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head v 27'15, mlcod=27'11)", 2024-09-13T07:01:26.097 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.528842+0000", 2024-09-13T07:01:26.097 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.466280062, 2024-09-13T07:01:26.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092562140000000005, 2024-09-13T07:01:26.098 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.098 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.098 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528842+0000", 2024-09-13T07:01:26.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528842+0000", 2024-09-13T07:01:26.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528841+0000", 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528853+0000", 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1518000000000001e-05 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528874+0000", 2024-09-13T07:01:26.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1455e-05 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528886+0000", 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1404e-05 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528953+0000", 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7644999999999994e-05 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528975+0000", 2024-09-13T07:01:26.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1492999999999999e-05 2024-09-13T07:01:26.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.538036+0000", 2024-09-13T07:01:26.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090608589999999992 2024-09-13T07:01:26.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.538098+0000", 2024-09-13T07:01:26.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2830000000000007e-05 2024-09-13T07:01:26.101 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.102 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.102 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.102 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1048 3.d e27/22 3:b3db72cd:::rbd_data.1194d22ad098.000000000000008f:head v 27'24, mlcod=27'19)", 2024-09-13T07:01:26.102 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.544658+0000", 2024-09-13T07:01:26.102 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.45046461100000001, 2024-09-13T07:01:26.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057056909999999997, 2024-09-13T07:01:26.102 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.102 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.102 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544658+0000", 2024-09-13T07:01:26.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544658+0000", 2024-09-13T07:01:26.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544656+0000", 2024-09-13T07:01:26.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544667+0000", 2024-09-13T07:01:26.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0210000000000001e-05 2024-09-13T07:01:26.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544686+0000", 2024-09-13T07:01:26.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9763e-05 2024-09-13T07:01:26.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544699+0000", 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2836e-05 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544778+0000", 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8611999999999999e-05 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544808+0000", 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0484000000000002e-05 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.550319+0000", 2024-09-13T07:01:26.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055108090000000002 2024-09-13T07:01:26.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.550363+0000", 2024-09-13T07:01:26.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4125999999999997e-05 2024-09-13T07:01:26.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.106 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.106 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.106 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:26.107 INFO:teuthology.orchestra.run.smithi118.stdout:128 trunc from 0xdc48834 to 0xb3b31d5 2024-09-13T07:01:26.108 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:01:26.109 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:26.101+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:26.140 INFO:teuthology.orchestra.run.smithi118.stdout:130 trunc from 0xb3b31d5 to 0x3f3ecd8 2024-09-13T07:01:26.180 INFO:teuthology.orchestra.run.smithi118.stdout:133 writesame 0x7f95e80 thru 0x7f99ad4 data_size 0x1(0x3c55 bytes) 2024-09-13T07:01:26.182 INFO:teuthology.orchestra.run.smithi118.stdout:135 read 0x2e02aaf thru 0x2e0a72c (0x7c7e bytes) 2024-09-13T07:01:26.183 INFO:teuthology.orchestra.run.smithi118.stdout:139 punch from 0x3ab19ca to 0x3abd59e, (0xbbd4 bytes) 2024-09-13T07:01:26.190 INFO:teuthology.orchestra.run.smithi118.stdout:141 trunc from 0x7f99ad5 to 0xb11d56e 2024-09-13T07:01:26.199 INFO:teuthology.orchestra.run.smithi118.stdout:142 read 0x87a1ee6 thru 0x87b17bb (0xf8d6 bytes) 2024-09-13T07:01:26.200 INFO:teuthology.orchestra.run.smithi118.stdout:143 read 0xa5976e0 thru 0xa59b7d2 (0x40f3 bytes) 2024-09-13T07:01:26.209 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:26.204+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:26.212 INFO:teuthology.orchestra.run.smithi118.stdout:144 trunc from 0xb11d56e to 0x5a06cf1 2024-09-13T07:01:26.223 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:26.223 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:26.223 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:26.223 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:26.226 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:26.233 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:01:26.247 INFO:teuthology.orchestra.run.smithi118.stdout:150 write 0x9d936b5 thru 0x9d95570 (0x1ebc bytes) 2024-09-13T07:01:26.257 INFO:teuthology.orchestra.run.smithi118.stdout:151 trunc from 0x9d95571 to 0xcb6761e 2024-09-13T07:01:26.266 INFO:teuthology.orchestra.run.smithi118.stdout:152 read 0x6955856 thru 0x695cada (0x7285 bytes) 2024-09-13T07:01:26.267 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:26.268 INFO:teuthology.orchestra.run.smithi118.stdout:154 writesame 0x402b3d3 thru 0x4037ddc data_size 0xb(0xca0a bytes) 2024-09-13T07:01:26.275 INFO:teuthology.orchestra.run.smithi118.stdout:155 write 0x10466f7 thru 0x1050e6a (0xa774 bytes) 2024-09-13T07:01:26.279 INFO:teuthology.orchestra.run.smithi118.stdout:156 write 0x269b85b thru 0x26ab320 (0xfac6 bytes) 2024-09-13T07:01:26.282 INFO:teuthology.orchestra.run.smithi118.stdout:157 write 0x72643d4 thru 0x7269d50 (0x597d bytes) 2024-09-13T07:01:26.285 INFO:teuthology.orchestra.run.smithi118.stdout:158 punch from 0xa0c72e to 0xa1bc14, (0xf4e6 bytes) 2024-09-13T07:01:26.285 INFO:teuthology.orchestra.run.smithi118.stdout:159 write 0x846163b thru 0x84626c6 (0x108c bytes) 2024-09-13T07:01:26.298 INFO:teuthology.orchestra.run.smithi118.stdout:160 trunc from 0xcb6761e to 0x893d417 2024-09-13T07:01:26.309 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:26.304+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:26.313 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:26.313 INFO:teuthology.orchestra.run.smithi118.stdout:164 punch from 0x11b2c2a to 0x11b7f34, (0x530a bytes) 2024-09-13T07:01:26.313 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:26.313 INFO:teuthology.orchestra.run.smithi118.stdout:167 read 0x6509457 thru 0x650fdff (0x69a9 bytes) 2024-09-13T07:01:26.314 INFO:teuthology.orchestra.run.smithi118.stdout:168 punch from 0x4def824 to 0x4dfd002, (0xd7de bytes) 2024-09-13T07:01:26.314 INFO:teuthology.orchestra.run.smithi118.stdout:169 write 0x725318d thru 0x725b49b (0x830f bytes) 2024-09-13T07:01:26.317 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:26.317 INFO:teuthology.orchestra.run.smithi118.stdout:176 punch from 0x7585ff5 to 0x7586db0, (0xdbb bytes) 2024-09-13T07:01:26.317 INFO:teuthology.orchestra.run.smithi118.stdout:178 read 0x1686bc9 thru 0x168802f (0x1467 bytes) 2024-09-13T07:01:26.318 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T07:01:26.320 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:01:26.324 INFO:teuthology.orchestra.run.smithi118.stdout:180 write 0x412bbaa thru 0x412ce96 (0x12ed bytes) 2024-09-13T07:01:26.330 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T07:01:26.330 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:26.331 INFO:teuthology.orchestra.run.smithi118.stdout:182 read 0x2f551b9 thru 0x2f56426 (0x126e bytes) 2024-09-13T07:01:26.337 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:26.337 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:26.339 INFO:teuthology.orchestra.run.smithi118.stdout:184 trunc from 0x893d417 to 0x3f9ebb6 2024-09-13T07:01:26.349 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:01:26.368 INFO:teuthology.orchestra.run.smithi118.stdout:186 write 0x6ba0b31 thru 0x6ba850a (0x79da bytes) 2024-09-13T07:01:26.371 INFO:teuthology.orchestra.run.smithi118.stdout:187 read 0x2152848 thru 0x215a68f (0x7e48 bytes) 2024-09-13T07:01:26.372 INFO:teuthology.orchestra.run.smithi118.stdout:188 write 0x6921e11 thru 0x692ebd7 (0xcdc7 bytes) 2024-09-13T07:01:26.375 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:26.389 INFO:teuthology.orchestra.run.smithi118.stdout:191 write 0x769c662 thru 0x76aacd1 (0xe670 bytes) 2024-09-13T07:01:26.389 INFO:teuthology.orchestra.run.smithi118.stdout:192 read 0x3c43050 thru 0x3c51a40 (0xe9f1 bytes) 2024-09-13T07:01:26.391 INFO:teuthology.orchestra.run.smithi118.stdout:193 read 0x9d9973 thru 0x9e0d91 (0x741f bytes) 2024-09-13T07:01:26.409 INFO:teuthology.orchestra.run.smithi118.stdout:194 write 0xb874b29 thru 0xb87e79b (0x9c73 bytes) 2024-09-13T07:01:26.410 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:26.404+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:26.414 INFO:teuthology.orchestra.run.smithi118.stdout:195 write 0x10074c0 thru 0x1010a5c (0x959d bytes) 2024-09-13T07:01:26.417 INFO:teuthology.orchestra.run.smithi118.stdout:197 punch from 0x9a28310 to 0x9a301f1, (0x7ee1 bytes) 2024-09-13T07:01:26.417 INFO:teuthology.orchestra.run.smithi118.stdout:198 read 0xb6a2802 thru 0xb6ae889 (0xc088 bytes) 2024-09-13T07:01:26.418 INFO:teuthology.orchestra.run.smithi118.stdout:201 writesame 0x4aa7f14 thru 0x4aa8c74 data_size 0x5(0xd61 bytes) 2024-09-13T07:01:26.421 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:26.421 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:26.421 INFO:teuthology.orchestra.run.smithi118.stdout:204 read 0xa793fdc thru 0xa798bb2 (0x4bd7 bytes) 2024-09-13T07:01:26.422 INFO:teuthology.orchestra.run.smithi118.stdout:205 read 0x82448ee thru 0x8250083 (0xb796 bytes) 2024-09-13T07:01:26.423 INFO:teuthology.orchestra.run.smithi118.stdout:206 write 0x8ccc2a0 thru 0x8cd1c3d (0x599e bytes) 2024-09-13T07:01:26.430 INFO:teuthology.orchestra.run.smithi118.stdout:207 trunc from 0xb87e79c to 0x8d96c71 2024-09-13T07:01:26.446 INFO:teuthology.orchestra.run.smithi118.stdout:208 read 0x7262060 thru 0x7262a95 (0xa36 bytes) 2024-09-13T07:01:26.454 INFO:teuthology.orchestra.run.smithi118.stdout:210 trunc from 0x8d96c71 to 0x45beb13 2024-09-13T07:01:26.456 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:581 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.416139+0000", 2024-09-13T07:01:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0356308219999999, 2024-09-13T07:01:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061636429999999999, 2024-09-13T07:01:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 581 2024-09-13T07:01:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416139+0000", 2024-09-13T07:01:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416139+0000", 2024-09-13T07:01:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416136+0000", 2024-09-13T07:01:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416171+0000", 2024-09-13T07:01:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4545000000000002e-05 2024-09-13T07:01:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416203+0000", 2024-09-13T07:01:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2085000000000002e-05 2024-09-13T07:01:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416224+0000", 2024-09-13T07:01:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1157000000000001e-05 2024-09-13T07:01:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416339+0000", 2024-09-13T07:01:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115455 2024-09-13T07:01:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416349+0000", 2024-09-13T07:01:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5729999999999997e-06 2024-09-13T07:01:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.419615+0000", 2024-09-13T07:01:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032657139999999999 2024-09-13T07:01:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.419766+0000", 2024-09-13T07:01:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015095000000000001 2024-09-13T07:01:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.420094+0000", 2024-09-13T07:01:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032800599999999999 2024-09-13T07:01:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.421267+0000", 2024-09-13T07:01:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011733889999999999 2024-09-13T07:01:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.422161+0000", 2024-09-13T07:01:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089368499999999997 2024-09-13T07:01:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.422233+0000", 2024-09-13T07:01:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2619000000000004e-05 2024-09-13T07:01:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.422303+0000", 2024-09-13T07:01:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9264999999999995e-05 2024-09-13T07:01:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:602 3.10 3:0b8a5fd5:::rbd_data.1194d22ad098.0000000000000004:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.442777+0000", 2024-09-13T07:01:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0089922760000001, 2024-09-13T07:01:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040112359999999996, 2024-09-13T07:01:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 602 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.442777+0000", 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.442777+0000", 2024-09-13T07:01:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.442775+0000", 2024-09-13T07:01:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.442793+0000", 2024-09-13T07:01:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7305000000000001e-05 2024-09-13T07:01:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.442816+0000", 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3760999999999998e-05 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.442832+0000", 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5106000000000001e-05 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.442927+0000", 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5025999999999994e-05 2024-09-13T07:01:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.443244+0000", 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031697700000000002 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.446789+0000", 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003545201 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:929 3.a 3:51375ed8:::rbd_data.1194d22ad098.0000000000000028:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.487437+0000", 2024-09-13T07:01:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.96433316999999996, 2024-09-13T07:01:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046615179999999999, 2024-09-13T07:01:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 929 2024-09-13T07:01:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487437+0000", 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487437+0000", 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487435+0000", 2024-09-13T07:01:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487443+0000", 2024-09-13T07:01:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1440000000000003e-06 2024-09-13T07:01:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487460+0000", 2024-09-13T07:01:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6815000000000001e-05 2024-09-13T07:01:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487469+0000", 2024-09-13T07:01:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8880000000000003e-06 2024-09-13T07:01:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487518+0000", 2024-09-13T07:01:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9042999999999998e-05 2024-09-13T07:01:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487607+0000", 2024-09-13T07:01:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9319000000000003e-05 2024-09-13T07:01:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487802+0000", 2024-09-13T07:01:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019425599999999999 2024-09-13T07:01:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.489153+0000", 2024-09-13T07:01:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001350895 2024-09-13T07:01:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.491933+0000", 2024-09-13T07:01:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027807130000000002 2024-09-13T07:01:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.491991+0000", 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-05 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492098+0000", 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010663299999999999 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:935 3.1e e27/22 3:798a593f:::rbd_data.1194d22ad098.0000000000000029:head v 27'6, mlcod=27'4)", 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.490905+0000", 2024-09-13T07:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.96086484299999997, 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040278900000000001, 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490905+0000", 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490905+0000", 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490904+0000", 2024-09-13T07:01:26.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:26.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490911+0000", 2024-09-13T07:01:26.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7569999999999998e-06 2024-09-13T07:01:26.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490923+0000", 2024-09-13T07:01:26.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1819e-05 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490932+0000", 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.833e-06 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490970+0000", 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8680000000000002e-05 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.490987+0000", 2024-09-13T07:01:26.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6228000000000001e-05 2024-09-13T07:01:26.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494895+0000", 2024-09-13T07:01:26.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039087549999999999 2024-09-13T07:01:26.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494933+0000", 2024-09-13T07:01:26.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7509999999999998e-05 2024-09-13T07:01:26.479 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.479 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.480 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.480 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:936 3.a 3:52cbfadd:::rbd_data.1194d22ad098.000000000000002f:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.480 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.491074+0000", 2024-09-13T07:01:26.480 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.96069561599999997, 2024-09-13T07:01:26.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039156900000000003, 2024-09-13T07:01:26.480 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.480 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.480 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 936 2024-09-13T07:01:26.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.481 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.491074+0000", 2024-09-13T07:01:26.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.491074+0000", 2024-09-13T07:01:26.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.491073+0000", 2024-09-13T07:01:26.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.491077+0000", 2024-09-13T07:01:26.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1069999999999998e-06 2024-09-13T07:01:26.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.491090+0000", 2024-09-13T07:01:26.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-13T07:01:26.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.491094+0000", 2024-09-13T07:01:26.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000002e-06 2024-09-13T07:01:26.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.491136+0000", 2024-09-13T07:01:26.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1547e-05 2024-09-13T07:01:26.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.491291+0000", 2024-09-13T07:01:26.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000155375 2024-09-13T07:01:26.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494990+0000", 2024-09-13T07:01:26.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036985249999999998 2024-09-13T07:01:26.484 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:944 3.10 3:0ef08359:::rbd_data.1194d22ad098.0000000000000034:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.492669+0000", 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.95910025099999996, 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049892310000000002, 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.485 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 944 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492669+0000", 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492669+0000", 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492668+0000", 2024-09-13T07:01:26.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492675+0000", 2024-09-13T07:01:26.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2600000000000002e-06 2024-09-13T07:01:26.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492707+0000", 2024-09-13T07:01:26.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1899000000000001e-05 2024-09-13T07:01:26.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492715+0000", 2024-09-13T07:01:26.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.422e-06 2024-09-13T07:01:26.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492750+0000", 2024-09-13T07:01:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4737000000000003e-05 2024-09-13T07:01:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492818+0000", 2024-09-13T07:01:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8021000000000005e-05 2024-09-13T07:01:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492969+0000", 2024-09-13T07:01:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000151614 2024-09-13T07:01:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494065+0000", 2024-09-13T07:01:26.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010958000000000001 2024-09-13T07:01:26.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497536+0000", 2024-09-13T07:01:26.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034705970000000002 2024-09-13T07:01:26.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497614+0000", 2024-09-13T07:01:26.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8617999999999998e-05 2024-09-13T07:01:26.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497659+0000", 2024-09-13T07:01:26.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4263e-05 2024-09-13T07:01:26.491 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.491 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.491 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.492 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:949 3.a 3:52cbfadd:::rbd_data.1194d22ad098.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.492 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.495440+0000", 2024-09-13T07:01:26.492 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.95632944200000003, 2024-09-13T07:01:26.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005241791, 2024-09-13T07:01:26.492 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.492 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.492 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.492 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.492 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.492 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 949 2024-09-13T07:01:26.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.492 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495440+0000", 2024-09-13T07:01:26.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495440+0000", 2024-09-13T07:01:26.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495439+0000", 2024-09-13T07:01:26.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495446+0000", 2024-09-13T07:01:26.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2269999999999996e-06 2024-09-13T07:01:26.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495460+0000", 2024-09-13T07:01:26.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3247999999999999e-05 2024-09-13T07:01:26.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495467+0000", 2024-09-13T07:01:26.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5129999999999999e-06 2024-09-13T07:01:26.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495521+0000", 2024-09-13T07:01:26.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4020000000000001e-05 2024-09-13T07:01:26.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495618+0000", 2024-09-13T07:01:26.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6934000000000004e-05 2024-09-13T07:01:26.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:26.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495860+0000", 2024-09-13T07:01:26.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024149800000000001 2024-09-13T07:01:26.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500444+0000", 2024-09-13T07:01:26.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045838909999999997 2024-09-13T07:01:26.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500595+0000", 2024-09-13T07:01:26.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015186099999999999 2024-09-13T07:01:26.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500648+0000", 2024-09-13T07:01:26.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2877999999999998e-05 2024-09-13T07:01:26.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500682+0000", 2024-09-13T07:01:26.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3818e-05 2024-09-13T07:01:26.498 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.498 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.498 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.498 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.498 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:26.498 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.392849069, 2024-09-13T07:01:26.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:26.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:01:26.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:01:26.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:01:26.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:01:26.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:01:26.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:01:26.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:01:26.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:01:26.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:01:26.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:01:26.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:01:26.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:01:26.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:26.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:01:26.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:01:26.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:01:26.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:01:26.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:01:26.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:01:26.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:01:26.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:01:26.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:01:26.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:01:26.504 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.504 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.504 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.505 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1534 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14365141] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.505 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:26.505 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.38602151800000001, 2024-09-13T07:01:26.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040478761000000002, 2024-09-13T07:01:26.505 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.505 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.505 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1534 2024-09-13T07:01:26.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.506 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:26.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:26.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065747+0000", 2024-09-13T07:01:26.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:26.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075258+0000", 2024-09-13T07:01:26.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095108710000000006 2024-09-13T07:01:26.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075288+0000", 2024-09-13T07:01:26.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0085000000000001e-05 2024-09-13T07:01:26.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075305+0000", 2024-09-13T07:01:26.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7748e-05 2024-09-13T07:01:26.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075367+0000", 2024-09-13T07:01:26.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1867999999999998e-05 2024-09-13T07:01:26.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075520+0000", 2024-09-13T07:01:26.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015298800000000001 2024-09-13T07:01:26.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:26.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.076247+0000", 2024-09-13T07:01:26.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072720499999999997 2024-09-13T07:01:26.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105157+0000", 2024-09-13T07:01:26.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028909642999999999 2024-09-13T07:01:26.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106124+0000", 2024-09-13T07:01:26.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096729400000000005 2024-09-13T07:01:26.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106173+0000", 2024-09-13T07:01:26.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8196999999999999e-05 2024-09-13T07:01:26.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106227+0000", 2024-09-13T07:01:26.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4388999999999997e-05 2024-09-13T07:01:26.511 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.511 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.511 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.511 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1557 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.512 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:26.512 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.311255848, 2024-09-13T07:01:26.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012339057, 2024-09-13T07:01:26.512 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.512 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.512 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.512 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.512 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.512 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1557 2024-09-13T07:01:26.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.512 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:26.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:26.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140502+0000", 2024-09-13T07:01:26.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999881 2024-09-13T07:01:26.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140526+0000", 2024-09-13T07:01:26.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4049999999999998e-05 2024-09-13T07:01:26.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140563+0000", 2024-09-13T07:01:26.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6569000000000002e-05 2024-09-13T07:01:26.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140580+0000", 2024-09-13T07:01:26.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7725000000000001e-05 2024-09-13T07:01:26.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140942+0000", 2024-09-13T07:01:26.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036136300000000002 2024-09-13T07:01:26.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.141130+0000", 2024-09-13T07:01:26.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018814999999999999 2024-09-13T07:01:26.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:26.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.141433+0000", 2024-09-13T07:01:26.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030329899999999998 2024-09-13T07:01:26.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.147925+0000", 2024-09-13T07:01:26.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064915240000000003 2024-09-13T07:01:26.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152757+0000", 2024-09-13T07:01:26.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048328709999999999 2024-09-13T07:01:26.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152802+0000", 2024-09-13T07:01:26.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4314000000000003e-05 2024-09-13T07:01:26.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152853+0000", 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1149000000000001e-05 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1558 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.31115810999999999, 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012353627000000001, 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:26.518 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1558 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140610+0000", 2024-09-13T07:01:26.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:26.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140619+0000", 2024-09-13T07:01:26.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8650000000000007e-06 2024-09-13T07:01:26.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140634+0000", 2024-09-13T07:01:26.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4515000000000001e-05 2024-09-13T07:01:26.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140646+0000", 2024-09-13T07:01:26.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1736000000000001e-05 2024-09-13T07:01:26.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.144258+0000", 2024-09-13T07:01:26.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036123499999999998 2024-09-13T07:01:26.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.145034+0000", 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00077638900000000005 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.145395+0000", 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036117499999999998 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.149775+0000", 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043795639999999999 2024-09-13T07:01:26.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152906+0000", 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031306929999999999 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152941+0000", 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4919000000000002e-05 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152965+0000", 2024-09-13T07:01:26.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4641e-05 2024-09-13T07:01:26.524 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.524 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.524 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.524 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1601 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,zero 6291456~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.524 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.212491+0000", 2024-09-13T07:01:26.524 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.239278298, 2024-09-13T07:01:26.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038625460000000001, 2024-09-13T07:01:26.524 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.524 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.525 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.525 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1601 2024-09-13T07:01:26.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.525 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212491+0000", 2024-09-13T07:01:26.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212491+0000", 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212490+0000", 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212499+0000", 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212520+0000", 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1948000000000001e-05 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212530+0000", 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9939999999999995e-06 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212609+0000", 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8082000000000002e-05 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212700+0000", 2024-09-13T07:01:26.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1818999999999996e-05 2024-09-13T07:01:26.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.216354+0000", 2024-09-13T07:01:26.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003653627 2024-09-13T07:01:26.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.528 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.529 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1602 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 6291456~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.529 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.212548+0000", 2024-09-13T07:01:26.529 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.23922146699999999, 2024-09-13T07:01:26.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048322590000000002, 2024-09-13T07:01:26.529 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.529 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.529 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1602 2024-09-13T07:01:26.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.529 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212548+0000", 2024-09-13T07:01:26.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212548+0000", 2024-09-13T07:01:26.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212548+0000", 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212551+0000", 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.366e-06 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212558+0000", 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212563+0000", 2024-09-13T07:01:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6760000000000004e-06 2024-09-13T07:01:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.213190+0000", 2024-09-13T07:01:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00062709300000000001 2024-09-13T07:01:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.213268+0000", 2024-09-13T07:01:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7517999999999999e-05 2024-09-13T07:01:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.217381+0000", 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041125780000000004 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1648 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,zero 10485760~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.297923+0000", 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.15384662800000001, 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053690209999999999, 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.533 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.534 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1648 2024-09-13T07:01:26.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.534 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297923+0000", 2024-09-13T07:01:26.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297923+0000", 2024-09-13T07:01:26.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297920+0000", 2024-09-13T07:01:26.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:01:26.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297941+0000", 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1628999999999999e-05 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297986+0000", 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4135999999999999e-05 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298009+0000", 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3206000000000001e-05 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298236+0000", 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000227471 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298526+0000", 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028929800000000001 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.303292+0000", 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047665570000000003 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1649 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 10485760~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.298045+0000", 2024-09-13T07:01:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.15372519000000001, 2024-09-13T07:01:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055560130000000003, 2024-09-13T07:01:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1649 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298045+0000", 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298045+0000", 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298043+0000", 2024-09-13T07:01:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298052+0000", 2024-09-13T07:01:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4829999999999999e-06 2024-09-13T07:01:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298069+0000", 2024-09-13T07:01:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7073000000000001e-05 2024-09-13T07:01:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298083+0000", 2024-09-13T07:01:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3977e-05 2024-09-13T07:01:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.300163+0000", 2024-09-13T07:01:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020800229999999999 2024-09-13T07:01:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.300370+0000", 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000207679 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.303601+0000", 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032301550000000002 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1661 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [read 551881~5223] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.317063+0000", 2024-09-13T07:01:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.13470627399999999, 2024-09-13T07:01:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066193119999999996, 2024-09-13T07:01:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1661 2024-09-13T07:01:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317063+0000", 2024-09-13T07:01:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317063+0000", 2024-09-13T07:01:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317062+0000", 2024-09-13T07:01:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317070+0000", 2024-09-13T07:01:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9370000000000004e-06 2024-09-13T07:01:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317086+0000", 2024-09-13T07:01:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5685e-05 2024-09-13T07:01:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317093+0000", 2024-09-13T07:01:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6190000000000002e-06 2024-09-13T07:01:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317144+0000", 2024-09-13T07:01:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0587999999999997e-05 2024-09-13T07:01:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317148+0000", 2024-09-13T07:01:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0949999999999998e-06 2024-09-13T07:01:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.323168+0000", 2024-09-13T07:01:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060199110000000002 2024-09-13T07:01:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.323288+0000", 2024-09-13T07:01:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120375 2024-09-13T07:01:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.323683+0000", 2024-09-13T07:01:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00039459599999999999 2024-09-13T07:01:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1675 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,zero 4194304~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.340801+0000", 2024-09-13T07:01:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.110968434, 2024-09-13T07:01:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004445867, 2024-09-13T07:01:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1675 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340801+0000", 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340801+0000", 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340799+0000", 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340812+0000", 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3395e-05 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340845+0000", 2024-09-13T07:01:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2886000000000003e-05 2024-09-13T07:01:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340862+0000", 2024-09-13T07:01:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6633000000000002e-05 2024-09-13T07:01:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340946+0000", 2024-09-13T07:01:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3961999999999993e-05 2024-09-13T07:01:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341064+0000", 2024-09-13T07:01:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118948 2024-09-13T07:01:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.345247+0000", 2024-09-13T07:01:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041826900000000002 2024-09-13T07:01:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1676 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.110879906, 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067554429999999999, 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1676 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340889+0000", 2024-09-13T07:01:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340896+0000", 2024-09-13T07:01:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1420000000000004e-06 2024-09-13T07:01:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340909+0000", 2024-09-13T07:01:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3329000000000001e-05 2024-09-13T07:01:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340920+0000", 2024-09-13T07:01:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1029999999999999e-05 2024-09-13T07:01:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341653+0000", 2024-09-13T07:01:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073243400000000001 2024-09-13T07:01:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341825+0000", 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017243099999999999 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.347645+0000", 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058200639999999998 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1737 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,zero 10485760~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.430451+0000", 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.021318449999999999, 2024-09-13T07:01:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031563820000000001, 2024-09-13T07:01:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1737 2024-09-13T07:01:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430451+0000", 2024-09-13T07:01:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430451+0000", 2024-09-13T07:01:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430449+0000", 2024-09-13T07:01:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430461+0000", 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2299e-05 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430495+0000", 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3593000000000002e-05 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430514+0000", 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8986000000000001e-05 2024-09-13T07:01:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430681+0000", 2024-09-13T07:01:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016667799999999999 2024-09-13T07:01:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430877+0000", 2024-09-13T07:01:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019687599999999999 2024-09-13T07:01:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.433608+0000", 2024-09-13T07:01:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027302250000000002 2024-09-13T07:01:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1738 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 10485760~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.430540+0000", 2024-09-13T07:01:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.021229373999999999, 2024-09-13T07:01:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039689249999999999, 2024-09-13T07:01:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1738 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430540+0000", 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430540+0000", 2024-09-13T07:01:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430539+0000", 2024-09-13T07:01:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430546+0000", 2024-09-13T07:01:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2729999999999999e-06 2024-09-13T07:01:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430577+0000", 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0822999999999999e-05 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.430590+0000", 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3351e-05 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.431992+0000", 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001402555 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.432196+0000", 2024-09-13T07:01:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020347799999999999 2024-09-13T07:01:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.434509+0000", 2024-09-13T07:01:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023133310000000001 2024-09-13T07:01:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:26.567 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:26.569 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:01:26.570 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:26.565+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:26.571 INFO:teuthology.orchestra.run.smithi118.stdout:211 read 0xfc82b thru 0xfedb6 (0x258c bytes) 2024-09-13T07:01:26.571 INFO:teuthology.orchestra.run.smithi118.stdout:212 punch from 0x1efcc0d to 0x1f02e7e, (0x6271 bytes) 2024-09-13T07:01:26.571 INFO:teuthology.orchestra.run.smithi118.stdout:214 write 0x7fcf706 thru 0x7fd0385 (0xc80 bytes) 2024-09-13T07:01:26.571 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:26.571 INFO:teuthology.orchestra.run.smithi118.stdout:217 read 0x70341a thru 0x70f42f (0xc016 bytes) 2024-09-13T07:01:26.571 INFO:teuthology.orchestra.run.smithi118.stdout:218 read 0x762bcb2 thru 0x7639873 (0xdbc2 bytes) 2024-09-13T07:01:26.571 INFO:teuthology.orchestra.run.smithi118.stdout:219 write 0xaa1157f thru 0xaa15734 (0x41b6 bytes) 2024-09-13T07:01:26.571 INFO:teuthology.orchestra.run.smithi118.stdout:220 punch from 0x83f1d5a to 0x83f5613, (0x38b9 bytes) 2024-09-13T07:01:26.571 INFO:teuthology.orchestra.run.smithi118.stdout:221 writesame 0x2a91ca4 thru 0x2a956c4 data_size 0x1(0x3a21 bytes) 2024-09-13T07:01:26.571 INFO:teuthology.orchestra.run.smithi118.stdout:222 read 0x119a2b0 thru 0x11a32aa (0x8ffb bytes) 2024-09-13T07:01:26.571 INFO:teuthology.orchestra.run.smithi118.stdout:224 trunc from 0xaa15735 to 0x2b9b6c5 2024-09-13T07:01:26.572 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:26.572 INFO:teuthology.orchestra.run.smithi118.stdout:227 read 0x1331c6c thru 0x134126b (0xf600 bytes) 2024-09-13T07:01:26.572 INFO:teuthology.orchestra.run.smithi118.stdout:228 punch from 0x6ccdb to 0x79810, (0xcb35 bytes) 2024-09-13T07:01:26.572 INFO:teuthology.orchestra.run.smithi118.stdout:230 write 0x297f1da thru 0x29838a8 (0x46cf bytes) 2024-09-13T07:01:26.572 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:26.572 INFO:teuthology.orchestra.run.smithi118.stdout:233 write 0x5261a98 thru 0x5262abf (0x1028 bytes) 2024-09-13T07:01:26.573 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:26.573 INFO:teuthology.orchestra.run.smithi118.stdout:236 write 0x3c4389d thru 0x3c46d5f (0x34c3 bytes) 2024-09-13T07:01:26.575 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:01:26.576 INFO:teuthology.orchestra.run.smithi118.stdout:238 punch from 0x350c76e to 0x350e39b, (0x1c2d bytes) 2024-09-13T07:01:26.576 INFO:teuthology.orchestra.run.smithi118.stdout:239 read 0x8163b8 thru 0x81d1e7 (0x6e30 bytes) 2024-09-13T07:01:26.588 INFO:teuthology.orchestra.run.smithi118.stdout:240 writesame 0x6a0c7da thru 0x6a181a1 data_size 0x29(0xb9c8 bytes) 2024-09-13T07:01:26.591 INFO:teuthology.orchestra.run.smithi118.stdout:242 write 0x265e5c4 thru 0x265e88f (0x2cc bytes) 2024-09-13T07:01:26.598 INFO:teuthology.orchestra.run.smithi118.stdout:243 trunc from 0x6a181a2 to 0x38ad424 2024-09-13T07:01:26.614 INFO:teuthology.orchestra.run.smithi118.stdout:244 writesame 0x320d101 thru 0x321627e data_size 0xb(0x917e bytes) 2024-09-13T07:01:26.619 INFO:teuthology.orchestra.run.smithi118.stdout:245 writesame 0x29393a8 thru 0x293bbe1 data_size 0x13(0x283a bytes) 2024-09-13T07:01:26.623 INFO:teuthology.orchestra.run.smithi118.stdout:246 punch from 0x950ab0 to 0x957e83, (0x73d3 bytes) 2024-09-13T07:01:26.629 INFO:teuthology.orchestra.run.smithi118.stdout:247 trunc from 0x38ad424 to 0x416090 2024-09-13T07:01:26.669 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:26.664+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:26.672 INFO:teuthology.orchestra.run.smithi118.stdout:250 write 0x279e1ad thru 0x27a63c6 (0x821a bytes) 2024-09-13T07:01:26.692 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:26.692 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:26.694 INFO:teuthology.orchestra.run.smithi118.stdout:252 write 0x825a838 thru 0x8267c53 (0xd41c bytes) 2024-09-13T07:01:26.696 INFO:teuthology.orchestra.run.smithi118.stdout:253 clone 3 order 24 su 524288 sc 7 2024-09-13T07:01:26.702 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:01:26.715 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:26.733 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:26.753 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:26.771 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:26.768+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:26.774 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:26.796 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:26.805 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:26.805 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:26.815 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:01:26.819 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:26.840 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:26.862 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:26.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:26.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:26.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:26.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:26.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:26.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:26.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:26.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:26.871 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:26.868+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:26.920 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:26.920 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:26.930 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:01:26.973 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:26.968+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:27.045 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:27.045 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:27.055 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:01:27.074 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:27.069+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:27.161 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:27.161 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:27.171 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:01:27.175 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:27.172+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:27.277 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:27.273+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:27.278 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:27.278 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:27.288 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:01:27.333 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:01:27.333 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T07:01:27.347 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T07:01:27.378 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:27.373+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:27.438 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:27.438 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:27.448 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:01:27.454 INFO:teuthology.orchestra.run.smithi118.stdout:flattening image image_client.0-clone2 2024-09-13T07:01:27.478 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:27.476+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:27.566 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:27.567 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:27.577 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:01:27.598 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-13T07:01:27.598 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:27.592+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:27.616 INFO:teuthology.orchestra.run.smithi118.stdout:254 trunc from 0x8267c54 to 0x1de1000 2024-09-13T07:01:27.665 INFO:teuthology.orchestra.run.smithi118.stdout:257 trunc from 0x1de1000 to 0xa898593 2024-09-13T07:01:27.674 INFO:teuthology.orchestra.run.smithi118.stdout:259 punch from 0x67ece3a to 0x67f738c, (0xa552 bytes) 2024-09-13T07:01:27.674 INFO:teuthology.orchestra.run.smithi118.stdout:262 write 0x57175c3 thru 0x572702e (0xfa6c bytes) 2024-09-13T07:01:27.678 INFO:teuthology.orchestra.run.smithi118.stdout:263 read 0x17e87c2 thru 0x17f3888 (0xb0c7 bytes) 2024-09-13T07:01:27.680 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:27.677+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:27.696 INFO:teuthology.orchestra.run.smithi118.stdout:264 writesame 0xe448343 thru 0xe44bb4e data_size 0x22(0x380c bytes) 2024-09-13T07:01:27.696 INFO:teuthology.orchestra.run.smithi118.stdout:265 read 0x800f74e thru 0x801161b (0x1ece bytes) 2024-09-13T07:01:27.697 INFO:teuthology.orchestra.run.smithi118.stdout:266 punch from 0xb0d0cdf to 0xb0d62a6, (0x55c7 bytes) 2024-09-13T07:01:27.697 INFO:teuthology.orchestra.run.smithi118.stdout:270 read 0xe383cec thru 0xe38b21d (0x7532 bytes) 2024-09-13T07:01:27.697 INFO:teuthology.orchestra.run.smithi118.stdout:272 write 0xf8d96b thru 0xf8f547 (0x1bdd bytes) 2024-09-13T07:01:27.699 INFO:teuthology.orchestra.run.smithi118.stdout:273 write 0x463c5ad thru 0x4647164 (0xabb8 bytes) 2024-09-13T07:01:27.701 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:27.702 INFO:teuthology.orchestra.run.smithi118.stdout:275 write 0x6ab16c4 thru 0x6ab7377 (0x5cb4 bytes) 2024-09-13T07:01:27.704 INFO:teuthology.orchestra.run.smithi118.stdout:276 write 0x4fc807b thru 0x4fd1aa4 (0x9a2a bytes) 2024-09-13T07:01:27.708 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:27.708 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:27.716 INFO:teuthology.orchestra.run.smithi118.stdout:280 trunc from 0xe44bb4f to 0x925f3f4 2024-09-13T07:01:27.721 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:01:27.727 INFO:teuthology.orchestra.run.smithi118.stdout:281 read 0x64146ae thru 0x641edd3 (0xa726 bytes) 2024-09-13T07:01:27.731 INFO:teuthology.orchestra.run.smithi118.stdout:283 trunc from 0x925f3f4 to 0xb26bef9 2024-09-13T07:01:27.743 INFO:teuthology.orchestra.run.smithi118.stdout:285 writesame 0x6b98c63 thru 0x6ba683a data_size 0x23(0xdbd8 bytes) 2024-09-13T07:01:27.747 INFO:teuthology.orchestra.run.smithi118.stdout:286 write 0x55745f1 thru 0x5579f5b (0x596b bytes) 2024-09-13T07:01:27.750 INFO:teuthology.orchestra.run.smithi118.stdout:287 writesame 0x9f27f94 thru 0x9f28610 data_size 0x1(0x67d bytes) 2024-09-13T07:01:27.759 INFO:teuthology.orchestra.run.smithi118.stdout:289 read 0x265298c thru 0x265a650 (0x7cc5 bytes) 2024-09-13T07:01:27.759 INFO:teuthology.orchestra.run.smithi118.stdout:290 write 0x6e06c18 thru 0x6e14cdc (0xe0c5 bytes) 2024-09-13T07:01:27.763 INFO:teuthology.orchestra.run.smithi118.stdout:291 read 0xaa47b08 thru 0xaa4ccd4 (0x51cd bytes) 2024-09-13T07:01:27.763 INFO:teuthology.orchestra.run.smithi118.stdout:292 clone 4 order 24 su 262144 sc 11 2024-09-13T07:01:27.782 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:27.776+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:27.831 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:27.831 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:27.840 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:01:27.883 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:27.877+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:27.985 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:27.980+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:27.993 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:27.993 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:28.002 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:01:28.085 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:28.080+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:28.105 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:28.106 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:28.115 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:01:28.187 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:28.184+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:28.219 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:28.219 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:28.232 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:01:28.288 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:28.281+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:28.337 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T07:01:28.341 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:28.341 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:28.348 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T07:01:28.351 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:01:28.388 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:28.384+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:28.436 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone3 from 0xb26bef9 (overlap 0x1de1000) to 0x2fce66 2024-09-13T07:01:28.459 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:28.459 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:28.473 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:01:28.517 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:28.513+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:28.537 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-13T07:01:28.583 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:28.583 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:28.591 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:28.588+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:28.593 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:01:28.692 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:28.685+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:28.704 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:28.704 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:28.713 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:01:28.773 INFO:teuthology.orchestra.run.smithi118.stdout:294 write 0xafd677e thru 0xafe36b3 (0xcf36 bytes) 2024-09-13T07:01:28.790 INFO:teuthology.orchestra.run.smithi118.stdout:297 read 0x65d569 thru 0x66acf5 (0xd78d bytes) 2024-09-13T07:01:28.791 INFO:teuthology.orchestra.run.smithi118.stdout:298 writesame 0x34fbc04 thru 0x35051f0 data_size 0x1(0x95ed bytes) 2024-09-13T07:01:28.794 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:28.788+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:28.825 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:28.825 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:28.830 INFO:teuthology.orchestra.run.smithi118.stdout:299 trunc from 0xb26bef9 to 0x35345e0 2024-09-13T07:01:28.835 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:01:28.870 INFO:teuthology.orchestra.run.smithi118.stdout:300 punch from 0x1c8d226 to 0x1c97a9e, (0xa878 bytes) 2024-09-13T07:01:28.876 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:01:28.893 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:28.889+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:28.895 INFO:teuthology.orchestra.run.smithi118.stdout:301 write 0xb8f2d11 thru 0xb902065 (0xf355 bytes) 2024-09-13T07:01:28.903 INFO:teuthology.orchestra.run.smithi118.stdout:303 read 0xb73bfe thru 0xb7a1a0 (0x65a3 bytes) 2024-09-13T07:01:28.904 INFO:teuthology.orchestra.run.smithi118.stdout:304 writesame 0x82c657f thru 0x82cc15e data_size 0x18(0x5be0 bytes) 2024-09-13T07:01:28.910 INFO:teuthology.orchestra.run.smithi118.stdout:305 trunc from 0xb902066 to 0xa300d0b 2024-09-13T07:01:28.932 INFO:teuthology.orchestra.run.smithi118.stdout:306 read 0x4dd4b8f thru 0x4dd7721 (0x2b93 bytes) 2024-09-13T07:01:28.933 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:28.933 INFO:teuthology.orchestra.run.smithi118.stdout:309 read 0x5192a6 thru 0x525be4 (0xc93f bytes) 2024-09-13T07:01:28.933 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:28.933 INFO:teuthology.orchestra.run.smithi118.stdout:311 write 0x9deb050 thru 0x9dfacf9 (0xfcaa bytes) 2024-09-13T07:01:28.941 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:28.941 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:28.951 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:01:28.953 INFO:teuthology.orchestra.run.smithi118.stdout:312 write 0xd619f14 thru 0xd61f2a3 (0x5390 bytes) 2024-09-13T07:01:28.981 INFO:teuthology.orchestra.run.smithi118.stdout:313 trunc from 0xd61f2a4 to 0xd28613 2024-09-13T07:01:28.994 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:28.988+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:29.024 INFO:teuthology.orchestra.run.smithi118.stdout:314 trunc from 0xd28613 to 0x88550ce 2024-09-13T07:01:29.034 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:29.055 INFO:teuthology.orchestra.run.smithi118.stdout:315 writesame 0xd80f2e5 thru 0xd81eeac data_size 0x1c(0xfbc8 bytes) 2024-09-13T07:01:29.055 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:29.055 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:29.055 INFO:teuthology.orchestra.run.smithi118.stdout:317 write 0x22c4f7c thru 0x22d01f0 (0xb275 bytes) 2024-09-13T07:01:29.059 INFO:teuthology.orchestra.run.smithi118.stdout:320 punch from 0x9b1a812 to 0x9b1e0cf, (0x38bd bytes) 2024-09-13T07:01:29.059 INFO:teuthology.orchestra.run.smithi118.stdout:321 writesame 0x672251d thru 0x6730626 data_size 0x23(0xe10a bytes) 2024-09-13T07:01:29.064 INFO:teuthology.orchestra.run.smithi118.stdout:323 writesame 0x89f0c00 thru 0x89ff514 data_size 0x1(0xe915 bytes) 2024-09-13T07:01:29.073 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T07:01:29.076 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:29.087 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T07:01:29.095 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:29.089+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:29.098 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:29.106 INFO:teuthology.orchestra.run.smithi118.stdout:324 punch from 0xa62a7ac to 0xa62c274, (0x1ac8 bytes) 2024-09-13T07:01:29.106 INFO:teuthology.orchestra.run.smithi118.stdout:325 read 0x3ea6fb thru 0x3f9804 (0xf10a bytes) 2024-09-13T07:01:29.117 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:29.118 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:29.119 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:29.119 INFO:teuthology.orchestra.run.smithi118.stdout:328 trunc from 0xd81eead to 0x9715281 2024-09-13T07:01:29.129 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:01:29.136 INFO:teuthology.orchestra.run.smithi118.stdout:329 read 0x658a73e thru 0x6594323 (0x9be6 bytes) 2024-09-13T07:01:29.137 INFO:teuthology.orchestra.run.smithi118.stdout:330 read 0x17d5158 thru 0x17dc0a7 (0x6f50 bytes) 2024-09-13T07:01:29.137 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:29.141 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:29.143 INFO:teuthology.orchestra.run.smithi118.stdout:333 trunc from 0x9715281 to 0xd130466 2024-09-13T07:01:29.151 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:29.151 INFO:teuthology.orchestra.run.smithi118.stdout:335 write 0xe3d134 thru 0xe4aa22 (0xd8ef bytes) 2024-09-13T07:01:29.154 INFO:teuthology.orchestra.run.smithi118.stdout:336 read 0x720e27b thru 0x721a8a8 (0xc62e bytes) 2024-09-13T07:01:29.155 INFO:teuthology.orchestra.run.smithi118.stdout:337 writesame 0x35e1657 thru 0x35e5673 data_size 0x1(0x401d bytes) 2024-09-13T07:01:29.161 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:29.169 INFO:teuthology.orchestra.run.smithi118.stdout:339 write 0xa700de6 thru 0xa709b4a (0x8d65 bytes) 2024-09-13T07:01:29.172 INFO:teuthology.orchestra.run.smithi118.stdout:340 read 0x516b708 thru 0x516d3f2 (0x1ceb bytes) 2024-09-13T07:01:29.172 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:29.172 INFO:teuthology.orchestra.run.smithi118.stdout:346 write 0x7851c09 thru 0x785b0d6 (0x94ce bytes) 2024-09-13T07:01:29.188 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:29.190 INFO:teuthology.orchestra.run.smithi118.stdout:347 trunc from 0xd130466 to 0x410f047 2024-09-13T07:01:29.190 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:29.191 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:29.191 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:29.191 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:29.191 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:29.191 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:29.191 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:29.191 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:29.195 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:29.189+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:29.203 INFO:teuthology.orchestra.run.smithi118.stdout:349 write 0x37f171f thru 0x37f8237 (0x6b19 bytes) 2024-09-13T07:01:29.206 INFO:teuthology.orchestra.run.smithi118.stdout:350 write 0x2b7e867 thru 0x2b850e5 (0x687f bytes) 2024-09-13T07:01:29.228 INFO:teuthology.orchestra.run.smithi118.stdout:351 write 0xb40a83f thru 0xb413b99 (0x935b bytes) 2024-09-13T07:01:29.234 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4226.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:11.223460+0000", 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.007031224999999, 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064502090000000002, 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4226", 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/1083678545", 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7 2024-09-13T07:01:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223460+0000", 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223460+0000", 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223458+0000", 2024-09-13T07:01:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223471+0000", 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2933e-05 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223501+0000", 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0647e-05 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223534+0000", 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2478999999999999e-05 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223675+0000", 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014109999999999999 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.223697+0000", 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1577e-05 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.227045+0000", 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033489090000000002 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.227172+0000", 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012647 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.227430+0000", 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025785499999999998 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.229137+0000", 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017073780000000001 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.229729+0000", 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00059170999999999996 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.229841+0000", 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112571 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:11.229910+0000", 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8965000000000001e-05 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:439 3.16 3:6cdf2866:::rbd_data.1194d22ad098.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.539100+0000", 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.6913914009999997, 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01241133, 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 439 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539100+0000", 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539100+0000", 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539098+0000", 2024-09-13T07:01:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539113+0000", 2024-09-13T07:01:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5653000000000001e-05 2024-09-13T07:01:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539148+0000", 2024-09-13T07:01:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4489000000000001e-05 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539162+0000", 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4533e-05 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539297+0000", 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013447399999999999 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539701+0000", 2024-09-13T07:01:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00040402400000000001 2024-09-13T07:01:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.542876+0000", 2024-09-13T07:01:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031755820000000001 2024-09-13T07:01:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551351+0000", 2024-09-13T07:01:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084747239999999995 2024-09-13T07:01:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551392+0000", 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0921999999999999e-05 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551457+0000", 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4925999999999995e-05 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551511+0000", 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4446e-05 2024-09-13T07:01:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:440 3.1c 3:3b975cee:::rbd_data.1194d22ad098.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.539301+0000", 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.6911908420000001, 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019168998999999999, 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 440 2024-09-13T07:01:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539301+0000", 2024-09-13T07:01:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539301+0000", 2024-09-13T07:01:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539300+0000", 2024-09-13T07:01:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539308+0000", 2024-09-13T07:01:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-06 2024-09-13T07:01:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539347+0000", 2024-09-13T07:01:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9063999999999997e-05 2024-09-13T07:01:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539353+0000", 2024-09-13T07:01:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3459999999999999e-06 2024-09-13T07:01:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539438+0000", 2024-09-13T07:01:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4757000000000002e-05 2024-09-13T07:01:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539706+0000", 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026819000000000002 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.543844+0000", 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004137338 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558385+0000", 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014541304999999999 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558397+0000", 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2269999999999999e-05 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558437+0000", 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0096999999999999e-05 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558470+0000", 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2242000000000001e-05 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:491 3.d e26/22 3:b4666c7f:::rbd_data.1194d22ad098.000000000000008e:head v 26'11, mlcod=26'9)", 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.626884+0000", 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.6036069160000004, 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065664440000000003, 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.626884+0000", 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.626884+0000", 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.626883+0000", 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.626990+0000", 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010761500000000001 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.627034+0000", 2024-09-13T07:01:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3782000000000002e-05 2024-09-13T07:01:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.627052+0000", 2024-09-13T07:01:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8408000000000001e-05 2024-09-13T07:01:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.627171+0000", 2024-09-13T07:01:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118946 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.627203+0000", 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1136000000000001e-05 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.633358+0000", 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061555459999999996 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.633451+0000", 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2807999999999996e-05 2024-09-13T07:01:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:492 3.d e26/22 3:b3db72cd:::rbd_data.1194d22ad098.000000000000008f:head v 26'12, mlcod=26'9)", 2024-09-13T07:01:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.628584+0000", 2024-09-13T07:01:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.6019072669999996, 2024-09-13T07:01:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054548449999999998, 2024-09-13T07:01:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628584+0000", 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628584+0000", 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628582+0000", 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628617+0000", 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4672999999999998e-05 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628669+0000", 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2404999999999997e-05 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628695+0000", 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5863e-05 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628808+0000", 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011226399999999999 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.628861+0000", 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3436000000000001e-05 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.633985+0000", 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051237619999999996 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.634039+0000", 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4172000000000002e-05 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:940 3.1c 3:3c4f52df:::rbd_data.1194d22ad098.0000000000000031:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.492246+0000", 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7382449630000001, 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054344950000000001, 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 940 2024-09-13T07:01:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492246+0000", 2024-09-13T07:01:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492246+0000", 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492246+0000", 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492252+0000", 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:01:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492264+0000", 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1742e-05 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492272+0000", 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8059999999999995e-06 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492329+0000", 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7017e-05 2024-09-13T07:01:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492419+0000", 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0506000000000004e-05 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497681+0000", 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005261712 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:945 3.1c 3:3b975cee:::rbd_data.1194d22ad098.0000000000000035:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.494952+0000", 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7355390509999999, 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058335799999999997, 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 945 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494952+0000", 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494952+0000", 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494951+0000", 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494958+0000", 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7870000000000004e-06 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494972+0000", 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.437e-05 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494981+0000", 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6070000000000008e-06 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495034+0000", 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3504999999999997e-05 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495132+0000", 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7428000000000003e-05 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495319+0000", 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018667099999999999 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500372+0000", 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050530929999999998 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500681+0000", 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030899299999999997 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500740+0000", 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9713000000000002e-05 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500786+0000", 2024-09-13T07:01:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5566999999999999e-05 2024-09-13T07:01:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:969 3.d e27/22 3:b06a3fa5:::rbd_data.1194d22ad098.0000000000000041:head v 27'18, mlcod=27'16)", 2024-09-13T07:01:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.504201+0000", 2024-09-13T07:01:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7262903970000001, 2024-09-13T07:01:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077030220000000003, 2024-09-13T07:01:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504201+0000", 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504201+0000", 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504200+0000", 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504209+0000", 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4800000000000001e-06 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504223+0000", 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4207e-05 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504230+0000", 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3819999999999998e-06 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504270+0000", 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9545999999999997e-05 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504290+0000", 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0636999999999999e-05 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.511866+0000", 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0075757680000000001 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.511904+0000", 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7821000000000003e-05 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:978 3.1d 3:bb753f8d:::rbd_data.1194d22ad098.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.504609+0000", 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7258821389999999, 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077548790000000001, 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 978 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504609+0000", 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504609+0000", 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504608+0000", 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504614+0000", 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6799999999999998e-06 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504626+0000", 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1858999999999999e-05 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504632+0000", 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0460000000000002e-06 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504685+0000", 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3542000000000003e-05 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.504781+0000", 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5704000000000004e-05 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.505000+0000", 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021894699999999999 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.511911+0000", 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069104489999999999 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.512277+0000", 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000366275 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.512314+0000", 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7571999999999999e-05 2024-09-13T07:01:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.512364+0000", 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9727e-05 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1008 3.16 3:6cc0310a:::rbd_data.1194d22ad098.0000000000000051:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.522237+0000", 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7082541500000001, 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050442450000000002, 2024-09-13T07:01:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1008 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522237+0000", 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522237+0000", 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522236+0000", 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522247+0000", 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1511999999999999e-05 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522264+0000", 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6796999999999999e-05 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522272+0000", 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3920000000000003e-06 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522343+0000", 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0909999999999997e-05 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522423+0000", 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9233000000000005e-05 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.522600+0000", 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017731299999999999 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.526882+0000", 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042823760000000001 2024-09-13T07:01:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527160+0000", 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027766800000000001 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527231+0000", 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.161e-05 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527282+0000", 2024-09-13T07:01:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.003e-05 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1015 3.17 e27/22 3:ecdfb694:::rbd_data.1194d22ad098.0000000000000056:head v 27'13, mlcod=27'11)", 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.525303+0000", 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7051883640000001, 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052399150000000004, 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525303+0000", 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525303+0000", 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525302+0000", 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525312+0000", 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7419999999999992e-06 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525331+0000", 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9176000000000001e-05 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525339+0000", 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5399999999999996e-06 2024-09-13T07:01:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525394+0000", 2024-09-13T07:01:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4941999999999997e-05 2024-09-13T07:01:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.525412+0000", 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7887999999999999e-05 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.530495+0000", 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050831640000000003 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.530543+0000", 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7518999999999997e-05 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1020 3.5 3:a20826fb:::rbd_data.1194d22ad098.000000000000005a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.527436+0000", 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.703055575, 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052420119999999999, 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1020 2024-09-13T07:01:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527436+0000", 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527436+0000", 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527435+0000", 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527441+0000", 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7549999999999997e-06 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527457+0000", 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5957999999999999e-05 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527471+0000", 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3773e-05 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527540+0000", 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9375000000000006e-05 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527677+0000", 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000136995 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.527951+0000", 2024-09-13T07:01:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027340300000000002 2024-09-13T07:01:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.529334+0000", 2024-09-13T07:01:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013827399999999999 2024-09-13T07:01:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.532580+0000", 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003246302 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.532628+0000", 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8622000000000003e-05 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.532678+0000", 2024-09-13T07:01:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359e-05 2024-09-13T07:01:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1023 3.17 e27/22 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head v 27'15, mlcod=27'11)", 2024-09-13T07:01:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.528842+0000", 2024-09-13T07:01:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7016492369999998, 2024-09-13T07:01:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092562140000000005, 2024-09-13T07:01:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528842+0000", 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528842+0000", 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528841+0000", 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528853+0000", 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1518000000000001e-05 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528874+0000", 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1455e-05 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528886+0000", 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1404e-05 2024-09-13T07:01:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528953+0000", 2024-09-13T07:01:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7644999999999994e-05 2024-09-13T07:01:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.528975+0000", 2024-09-13T07:01:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1492999999999999e-05 2024-09-13T07:01:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.538036+0000", 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090608589999999992 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.538098+0000", 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2830000000000007e-05 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1048 3.d e27/22 3:b3db72cd:::rbd_data.1194d22ad098.000000000000008f:head v 27'24, mlcod=27'19)", 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.544658+0000", 2024-09-13T07:01:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.6858337859999999, 2024-09-13T07:01:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057056909999999997, 2024-09-13T07:01:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544658+0000", 2024-09-13T07:01:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544658+0000", 2024-09-13T07:01:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544656+0000", 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544667+0000", 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0210000000000001e-05 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544686+0000", 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9763e-05 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544699+0000", 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2836e-05 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544778+0000", 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8611999999999999e-05 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.544808+0000", 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0484000000000002e-05 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.550319+0000", 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055108090000000002 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.550363+0000", 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4125999999999997e-05 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1531 3.c 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.180979722, 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063168486999999995, 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1531 2024-09-13T07:01:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049511+0000", 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069234+0000", 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019723920999999998 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069269+0000", 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4446999999999997e-05 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069286+0000", 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7405999999999999e-05 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069370+0000", 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-05 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069511+0000", 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014134299999999999 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070463+0000", 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095167899999999998 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105076+0000", 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034613139000000001 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112562+0000", 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074861429999999998 2024-09-13T07:01:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112631+0000", 2024-09-13T07:01:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:01:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112680+0000", 2024-09-13T07:01:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9607000000000001e-05 2024-09-13T07:01:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1536 3.17 e27/22 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head v 27'16, mlcod=27'13)", 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.097407+0000", 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.1330845250000001, 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081808699999999998, 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.097407+0000", 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.097407+0000", 2024-09-13T07:01:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.097405+0000", 2024-09-13T07:01:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.097419+0000", 2024-09-13T07:01:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3893000000000001e-05 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.097439+0000", 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9984999999999999e-05 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.097451+0000", 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1636e-05 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.097529+0000", 2024-09-13T07:01:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7948999999999995e-05 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.097556+0000", 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6849999999999999e-05 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105533+0000", 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079769049999999994 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105588+0000", 2024-09-13T07:01:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5072999999999999e-05 2024-09-13T07:01:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1551 3.c 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head [stat,truncate 5500120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.140375+0000", 2024-09-13T07:01:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.0901165349999999, 2024-09-13T07:01:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069047249999999996, 2024-09-13T07:01:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1551 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140375+0000", 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140375+0000", 2024-09-13T07:01:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140372+0000", 2024-09-13T07:01:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140389+0000", 2024-09-13T07:01:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6773999999999999e-05 2024-09-13T07:01:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140418+0000", 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9459999999999999e-05 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140439+0000", 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1302e-05 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140577+0000", 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138054 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140833+0000", 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025542000000000002 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.141182+0000", 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00034952399999999998 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.145311+0000", 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041285130000000003 2024-09-13T07:01:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.147155+0000", 2024-09-13T07:01:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018440450000000001 2024-09-13T07:01:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.147234+0000", 2024-09-13T07:01:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9352999999999997e-05 2024-09-13T07:01:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.147280+0000", 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5259999999999997e-05 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1554 3.16 3:6a67e608:::rbd_data.11943a018f34.0000000000000008:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.142368+0000", 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.0881229719999999, 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061689750000000002, 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1554 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.142368+0000", 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.142368+0000", 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.142368+0000", 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.142374+0000", 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4280000000000001e-06 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.142386+0000", 2024-09-13T07:01:29.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1919999999999999e-05 2024-09-13T07:01:29.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.142395+0000", 2024-09-13T07:01:29.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6549999999999993e-06 2024-09-13T07:01:29.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.142451+0000", 2024-09-13T07:01:29.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5989000000000002e-05 2024-09-13T07:01:29.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.142540+0000", 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9928999999999999e-05 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.142681+0000", 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014007900000000001 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.146583+0000", 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039026970000000001 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.148383+0000", 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017994339999999999 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.148466+0000", 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3341e-05 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.148537+0000", 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1423999999999999e-05 2024-09-13T07:01:29.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1794 3.16 3:6a67e608:::rbd_data.11943a018f34.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.510292+0000", 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.7201998170000001, 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015887680000000001, 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1794 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510292+0000", 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510292+0000", 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510289+0000", 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510303+0000", 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3933e-05 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510335+0000", 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2169000000000003e-05 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510349+0000", 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4205999999999999e-05 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510458+0000", 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010835100000000001 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510597+0000", 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013911699999999999 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.513457+0000", 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028606439999999999 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.526058+0000", 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012600849000000001 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.526090+0000", 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2036999999999998e-05 2024-09-13T07:01:29.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.526139+0000", 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9042999999999998e-05 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.526179+0000", 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0077999999999998e-05 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1851 3.16 3:6a67e608:::rbd_data.11943a018f34.0000000000000008:head [stat,zero 4194304~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.598071+0000", 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.6324202240000001, 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074482929999999999, 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1851 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598071+0000", 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598071+0000", 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598070+0000", 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598078+0000", 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5490000000000006e-06 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598133+0000", 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4877999999999999e-05 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598149+0000", 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5594e-05 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598280+0000", 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013090000000000001 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598510+0000", 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023055499999999999 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.605519+0000", 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070091229999999999 2024-09-13T07:01:29.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.342 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.342 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.342 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:29.344 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:01:29.344 INFO:teuthology.orchestra.run.smithi118.stdout:352 trunc from 0xb413b9a to 0xc877442 2024-09-13T07:01:29.344 INFO:teuthology.orchestra.run.smithi118.stdout:353 read 0x88591c3 thru 0x885e8b4 (0x56f2 bytes) 2024-09-13T07:01:29.344 INFO:teuthology.orchestra.run.smithi118.stdout:355 read 0x6b800e7 thru 0x6b8aa9a (0xa9b4 bytes) 2024-09-13T07:01:29.345 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:29.345 INFO:teuthology.orchestra.run.smithi118.stdout:357 punch from 0xa5f396f to 0xa5feb5e, (0xb1ef bytes) 2024-09-13T07:01:29.345 INFO:teuthology.orchestra.run.smithi118.stdout:358 write 0x1483a2c thru 0x1491a63 (0xe038 bytes) 2024-09-13T07:01:29.345 INFO:teuthology.orchestra.run.smithi118.stdout:360 write 0x25bbc27 thru 0x25c43a6 (0x8780 bytes) 2024-09-13T07:01:29.345 INFO:teuthology.orchestra.run.smithi118.stdout:361 writesame 0x282f122 thru 0x283abb8 data_size 0x1(0xba97 bytes) 2024-09-13T07:01:29.345 INFO:teuthology.orchestra.run.smithi118.stdout:362 trunc from 0xc877442 to 0x7d984 2024-09-13T07:01:29.345 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:29.340+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:29.346 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:01:29.350 INFO:teuthology.orchestra.run.smithi118.stdout:363 read 0x4b3cc thru 0x4f699 (0x42ce bytes) 2024-09-13T07:01:29.350 INFO:teuthology.orchestra.run.smithi118.stdout:364 punch from 0x56abf to 0x61dec, (0xb32d bytes) 2024-09-13T07:01:29.368 INFO:teuthology.orchestra.run.smithi118.stdout:366 trunc from 0x7d984 to 0xab85747 2024-09-13T07:01:29.381 INFO:teuthology.orchestra.run.smithi118.stdout:370 trunc from 0xab85747 to 0xc3afd1b 2024-09-13T07:01:29.392 INFO:teuthology.orchestra.run.smithi118.stdout:371 read 0x676de96 thru 0x676deb8 (0x23 bytes) 2024-09-13T07:01:29.392 INFO:teuthology.orchestra.run.smithi118.stdout:372 write 0x1f2a83e thru 0x1f349a8 (0xa16b bytes) 2024-09-13T07:01:29.400 INFO:teuthology.orchestra.run.smithi118.stdout:373 punch from 0x8efca89 to 0x8f01ef1, (0x5468 bytes) 2024-09-13T07:01:29.400 INFO:teuthology.orchestra.run.smithi118.stdout:374 write 0x9b904f3 thru 0x9b97955 (0x7463 bytes) 2024-09-13T07:01:29.406 INFO:teuthology.orchestra.run.smithi118.stdout:378 writesame 0xacd52f5 thru 0xacd8f1f data_size 0x1(0x3c2b bytes) 2024-09-13T07:01:29.422 INFO:teuthology.orchestra.run.smithi118.stdout:379 writesame 0x169c3e8 thru 0x169cd75 data_size 0x1(0x98e bytes) 2024-09-13T07:01:29.429 INFO:teuthology.orchestra.run.smithi118.stdout:380 read 0x70c1768 thru 0x70c363a (0x1ed3 bytes) 2024-09-13T07:01:29.442 INFO:teuthology.orchestra.run.smithi118.stdout:383 write 0xdbf4e97 thru 0xdc00c24 (0xbd8e bytes) 2024-09-13T07:01:29.443 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:29.437+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:29.450 INFO:teuthology.orchestra.run.smithi118.stdout:384 trunc from 0xdc00c25 to 0xe22aefb 2024-09-13T07:01:29.461 INFO:teuthology.orchestra.run.smithi118.stdout:385 write 0x67d4cc5 thru 0x67d5508 (0x844 bytes) 2024-09-13T07:01:29.465 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:29.465 INFO:teuthology.orchestra.run.smithi118.stdout:387 write 0x749f3be thru 0x74a8189 (0x8dcc bytes) 2024-09-13T07:01:29.470 INFO:teuthology.orchestra.run.smithi118.stdout:390 read 0xc2e024 thru 0xc31cba (0x3c97 bytes) 2024-09-13T07:01:29.470 INFO:teuthology.orchestra.run.smithi118.stdout:391 read 0x687c802 thru 0x688c207 (0xfa06 bytes) 2024-09-13T07:01:29.470 INFO:teuthology.orchestra.run.smithi118.stdout:392 read 0xbddf6c5 thru 0xbdebbb6 (0xc4f2 bytes) 2024-09-13T07:01:29.471 INFO:teuthology.orchestra.run.smithi118.stdout:393 read 0x1c0efd3 thru 0x1c14e32 (0x5e60 bytes) 2024-09-13T07:01:29.472 INFO:teuthology.orchestra.run.smithi118.stdout:394 read 0x60e280b thru 0x60e7679 (0x4e6f bytes) 2024-09-13T07:01:29.472 INFO:teuthology.orchestra.run.smithi118.stdout:395 write 0x945453b thru 0x946287e (0xe344 bytes) 2024-09-13T07:01:29.490 INFO:teuthology.orchestra.run.smithi118.stdout:396 trunc from 0xe22aefb to 0x74d7590 2024-09-13T07:01:29.502 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:29.503 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:29.510 INFO:teuthology.orchestra.run.smithi118.stdout:397 read 0x8d40f8 thru 0x8e0976 (0xc87f bytes) 2024-09-13T07:01:29.513 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:01:29.516 INFO:teuthology.orchestra.run.smithi118.stdout:398 trunc from 0x74d7590 to 0x40b1048 2024-09-13T07:01:29.544 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:29.540+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:29.556 INFO:teuthology.orchestra.run.smithi118.stdout:399 write 0xe3a7bcf thru 0xe3a8e59 (0x128b bytes) 2024-09-13T07:01:29.561 INFO:teuthology.orchestra.run.smithi118.stdout:400 read 0x8a775fc thru 0x8a7df57 (0x695c bytes) 2024-09-13T07:01:29.562 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:29.563 INFO:teuthology.orchestra.run.smithi118.stdout:402 trunc from 0xe3a8e5a to 0xde79ad6 2024-09-13T07:01:29.576 INFO:teuthology.orchestra.run.smithi118.stdout:403 read 0x325d824 thru 0x325e17d (0x95a bytes) 2024-09-13T07:01:29.576 INFO:teuthology.orchestra.run.smithi118.stdout:404 punch from 0x3d0a8ee to 0x3d188dd, (0xdfef bytes) 2024-09-13T07:01:29.577 INFO:teuthology.orchestra.run.smithi118.stdout:406 read 0x47067f2 thru 0x470e982 (0x8191 bytes) 2024-09-13T07:01:29.577 INFO:teuthology.orchestra.run.smithi118.stdout:408 clone 5 order 23 su 131072 sc 10 2024-09-13T07:01:29.616 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:29.617 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:29.617 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:29.617 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:29.617 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:29.626 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:01:29.645 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:29.641+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:29.737 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:581 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.416139+0000", 2024-09-13T07:01:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3169937569999997, 2024-09-13T07:01:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061636429999999999, 2024-09-13T07:01:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 581 2024-09-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416139+0000", 2024-09-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416139+0000", 2024-09-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416136+0000", 2024-09-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416171+0000", 2024-09-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4545000000000002e-05 2024-09-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416203+0000", 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2085000000000002e-05 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416224+0000", 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1157000000000001e-05 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416339+0000", 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115455 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416349+0000", 2024-09-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5729999999999997e-06 2024-09-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.419615+0000", 2024-09-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032657139999999999 2024-09-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.419766+0000", 2024-09-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015095000000000001 2024-09-13T07:01:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.420094+0000", 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032800599999999999 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.421267+0000", 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011733889999999999 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.422161+0000", 2024-09-13T07:01:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089368499999999997 2024-09-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.422233+0000", 2024-09-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2619000000000004e-05 2024-09-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.422303+0000", 2024-09-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9264999999999995e-05 2024-09-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:929 3.a 3:51375ed8:::rbd_data.1194d22ad098.0000000000000028:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.487437+0000", 2024-09-13T07:01:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2456961050000004, 2024-09-13T07:01:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046615179999999999, 2024-09-13T07:01:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 929 2024-09-13T07:01:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487437+0000", 2024-09-13T07:01:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487437+0000", 2024-09-13T07:01:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487435+0000", 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487443+0000", 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1440000000000003e-06 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487460+0000", 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6815000000000001e-05 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487469+0000", 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8880000000000003e-06 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487518+0000", 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9042999999999998e-05 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487607+0000", 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9319000000000003e-05 2024-09-13T07:01:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.487802+0000", 2024-09-13T07:01:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019425599999999999 2024-09-13T07:01:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.489153+0000", 2024-09-13T07:01:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001350895 2024-09-13T07:01:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.491933+0000", 2024-09-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027807130000000002 2024-09-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.491991+0000", 2024-09-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-05 2024-09-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492098+0000", 2024-09-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010663299999999999 2024-09-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:944 3.10 3:0ef08359:::rbd_data.1194d22ad098.0000000000000034:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.492669+0000", 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2404631860000004, 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049892310000000002, 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 944 2024-09-13T07:01:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492669+0000", 2024-09-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492669+0000", 2024-09-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492668+0000", 2024-09-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492675+0000", 2024-09-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2600000000000002e-06 2024-09-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492707+0000", 2024-09-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1899000000000001e-05 2024-09-13T07:01:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492715+0000", 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.422e-06 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492750+0000", 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4737000000000003e-05 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492818+0000", 2024-09-13T07:01:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8021000000000005e-05 2024-09-13T07:01:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.492969+0000", 2024-09-13T07:01:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000151614 2024-09-13T07:01:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.494065+0000", 2024-09-13T07:01:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010958000000000001 2024-09-13T07:01:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497536+0000", 2024-09-13T07:01:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034705970000000002 2024-09-13T07:01:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497614+0000", 2024-09-13T07:01:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8617999999999998e-05 2024-09-13T07:01:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.497659+0000", 2024-09-13T07:01:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4263e-05 2024-09-13T07:01:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:949 3.a 3:52cbfadd:::rbd_data.1194d22ad098.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.495440+0000", 2024-09-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2376923770000001, 2024-09-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005241791, 2024-09-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 949 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495440+0000", 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495440+0000", 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495439+0000", 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495446+0000", 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2269999999999996e-06 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495460+0000", 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3247999999999999e-05 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495467+0000", 2024-09-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5129999999999999e-06 2024-09-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495521+0000", 2024-09-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4020000000000001e-05 2024-09-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495618+0000", 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6934000000000004e-05 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495860+0000", 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024149800000000001 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500444+0000", 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045838909999999997 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500595+0000", 2024-09-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015186099999999999 2024-09-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500648+0000", 2024-09-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2877999999999998e-05 2024-09-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500682+0000", 2024-09-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3818e-05 2024-09-13T07:01:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.6742120040000001, 2024-09-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:01:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:01:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:01:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:01:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:01:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:01:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:01:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:01:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:01:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:01:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:01:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:01:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:01:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1534 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14365141] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.6673844529999999, 2024-09-13T07:01:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040478761000000002, 2024-09-13T07:01:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1534 2024-09-13T07:01:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065747+0000", 2024-09-13T07:01:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075258+0000", 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095108710000000006 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075288+0000", 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0085000000000001e-05 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075305+0000", 2024-09-13T07:01:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7748e-05 2024-09-13T07:01:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075367+0000", 2024-09-13T07:01:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1867999999999998e-05 2024-09-13T07:01:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075520+0000", 2024-09-13T07:01:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015298800000000001 2024-09-13T07:01:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.076247+0000", 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072720499999999997 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105157+0000", 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028909642999999999 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106124+0000", 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096729400000000005 2024-09-13T07:01:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106173+0000", 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8196999999999999e-05 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106227+0000", 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4388999999999997e-05 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1557 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.5926187829999998, 2024-09-13T07:01:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012339057, 2024-09-13T07:01:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1557 2024-09-13T07:01:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140502+0000", 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999881 2024-09-13T07:01:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140526+0000", 2024-09-13T07:01:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4049999999999998e-05 2024-09-13T07:01:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140563+0000", 2024-09-13T07:01:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6569000000000002e-05 2024-09-13T07:01:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140580+0000", 2024-09-13T07:01:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7725000000000001e-05 2024-09-13T07:01:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140942+0000", 2024-09-13T07:01:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036136300000000002 2024-09-13T07:01:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.141130+0000", 2024-09-13T07:01:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018814999999999999 2024-09-13T07:01:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.141433+0000", 2024-09-13T07:01:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030329899999999998 2024-09-13T07:01:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.147925+0000", 2024-09-13T07:01:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064915240000000003 2024-09-13T07:01:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152757+0000", 2024-09-13T07:01:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048328709999999999 2024-09-13T07:01:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152802+0000", 2024-09-13T07:01:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4314000000000003e-05 2024-09-13T07:01:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152853+0000", 2024-09-13T07:01:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1149000000000001e-05 2024-09-13T07:01:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1558 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.5925210449999998, 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012353627000000001, 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1558 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140610+0000", 2024-09-13T07:01:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140619+0000", 2024-09-13T07:01:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8650000000000007e-06 2024-09-13T07:01:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140634+0000", 2024-09-13T07:01:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4515000000000001e-05 2024-09-13T07:01:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140646+0000", 2024-09-13T07:01:29.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1736000000000001e-05 2024-09-13T07:01:29.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.144258+0000", 2024-09-13T07:01:29.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036123499999999998 2024-09-13T07:01:29.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.145034+0000", 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00077638900000000005 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.145395+0000", 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036117499999999998 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.149775+0000", 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043795639999999999 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152906+0000", 2024-09-13T07:01:29.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031306929999999999 2024-09-13T07:01:29.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152941+0000", 2024-09-13T07:01:29.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4919000000000002e-05 2024-09-13T07:01:29.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152965+0000", 2024-09-13T07:01:29.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4641e-05 2024-09-13T07:01:29.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.788 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.788 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.788 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1602 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 6291456~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.788 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.212548+0000", 2024-09-13T07:01:29.788 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.5205844019999999, 2024-09-13T07:01:29.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048322590000000002, 2024-09-13T07:01:29.788 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.788 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:29.788 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.788 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.788 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.789 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1602 2024-09-13T07:01:29.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.789 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212548+0000", 2024-09-13T07:01:29.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212548+0000", 2024-09-13T07:01:29.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212548+0000", 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212551+0000", 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.366e-06 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212558+0000", 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.212563+0000", 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6760000000000004e-06 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.213190+0000", 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00062709300000000001 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.213268+0000", 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7517999999999999e-05 2024-09-13T07:01:29.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.217381+0000", 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041125780000000004 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1648 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,zero 10485760~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.297923+0000", 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4352095629999999, 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053690209999999999, 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.792 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:29.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.793 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1648 2024-09-13T07:01:29.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.793 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297923+0000", 2024-09-13T07:01:29.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297923+0000", 2024-09-13T07:01:29.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297920+0000", 2024-09-13T07:01:29.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:01:29.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297941+0000", 2024-09-13T07:01:29.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1628999999999999e-05 2024-09-13T07:01:29.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297986+0000", 2024-09-13T07:01:29.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4135999999999999e-05 2024-09-13T07:01:29.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298009+0000", 2024-09-13T07:01:29.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3206000000000001e-05 2024-09-13T07:01:29.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298236+0000", 2024-09-13T07:01:29.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000227471 2024-09-13T07:01:29.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298526+0000", 2024-09-13T07:01:29.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028929800000000001 2024-09-13T07:01:29.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.303292+0000", 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047665570000000003 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1649 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 10485760~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.298045+0000", 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.435088125, 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055560130000000003, 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.797 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1649 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298045+0000", 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298045+0000", 2024-09-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298043+0000", 2024-09-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298052+0000", 2024-09-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4829999999999999e-06 2024-09-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298069+0000", 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7073000000000001e-05 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298083+0000", 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3977e-05 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.300163+0000", 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020800229999999999 2024-09-13T07:01:29.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.300370+0000", 2024-09-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000207679 2024-09-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.303601+0000", 2024-09-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032301550000000002 2024-09-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1661 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [read 551881~5223] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.317063+0000", 2024-09-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4160692089999998, 2024-09-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066193119999999996, 2024-09-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1661 2024-09-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.802 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317063+0000", 2024-09-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317063+0000", 2024-09-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317062+0000", 2024-09-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317070+0000", 2024-09-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9370000000000004e-06 2024-09-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317086+0000", 2024-09-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5685e-05 2024-09-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317093+0000", 2024-09-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6190000000000002e-06 2024-09-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317144+0000", 2024-09-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0587999999999997e-05 2024-09-13T07:01:29.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317148+0000", 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0949999999999998e-06 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.323168+0000", 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060199110000000002 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.323288+0000", 2024-09-13T07:01:29.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120375 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.323683+0000", 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00039459599999999999 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1675 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,zero 4194304~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.340801+0000", 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.3923313689999999, 2024-09-13T07:01:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004445867, 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1675 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340801+0000", 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340801+0000", 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340799+0000", 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340812+0000", 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3395e-05 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340845+0000", 2024-09-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2886000000000003e-05 2024-09-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340862+0000", 2024-09-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6633000000000002e-05 2024-09-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340946+0000", 2024-09-13T07:01:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3961999999999993e-05 2024-09-13T07:01:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341064+0000", 2024-09-13T07:01:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118948 2024-09-13T07:01:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.345247+0000", 2024-09-13T07:01:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041826900000000002 2024-09-13T07:01:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1676 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.3922428409999998, 2024-09-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067554429999999999, 2024-09-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1676 2024-09-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340889+0000", 2024-09-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340896+0000", 2024-09-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1420000000000004e-06 2024-09-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340909+0000", 2024-09-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3329000000000001e-05 2024-09-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340920+0000", 2024-09-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1029999999999999e-05 2024-09-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341653+0000", 2024-09-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073243400000000001 2024-09-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341825+0000", 2024-09-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017243099999999999 2024-09-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.347645+0000", 2024-09-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058200639999999998 2024-09-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1761 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.454709+0000", 2024-09-13T07:01:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.2784233450000002, 2024-09-13T07:01:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049941610000000004, 2024-09-13T07:01:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1761 2024-09-13T07:01:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454709+0000", 2024-09-13T07:01:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454709+0000", 2024-09-13T07:01:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454708+0000", 2024-09-13T07:01:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454714+0000", 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1970000000000001e-06 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454728+0000", 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3212e-05 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454738+0000", 2024-09-13T07:01:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0475000000000001e-05 2024-09-13T07:01:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.455883+0000", 2024-09-13T07:01:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001144438 2024-09-13T07:01:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.456054+0000", 2024-09-13T07:01:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000171402 2024-09-13T07:01:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.459703+0000", 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036494819999999999 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1807 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.542393+0000", 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.190739239, 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004754515, 2024-09-13T07:01:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1807 2024-09-13T07:01:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542393+0000", 2024-09-13T07:01:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542393+0000", 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542390+0000", 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542412+0000", 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2206999999999999e-05 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542454+0000", 2024-09-13T07:01:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1508999999999998e-05 2024-09-13T07:01:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542475+0000", 2024-09-13T07:01:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1086000000000001e-05 2024-09-13T07:01:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542692+0000", 2024-09-13T07:01:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000217235 2024-09-13T07:01:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542944+0000", 2024-09-13T07:01:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025154799999999998 2024-09-13T07:01:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.543278+0000", 2024-09-13T07:01:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00033458899999999999 2024-09-13T07:01:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.546350+0000", 2024-09-13T07:01:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003071463 2024-09-13T07:01:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.546987+0000", 2024-09-13T07:01:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00063671699999999997 2024-09-13T07:01:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547075+0000", 2024-09-13T07:01:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8171999999999995e-05 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547148+0000", 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3172999999999999e-05 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1808 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.542510+0000", 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.1906226389999999, 2024-09-13T07:01:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050197829999999999, 2024-09-13T07:01:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1808 2024-09-13T07:01:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542510+0000", 2024-09-13T07:01:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542510+0000", 2024-09-13T07:01:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542509+0000", 2024-09-13T07:01:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542516+0000", 2024-09-13T07:01:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2200000000000003e-06 2024-09-13T07:01:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542533+0000", 2024-09-13T07:01:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6957e-05 2024-09-13T07:01:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542549+0000", 2024-09-13T07:01:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5959000000000001e-05 2024-09-13T07:01:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.545392+0000", 2024-09-13T07:01:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028432330000000001 2024-09-13T07:01:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.545520+0000", 2024-09-13T07:01:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012757400000000001 2024-09-13T07:01:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.545722+0000", 2024-09-13T07:01:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020242400000000001 2024-09-13T07:01:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547178+0000", 2024-09-13T07:01:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014557509999999999 2024-09-13T07:01:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547448+0000", 2024-09-13T07:01:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026981900000000003 2024-09-13T07:01:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547496+0000", 2024-09-13T07:01:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8680000000000001e-05 2024-09-13T07:01:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547530+0000", 2024-09-13T07:01:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3466000000000001e-05 2024-09-13T07:01:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1854 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,zero 4194304~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.135123122, 2024-09-13T07:01:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070961979999999997, 2024-09-13T07:01:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1854 2024-09-13T07:01:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598007+0000", 2024-09-13T07:01:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598022+0000", 2024-09-13T07:01:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-13T07:01:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598062+0000", 2024-09-13T07:01:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9552999999999999e-05 2024-09-13T07:01:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598081+0000", 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9247999999999999e-05 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598256+0000", 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000174482 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598365+0000", 2024-09-13T07:01:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109108 2024-09-13T07:01:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.605106+0000", 2024-09-13T07:01:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067409189999999997 2024-09-13T07:01:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1855 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.1350148419999999, 2024-09-13T07:01:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074110900000000004, 2024-09-13T07:01:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1855 2024-09-13T07:01:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598117+0000", 2024-09-13T07:01:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598125+0000", 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3110000000000006e-06 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598147+0000", 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2484999999999999e-05 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598159+0000", 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1939999999999999e-05 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.599204+0000", 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010449680000000001 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.599409+0000", 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020456700000000001 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.605529+0000", 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061199790000000002 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1865 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.614475+0000", 2024-09-13T07:01:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.1186577189999998, 2024-09-13T07:01:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043504609999999999, 2024-09-13T07:01:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1865 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.614475+0000", 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.614475+0000", 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.614473+0000", 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.614485+0000", 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1798e-05 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.614506+0000", 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1725e-05 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.614517+0000", 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0756e-05 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.614583+0000", 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5606999999999997e-05 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.614677+0000", 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3818000000000002e-05 2024-09-13T07:01:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.615317+0000", 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000640472 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.618301+0000", 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029835650000000001 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.618671+0000", 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00037034600000000002 2024-09-13T07:01:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.618748+0000", 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7051999999999993e-05 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.618825+0000", 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7401e-05 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:29.848 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:29.850 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:01:29.850 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:29.844+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:29.951 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:29.945+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:30.003 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:30.003 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:30.003 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:30.003 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:30.013 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:01:30.057 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:30.053+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:30.117 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:30.117 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:30.118 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:30.118 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:30.118 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:30.127 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:01:30.152 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:30.149+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:30.237 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.512821+0000", 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.719379173, 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012539794, 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.512821+0000", 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.512821+0000", 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.512820+0000", 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.512972+0000", 2024-09-13T07:01:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015206900000000001 2024-09-13T07:01:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.512993+0000", 2024-09-13T07:01:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0993e-05 2024-09-13T07:01:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513022+0000", 2024-09-13T07:01:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8656000000000001e-05 2024-09-13T07:01:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.519544+0000", 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065228389999999999 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.519691+0000", 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014618000000000001 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.519917+0000", 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000226185 2024-09-13T07:01:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.523679+0000", 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003762086 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525217+0000", 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001537846 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525307+0000", 2024-09-13T07:01:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0531000000000004e-05 2024-09-13T07:01:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525361+0000", 2024-09-13T07:01:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3851999999999999e-05 2024-09-13T07:01:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513067+0000", 2024-09-13T07:01:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.719133926000001, 2024-09-13T07:01:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013413369, 2024-09-13T07:01:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-13T07:01:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513067+0000", 2024-09-13T07:01:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513067+0000", 2024-09-13T07:01:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513065+0000", 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513078+0000", 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2819999999999999e-05 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513105+0000", 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7501999999999999e-05 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513129+0000", 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4392999999999998e-05 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.521691+0000", 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0085619879999999995 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.521810+0000", 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118775 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.521988+0000", 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017780000000000001 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525380+0000", 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003392445 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526310+0000", 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00092912300000000001 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526396+0000", 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6163999999999997e-05 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526480+0000", 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4215000000000006e-05 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513148+0000", 2024-09-13T07:01:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.719052739999999, 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013731426999999999, 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513148+0000", 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513148+0000", 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513147+0000", 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513157+0000", 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0686000000000001e-05 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513180+0000", 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2160000000000001e-05 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513193+0000", 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.522568+0000", 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093749149999999993 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.522733+0000", 2024-09-13T07:01:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016480999999999999 2024-09-13T07:01:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.522992+0000", 2024-09-13T07:01:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000259258 2024-09-13T07:01:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526515+0000", 2024-09-13T07:01:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035229509999999999 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526680+0000", 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016493000000000001 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526811+0000", 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013147199999999999 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526879+0000", 2024-09-13T07:01:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7836000000000006e-05 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513206+0000", 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.718994245000001, 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014937498, 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513206+0000", 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513206+0000", 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513205+0000", 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513444+0000", 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000238989 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513477+0000", 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3118999999999999e-05 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513508+0000", 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1259e-05 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.523747+0000", 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010238858999999999 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.523925+0000", 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000177846 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.524169+0000", 2024-09-13T07:01:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000243818 2024-09-13T07:01:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.527925+0000", 2024-09-13T07:01:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037561029999999998 2024-09-13T07:01:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528043+0000", 2024-09-13T07:01:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011843 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528097+0000", 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3612000000000001e-05 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528144+0000", 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7057e-05 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513585+0000", 2024-09-13T07:01:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.718615982999999, 2024-09-13T07:01:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014919218, 2024-09-13T07:01:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-13T07:01:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513585+0000", 2024-09-13T07:01:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513585+0000", 2024-09-13T07:01:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513583+0000", 2024-09-13T07:01:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513595+0000", 2024-09-13T07:01:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1878999999999999e-05 2024-09-13T07:01:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513612+0000", 2024-09-13T07:01:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.721e-05 2024-09-13T07:01:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513639+0000", 2024-09-13T07:01:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6733e-05 2024-09-13T07:01:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525467+0000", 2024-09-13T07:01:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828200000000001 2024-09-13T07:01:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525618+0000", 2024-09-13T07:01:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015065 2024-09-13T07:01:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.525783+0000", 2024-09-13T07:01:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016588099999999999 2024-09-13T07:01:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528270+0000", 2024-09-13T07:01:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024861330000000002 2024-09-13T07:01:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528292+0000", 2024-09-13T07:01:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2127e-05 2024-09-13T07:01:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528405+0000", 2024-09-13T07:01:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113583 2024-09-13T07:01:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528504+0000", 2024-09-13T07:01:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8492000000000005e-05 2024-09-13T07:01:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.718535585000001, 2024-09-13T07:01:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018207798000000001, 2024-09-13T07:01:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-13T07:01:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513664+0000", 2024-09-13T07:01:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513674+0000", 2024-09-13T07:01:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8770000000000005e-06 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513695+0000", 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1319e-05 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513714+0000", 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.914e-05 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526929+0000", 2024-09-13T07:01:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013215146000000001 2024-09-13T07:01:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.527080+0000", 2024-09-13T07:01:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000150824 2024-09-13T07:01:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.527312+0000", 2024-09-13T07:01:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023218100000000001 2024-09-13T07:01:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530679+0000", 2024-09-13T07:01:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003366529 2024-09-13T07:01:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531659+0000", 2024-09-13T07:01:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0009803940000000001 2024-09-13T07:01:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531784+0000", 2024-09-13T07:01:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012519399999999999 2024-09-13T07:01:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531873+0000", 2024-09-13T07:01:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8307999999999995e-05 2024-09-13T07:01:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.718473316000001, 2024-09-13T07:01:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018374253, 2024-09-13T07:01:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-13T07:01:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513726+0000", 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513933+0000", 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020722500000000001 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513984+0000", 2024-09-13T07:01:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0719e-05 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513999+0000", 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4877e-05 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528581+0000", 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014581604999999999 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528674+0000", 2024-09-13T07:01:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3021000000000003e-05 2024-09-13T07:01:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528803+0000", 2024-09-13T07:01:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012964199999999999 2024-09-13T07:01:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531891+0000", 2024-09-13T07:01:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00308764 2024-09-13T07:01:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531990+0000", 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9407999999999994e-05 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532064+0000", 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3793000000000003e-05 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532101+0000", 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7357000000000002e-05 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.718184056999998, 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018326366, 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-13T07:01:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514015+0000", 2024-09-13T07:01:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514026+0000", 2024-09-13T07:01:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1053000000000001e-05 2024-09-13T07:01:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514047+0000", 2024-09-13T07:01:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0687e-05 2024-09-13T07:01:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514065+0000", 2024-09-13T07:01:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8799000000000001e-05 2024-09-13T07:01:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529495+0000", 2024-09-13T07:01:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015429877 2024-09-13T07:01:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529642+0000", 2024-09-13T07:01:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014682399999999999 2024-09-13T07:01:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529871+0000", 2024-09-13T07:01:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022936399999999999 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532183+0000", 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023112889999999998 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532199+0000", 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6082e-05 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532269+0000", 2024-09-13T07:01:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9812000000000002e-05 2024-09-13T07:01:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532343+0000", 2024-09-13T07:01:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4141999999999996e-05 2024-09-13T07:01:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.718121821, 2024-09-13T07:01:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018420999, 2024-09-13T07:01:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-13T07:01:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514078+0000", 2024-09-13T07:01:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514088+0000", 2024-09-13T07:01:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0433e-05 2024-09-13T07:01:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514104+0000", 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5616000000000001e-05 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514133+0000", 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9453000000000001e-05 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530751+0000", 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016617996999999999 2024-09-13T07:01:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530926+0000", 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017493599999999999 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531174+0000", 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024812100000000002 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532358+0000", 2024-09-13T07:01:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00118396 2024-09-13T07:01:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532421+0000", 2024-09-13T07:01:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2749999999999994e-05 2024-09-13T07:01:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532467+0000", 2024-09-13T07:01:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6427000000000001e-05 2024-09-13T07:01:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532500+0000", 2024-09-13T07:01:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2329000000000002e-05 2024-09-13T07:01:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.071028+0000", 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.161172579999999, 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011230551, 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 136 2024-09-13T07:01:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071028+0000", 2024-09-13T07:01:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071028+0000", 2024-09-13T07:01:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071028+0000", 2024-09-13T07:01:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071031+0000", 2024-09-13T07:01:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1810000000000001e-06 2024-09-13T07:01:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071035+0000", 2024-09-13T07:01:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5129999999999998e-06 2024-09-13T07:01:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071044+0000", 2024-09-13T07:01:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2679999999999994e-06 2024-09-13T07:01:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.075902+0000", 2024-09-13T07:01:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048583050000000003 2024-09-13T07:01:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.075969+0000", 2024-09-13T07:01:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7552000000000006e-05 2024-09-13T07:01:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.076084+0000", 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114164 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.080975+0000", 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048912460000000001 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082155+0000", 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001180277 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082201+0000", 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5958999999999998e-05 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082258+0000", 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7410000000000001e-05 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.071106+0000", 2024-09-13T07:01:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.161094318, 2024-09-13T07:01:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011381179999999999, 2024-09-13T07:01:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 137 2024-09-13T07:01:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071106+0000", 2024-09-13T07:01:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071106+0000", 2024-09-13T07:01:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071106+0000", 2024-09-13T07:01:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071197+0000", 2024-09-13T07:01:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1367999999999997e-05 2024-09-13T07:01:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071205+0000", 2024-09-13T07:01:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6769999999999995e-06 2024-09-13T07:01:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071210+0000", 2024-09-13T07:01:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.135e-06 2024-09-13T07:01:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.076472+0000", 2024-09-13T07:01:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052622579999999997 2024-09-13T07:01:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.076578+0000", 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010632999999999999 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.076721+0000", 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014302100000000001 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082014+0000", 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052930720000000002 2024-09-13T07:01:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082342+0000", 2024-09-13T07:01:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000327208 2024-09-13T07:01:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082408+0000", 2024-09-13T07:01:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5889000000000006e-05 2024-09-13T07:01:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082487+0000", 2024-09-13T07:01:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9857999999999993e-05 2024-09-13T07:01:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.071216+0000", 2024-09-13T07:01:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.160984825, 2024-09-13T07:01:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011448228, 2024-09-13T07:01:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-13T07:01:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071216+0000", 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071216+0000", 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071215+0000", 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071219+0000", 2024-09-13T07:01:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4570000000000001e-06 2024-09-13T07:01:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071223+0000", 2024-09-13T07:01:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5120000000000002e-06 2024-09-13T07:01:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071227+0000", 2024-09-13T07:01:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1509999999999997e-06 2024-09-13T07:01:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.077362+0000", 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061349619999999999 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.077445+0000", 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2330000000000006e-05 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.077576+0000", 2024-09-13T07:01:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013119100000000001 2024-09-13T07:01:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082509+0000", 2024-09-13T07:01:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049333729999999996 2024-09-13T07:01:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082575+0000", 2024-09-13T07:01:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5507999999999997e-05 2024-09-13T07:01:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082612+0000", 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7058000000000003e-05 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082664+0000", 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2188000000000002e-05 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.071232+0000", 2024-09-13T07:01:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.160968907000001, 2024-09-13T07:01:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011600325999999999, 2024-09-13T07:01:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-13T07:01:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071232+0000", 2024-09-13T07:01:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071232+0000", 2024-09-13T07:01:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071231+0000", 2024-09-13T07:01:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071234+0000", 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9160000000000001e-06 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071239+0000", 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4830000000000001e-06 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071243+0000", 2024-09-13T07:01:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9079999999999997e-06 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.078327+0000", 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070842199999999996 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.078438+0000", 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111397 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.078591+0000", 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000153034 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082518+0000", 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039266989999999996 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082730+0000", 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021233999999999999 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082788+0000", 2024-09-13T07:01:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7271000000000001e-05 2024-09-13T07:01:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082832+0000", 2024-09-13T07:01:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4384000000000001e-05 2024-09-13T07:01:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.071306+0000", 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.160894814999999, 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011684793000000001, 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-13T07:01:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071306+0000", 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071306+0000", 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071305+0000", 2024-09-13T07:01:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071400+0000", 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4734999999999993e-05 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071408+0000", 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9789999999999993e-06 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071413+0000", 2024-09-13T07:01:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1170000000000003e-06 2024-09-13T07:01:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.079114+0000", 2024-09-13T07:01:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077009069999999999 2024-09-13T07:01:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.079215+0000", 2024-09-13T07:01:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101072 2024-09-13T07:01:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.079361+0000", 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146002 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082530+0000", 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031688290000000002 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082884+0000", 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000354662 2024-09-13T07:01:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082939+0000", 2024-09-13T07:01:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4768000000000001e-05 2024-09-13T07:01:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.082990+0000", 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1348999999999999e-05 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.071419+0000", 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.160781918000001, 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012118689, 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-13T07:01:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071419+0000", 2024-09-13T07:01:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071419+0000", 2024-09-13T07:01:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071418+0000", 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071422+0000", 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7740000000000002e-06 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071427+0000", 2024-09-13T07:01:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6480000000000002e-06 2024-09-13T07:01:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.071431+0000", 2024-09-13T07:01:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0980000000000004e-06 2024-09-13T07:01:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.081143+0000", 2024-09-13T07:01:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097121129999999996 2024-09-13T07:01:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.081240+0000", 2024-09-13T07:01:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6935000000000006e-05 2024-09-13T07:01:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.081445+0000", 2024-09-13T07:01:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020548300000000001 2024-09-13T07:01:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.083059+0000", 2024-09-13T07:01:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016134840000000001 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.083408+0000", 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00034926199999999998 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.083491+0000", 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3140999999999996e-05 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.083537+0000", 2024-09-13T07:01:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6193999999999999e-05 2024-09-13T07:01:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:01:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.850389+0000", 2024-09-13T07:01:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 26.381811503000002, 2024-09-13T07:01:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01082141, 2024-09-13T07:01:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-13T07:01:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850389+0000", 2024-09-13T07:01:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850389+0000", 2024-09-13T07:01:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850388+0000", 2024-09-13T07:01:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850570+0000", 2024-09-13T07:01:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000182299 2024-09-13T07:01:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850596+0000", 2024-09-13T07:01:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5582999999999999e-05 2024-09-13T07:01:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850610+0000", 2024-09-13T07:01:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4209999999999999e-05 2024-09-13T07:01:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.857529+0000", 2024-09-13T07:01:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069189580000000002 2024-09-13T07:01:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.857676+0000", 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146797 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.857926+0000", 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024991199999999997 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.860954+0000", 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030283889999999998 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.861011+0000", 2024-09-13T07:01:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6783999999999997e-05 2024-09-13T07:01:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.861129+0000", 2024-09-13T07:01:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117738 2024-09-13T07:01:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.861210+0000", 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1760000000000003e-05 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:03.850631+0000", 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 26.381569904999999, 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011927366, 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850631+0000", 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850631+0000", 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850630+0000", 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850640+0000", 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0193e-05 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850680+0000", 2024-09-13T07:01:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9818999999999999e-05 2024-09-13T07:01:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.850694+0000", 2024-09-13T07:01:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4752999999999999e-05 2024-09-13T07:01:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.860129+0000", 2024-09-13T07:01:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094350860000000005 2024-09-13T07:01:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.860260+0000", 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013028900000000001 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.860400+0000", 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014016300000000001 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.861852+0000", 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001452231 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.862448+0000", 2024-09-13T07:01:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00059627400000000004 2024-09-13T07:01:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.862494+0000", 2024-09-13T07:01:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5785000000000003e-05 2024-09-13T07:01:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:03.862558+0000", 2024-09-13T07:01:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3806000000000005e-05 2024-09-13T07:01:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1174 3.13 3:cb2ed8dc:::rbd_data.11943a018f34.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 6419344~27893] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:01:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3484038460000001, 2024-09-13T07:01:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041813481999999999, 2024-09-13T07:01:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1174 2024-09-13T07:01:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:01:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:01:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883795+0000", 2024-09-13T07:01:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894234+0000", 2024-09-13T07:01:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010439133 2024-09-13T07:01:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894276+0000", 2024-09-13T07:01:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429000000000003e-05 2024-09-13T07:01:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894296+0000", 2024-09-13T07:01:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9279999999999998e-05 2024-09-13T07:01:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894477+0000", 2024-09-13T07:01:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018160600000000001 2024-09-13T07:01:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894889+0000", 2024-09-13T07:01:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041202399999999998 2024-09-13T07:01:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.898212+0000", 2024-09-13T07:01:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033228400000000001 2024-09-13T07:01:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.922541+0000", 2024-09-13T07:01:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024329052 2024-09-13T07:01:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925458+0000", 2024-09-13T07:01:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002916891 2024-09-13T07:01:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925531+0000", 2024-09-13T07:01:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2781999999999995e-05 2024-09-13T07:01:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925610+0000", 2024-09-13T07:01:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9355000000000002e-05 2024-09-13T07:01:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:01:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:01:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1780280449999996, 2024-09-13T07:01:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:01:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:01:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:01:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:01:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:01:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:01:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:01:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:01:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:01:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:01:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:01:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:01:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:01:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:01:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:01:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:01:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:01:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:01:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1535 3.12 e27/22 3:4d1f9422:::rbd_object_map.11943a018f34:head v 27'27, mlcod=27'25)", 2024-09-13T07:01:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:01:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1640712139999998, 2024-09-13T07:01:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027873574000000002, 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068128+0000", 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068140+0000", 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2279000000000001e-05 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068175+0000", 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4872000000000001e-05 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068193+0000", 2024-09-13T07:01:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7540000000000001e-05 2024-09-13T07:01:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068302+0000", 2024-09-13T07:01:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010928 2024-09-13T07:01:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068351+0000", 2024-09-13T07:01:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036000000000003e-05 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.095950+0000", 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027598988000000001 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.096003+0000", 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2722000000000001e-05 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:30.378 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:30.379 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T07:01:30.379 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:01:30.380 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:30.373+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:30.381 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:01:30.381 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T07:01:30.482 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:30.476+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:30.490 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone4 from 0xde79ad6 (overlap 0x7d984) to 0x0 2024-09-13T07:01:30.545 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:30.545 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:30.545 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:30.545 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:30.556 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:01:30.583 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:30.580+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:30.615 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-13T07:01:30.627 INFO:teuthology.orchestra.run.smithi118.stdout:409 read 0x4661194 thru 0x4667e95 (0x6d02 bytes) 2024-09-13T07:01:30.628 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:30.636 INFO:teuthology.orchestra.run.smithi118.stdout:411 trunc from 0xde79ad6 to 0x84f46bd 2024-09-13T07:01:30.665 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:30.665 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:30.665 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:30.665 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:30.665 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:30.680 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:01:30.700 INFO:teuthology.orchestra.run.smithi118.stdout:412 writesame 0x9991e2c thru 0x99a05ad data_size 0x1(0xe782 bytes) 2024-09-13T07:01:30.701 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:30.696+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:30.702 INFO:teuthology.orchestra.run.smithi118.stdout:414 write 0x54b3bac thru 0x54bb435 (0x788a bytes) 2024-09-13T07:01:30.705 INFO:teuthology.orchestra.run.smithi118.stdout:415 writesame 0x48a94a thru 0x497e1d data_size 0x35(0xd4d4 bytes) 2024-09-13T07:01:30.722 INFO:teuthology.orchestra.run.smithi118.stdout:416 trunc from 0x99a05ae to 0x4f76fa0 2024-09-13T07:01:30.784 INFO:teuthology.orchestra.run.smithi118.stdout:417 write 0xb9baf1d thru 0xb9bf518 (0x45fc bytes) 2024-09-13T07:01:30.784 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:30.780+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:30.789 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:30.801 INFO:teuthology.orchestra.run.smithi118.stdout:419 write 0xc281fb5 thru 0xc291705 (0xf751 bytes) 2024-09-13T07:01:30.808 INFO:teuthology.orchestra.run.smithi118.stdout:420 read 0x52dcac4 thru 0x52e58e4 (0x8e21 bytes) 2024-09-13T07:01:30.808 INFO:teuthology.orchestra.run.smithi118.stdout:421 write 0x88eaa18 thru 0x88eb8c3 (0xeac bytes) 2024-09-13T07:01:30.813 INFO:teuthology.orchestra.run.smithi118.stdout:422 read 0xa5d8162 thru 0xa5da115 (0x1fb4 bytes) 2024-09-13T07:01:30.813 INFO:teuthology.orchestra.run.smithi118.stdout:423 read 0x698b100 thru 0x6996e71 (0xbd72 bytes) 2024-09-13T07:01:30.814 INFO:teuthology.orchestra.run.smithi118.stdout:425 read 0x7999bbc thru 0x799d188 (0x35cd bytes) 2024-09-13T07:01:30.814 INFO:teuthology.orchestra.run.smithi118.stdout:426 write 0x6673e5a thru 0x6682bb2 (0xed59 bytes) 2024-09-13T07:01:30.820 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:30.820 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:30.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:30.820 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:30.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.820 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:30.820 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:30.820 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.312114824, 2024-09-13T07:01:30.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:01:30.820 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:01:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:01:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:01:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:01:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:01:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:01:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:01:30.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:01:30.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.2872871139999997, 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.826 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:01:30.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:30.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:01:30.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.74861529, 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.831 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:30.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:01:30.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:30.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:01:30.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:01:30.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:01:30.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:01:30.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:01:30.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:01:30.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:01:30.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:01:30.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:01:30.836 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7189900969999998, 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:01:30.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:30.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:01:30.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:01:30.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:01:30.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:01:30.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:30.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:30.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:30.842 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.1862764659999998, 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:01:30.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:01:30.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:01:30.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:01:30.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:30.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:01:30.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:01:30.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.1861940100000004, 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.848 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:01:30.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:01:30.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:01:30.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:01:30.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:30.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:01:30.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:01:30.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:01:30.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:01:30.853 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.853 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.1861622000000001, 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:01:30.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:01:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.859 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.1861070419999997, 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:30.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:01:30.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:01:30.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:01:30.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:01:30.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:01:30.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:01:30.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:01:30.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:30.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:01:30.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:01:30.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:30.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:01:30.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.1860617629999997, 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:01:30.865 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:30.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:01:30.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:01:30.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:01:30.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.870 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.1860232289999999, 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.1859616099999997, 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.876 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.877 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:01:30.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.877 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:30.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:01:30.881 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.1717389269999998, 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.882 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:01:30.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3951 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13369344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:30.887 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657045+0000", 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.15988039100000001, 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090047159999999994, 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3951 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657045+0000", 2024-09-13T07:01:30.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657045+0000", 2024-09-13T07:01:30.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657044+0000", 2024-09-13T07:01:30.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657046+0000", 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657052+0000", 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8980000000000001e-06 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657057+0000", 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7790000000000003e-06 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665904+0000", 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0088473179999999998 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665974+0000", 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0038999999999998e-05 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666049+0000", 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4777999999999995e-05 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.15986040700000001, 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:01:30.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:01:30.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:01:30.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:01:30.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:01:30.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:01:30.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:01:30.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:01:30.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:01:30.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:01:30.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:01:30.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:01:30.896 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.896 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.896 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.896 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4000 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:30.896 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657605+0000", 2024-09-13T07:01:30.896 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.159320293, 2024-09-13T07:01:30.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089452449999999992, 2024-09-13T07:01:30.896 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.896 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:30.896 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.897 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4000 2024-09-13T07:01:30.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.897 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657605+0000", 2024-09-13T07:01:30.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657605+0000", 2024-09-13T07:01:30.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657604+0000", 2024-09-13T07:01:30.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657606+0000", 2024-09-13T07:01:30.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.018e-06 2024-09-13T07:01:30.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657612+0000", 2024-09-13T07:01:30.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9519999999999999e-06 2024-09-13T07:01:30.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657617+0000", 2024-09-13T07:01:30.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8069999999999999e-06 2024-09-13T07:01:30.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666410+0000", 2024-09-13T07:01:30.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087930149999999995 2024-09-13T07:01:30.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666478+0000", 2024-09-13T07:01:30.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8023999999999998e-05 2024-09-13T07:01:30.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666550+0000", 2024-09-13T07:01:30.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1796000000000002e-05 2024-09-13T07:01:30.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.901 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.901 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:30.901 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:30.901 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.15930334600000001, 2024-09-13T07:01:30.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:30.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:01:30.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:01:30.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:01:30.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:01:30.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:01:30.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:01:30.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:01:30.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:01:30.905 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.905 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.15928119099999999, 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.906 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:01:30.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.15926147399999999, 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:01:30.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:01:30.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:01:30.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:01:30.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:01:30.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:01:30.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:01:30.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:01:30.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:01:30.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:01:30.914 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.914 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.914 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.159244144, 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.918 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.919 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.919 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:30.919 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:30.919 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.158948482, 2024-09-13T07:01:30.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:01:30.919 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:30.919 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:30.919 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:30.919 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:30.919 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:30.919 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:01:30.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.920 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:30.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:30.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:30.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:30.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:30.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:30.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:01:30.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:01:30.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:01:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:01:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:30.923 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:30.925 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:30.925 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:30.925 INFO:teuthology.orchestra.run.smithi118.stdout:431 read 0x67beec8 thru 0x67c7ae6 (0x8c1f bytes) 2024-09-13T07:01:30.925 INFO:teuthology.orchestra.run.smithi118.stdout:432 punch from 0x8e284b9 to 0x8e32cc9, (0xa810 bytes) 2024-09-13T07:01:30.925 INFO:teuthology.orchestra.run.smithi118.stdout:433 punch from 0x92511b2 to 0x9260a5a, (0xf8a8 bytes) 2024-09-13T07:01:30.925 INFO:teuthology.orchestra.run.smithi118.stdout:435 punch from 0xb7ebfc3 to 0xb7f9372, (0xd3af bytes) 2024-09-13T07:01:30.925 INFO:teuthology.orchestra.run.smithi118.stdout:436 read 0xb2f8b74 thru 0xb2fb29c (0x2729 bytes) 2024-09-13T07:01:30.925 INFO:teuthology.orchestra.run.smithi118.stdout:437 writesame 0x99b9d95 thru 0x99c6438 data_size 0x2(0xc6a4 bytes) 2024-09-13T07:01:30.926 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:01:30.927 INFO:teuthology.orchestra.run.smithi118.stdout:438 writesame 0x8ed39d thru 0x8f266d data_size 0x3(0x52d1 bytes) 2024-09-13T07:01:30.927 INFO:teuthology.orchestra.run.smithi118.stdout:439 read 0xa9b171b thru 0xa9bb325 (0x9c0b bytes) 2024-09-13T07:01:30.928 INFO:teuthology.orchestra.run.smithi118.stdout:440 write 0x318f119 thru 0x3197f47 (0x8e2f bytes) 2024-09-13T07:01:30.928 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:30.928 INFO:teuthology.orchestra.run.smithi118.stdout:444 writesame 0xc6ba2fe thru 0xc6bc1bf data_size 0x1f(0x1ec2 bytes) 2024-09-13T07:01:30.928 INFO:teuthology.orchestra.run.smithi118.stdout:445 punch from 0x2d02703 to 0x2d10523, (0xde20 bytes) 2024-09-13T07:01:30.928 INFO:teuthology.orchestra.run.smithi118.stdout:446 trunc from 0xc6bc1c0 to 0xd3e1b94 2024-09-13T07:01:30.928 INFO:teuthology.orchestra.run.smithi118.stdout:447 write 0xc3d0de0 thru 0xc3d1a05 (0xc26 bytes) 2024-09-13T07:01:30.928 INFO:teuthology.orchestra.run.smithi118.stdout:451 writesame 0x818d30 thru 0x81f767 data_size 0x67(0x6a38 bytes) 2024-09-13T07:01:30.928 INFO:teuthology.orchestra.run.smithi118.stdout:452 read 0x1c1d6a5 thru 0x1c22963 (0x52bf bytes) 2024-09-13T07:01:30.928 INFO:teuthology.orchestra.run.smithi118.stdout:453 writesame 0x6f82693 thru 0x6f91970 data_size 0x7(0xf2de bytes) 2024-09-13T07:01:30.928 INFO:teuthology.orchestra.run.smithi118.stdout:455 write 0x146f84 thru 0x14ce80 (0x5efd bytes) 2024-09-13T07:01:30.929 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:30.921+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:30.930 INFO:teuthology.orchestra.run.smithi118.stdout:457 write 0xaeaf9e2 thru 0xaeb2bb3 (0x31d2 bytes) 2024-09-13T07:01:30.936 INFO:teuthology.orchestra.run.smithi118.stdout:458 clone 6 order 23 su 2097152 sc 5 2024-09-13T07:01:31.026 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:31.021+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:31.041 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:31.041 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:31.042 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:31.042 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:31.052 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:01:31.128 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:31.124+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:31.202 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:01:31.205 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:31.205 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:31.205 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:31.205 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:31.205 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:31.215 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:01:31.230 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:31.225+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:31.330 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:31.328+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:31.342 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:31.363 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:31.365 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T07:01:31.367 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:31.367 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:31.378 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T07:01:31.378 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:01:31.379 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:01:31.382 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:31.402 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:31.425 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:31.431 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:31.428+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:31.463 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:31.490 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:31.490 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:31.492 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:31.501 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:01:31.514 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone5 from 0xd3e1b94 (overlap 0x4f76fa0) to 0x1fc930c 2024-09-13T07:01:31.515 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:31.519 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:31.519 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:31.519 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:31.519 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:31.519 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:31.519 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:31.519 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:31.519 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:31.533 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:31.528+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:31.617 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:31.617 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:31.627 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:01:31.634 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:31.628+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:31.671 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-13T07:01:31.700 INFO:teuthology.orchestra.run.smithi118.stdout:459 writesame 0xe493119 thru 0xe4a1cc7 data_size 0xb(0xebaf bytes) 2024-09-13T07:01:31.707 INFO:teuthology.orchestra.run.smithi118.stdout:460 writesame 0x2e91266 thru 0x2e9db13 data_size 0x11(0xc8ae bytes) 2024-09-13T07:01:31.721 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:31.721 INFO:teuthology.orchestra.run.smithi118.stdout:463 read 0x408748d thru 0x408e87b (0x73ef bytes) 2024-09-13T07:01:31.721 INFO:teuthology.orchestra.run.smithi118.stdout:464 write 0x9fd776 thru 0xa09e3e (0xc6c9 bytes) 2024-09-13T07:01:31.735 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:31.729+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:31.743 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:31.745 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:01:31.747 INFO:teuthology.orchestra.run.smithi118.stdout:465 punch from 0x7fb0b08 to 0x7fb3703, (0x2bfb bytes) 2024-09-13T07:01:31.747 INFO:teuthology.orchestra.run.smithi118.stdout:466 writesame 0x359ec41 thru 0x35added data_size 0x29(0xf1ad bytes) 2024-09-13T07:01:31.756 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:31.756 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:31.757 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-13T07:01:31.358868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":198450839,"num_objects":215,"num_object_clones":46,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":125,"num_read":6831,"num_read_kb":11922,"num_write":1357,"num_write_kb":44323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1236,"ondisk_log_size":1236,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254792,"kb_used_data":9816,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754719928,"statfs":{"total":773094113280,"available":772833206272,"internally_reserved":0,"allocated":10051584,"data_stored":8097004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":250596920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":178453971,"num_objects":55,"num_object_clones":18,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":12,"num_read":4753,"num_read_kb":9426,"num_write":768,"num_write_kb":40375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033900"},"pg_stats":[{"pgid":"3.1f","version":"32'30","reported_seq":185,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.667027+0000","last_change":"2024-09-13T07:01:21.306378+0000","last_active":"2024-09-13T07:01:30.667027+0000","last_peered":"2024-09-13T07:01:30.667027+0000","last_clean":"2024-09-13T07:01:30.667027+0000","last_became_active":"2024-09-13T07:01:19.282082+0000","last_became_peered":"2024-09-13T07:01:19.282082+0000","last_unstale":"2024-09-13T07:01:30.667027+0000","last_undegraded":"2024-09-13T07:01:30.667027+0000","last_fullsized":"2024-09-13T07:01:30.667027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:41.783324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013321120000000001,"stat_sum":{"num_bytes":20080440,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":151,"num_read_kb":58,"num_write":25,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"32'18","reported_seq":99,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.670487+0000","last_change":"2024-09-13T07:01:21.291845+0000","last_active":"2024-09-13T07:01:30.670487+0000","last_peered":"2024-09-13T07:01:30.670487+0000","last_clean":"2024-09-13T07:01:30.670487+0000","last_became_active":"2024-09-13T07:01:19.281907+0000","last_became_peered":"2024-09-13T07:01:19.281907+0000","last_unstale":"2024-09-13T07:01:30.670487+0000","last_undegraded":"2024-09-13T07:01:30.670487+0000","last_fullsized":"2024-09-13T07:01:30.670487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:43.514663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001950586,"stat_sum":{"num_bytes":6029312,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":68,"num_read_kb":51,"num_write":14,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"27'5","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.125106+0000","last_change":"2024-09-13T07:01:21.293288+0000","last_active":"2024-09-13T07:01:29.125106+0000","last_peered":"2024-09-13T07:01:29.125106+0000","last_clean":"2024-09-13T07:01:29.125106+0000","last_became_active":"2024-09-13T07:01:19.276445+0000","last_became_peered":"2024-09-13T07:01:19.276445+0000","last_unstale":"2024-09-13T07:01:29.125106+0000","last_undegraded":"2024-09-13T07:01:29.125106+0000","last_fullsized":"2024-09-13T07:01:29.125106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:16:37.731881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076373900000000004,"stat_sum":{"num_bytes":86,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10,"num_read_kb":4,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"32'13","reported_seq":72,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.603140+0000","last_change":"2024-09-13T07:01:21.292234+0000","last_active":"2024-09-13T07:01:30.603140+0000","last_peered":"2024-09-13T07:01:30.603140+0000","last_clean":"2024-09-13T07:01:30.603140+0000","last_became_active":"2024-09-13T07:01:19.276459+0000","last_became_peered":"2024-09-13T07:01:19.276459+0000","last_unstale":"2024-09-13T07:01:30.603140+0000","last_undegraded":"2024-09-13T07:01:30.603140+0000","last_fullsized":"2024-09-13T07:01:30.603140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:50:50.173262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050866599999999996,"stat_sum":{"num_bytes":983078,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":61,"num_read_kb":71,"num_write":11,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"32'24","reported_seq":114,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.666284+0000","last_change":"2024-09-13T07:01:21.305771+0000","last_active":"2024-09-13T07:01:30.666284+0000","last_peered":"2024-09-13T07:01:30.666284+0000","last_clean":"2024-09-13T07:01:30.666284+0000","last_became_active":"2024-09-13T07:01:19.282549+0000","last_became_peered":"2024-09-13T07:01:19.282549+0000","last_unstale":"2024-09-13T07:01:30.666284+0000","last_undegraded":"2024-09-13T07:01:30.666284+0000","last_fullsized":"2024-09-13T07:01:30.666284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:37:00.384884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083111000000000003,"stat_sum":{"num_bytes":6832169,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":5,"num_read":84,"num_read_kb":99,"num_write":19,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"28'24","reported_seq":209,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.597989+0000","last_change":"2024-09-13T07:01:21.297776+0000","last_active":"2024-09-13T07:01:30.597989+0000","last_peered":"2024-09-13T07:01:30.597989+0000","last_clean":"2024-09-13T07:01:30.597989+0000","last_became_active":"2024-09-13T07:01:19.275470+0000","last_became_peered":"2024-09-13T07:01:19.275470+0000","last_unstale":"2024-09-13T07:01:30.597989+0000","last_undegraded":"2024-09-13T07:01:30.597989+0000","last_fullsized":"2024-09-13T07:01:30.597989+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:15.860822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001799911,"stat_sum":{"num_bytes":11910247,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":180,"num_read_kb":248,"num_write":20,"num_write_kb":228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"32'39","reported_seq":141,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.675697+0000","last_change":"2024-09-13T07:01:21.300421+0000","last_active":"2024-09-13T07:01:30.675697+0000","last_peered":"2024-09-13T07:01:30.675697+0000","last_clean":"2024-09-13T07:01:30.675697+0000","last_became_active":"2024-09-13T07:01:19.280734+0000","last_became_peered":"2024-09-13T07:01:19.280734+0000","last_unstale":"2024-09-13T07:01:30.675697+0000","last_undegraded":"2024-09-13T07:01:30.675697+0000","last_fullsized":"2024-09-13T07:01:30.675697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:02.332240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055727200000000004,"stat_sum":{"num_bytes":8545265,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":163,"num_read_kb":117,"num_write":39,"num_write_kb":228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"32'32","reported_seq":191,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.616564+0000","last_change":"2024-09-13T07:01:21.290482+0000","last_active":"2024-09-13T07:01:30.616564+0000","last_peered":"2024-09-13T07:01:30.616564+0000","last_clean":"2024-09-13T07:01:30.616564+0000","last_became_active":"2024-09-13T07:01:19.282690+0000","last_became_peered":"2024-09-13T07:01:19.282690+0000","last_unstale":"2024-09-13T07:01:30.616564+0000","last_undegraded":"2024-09-13T07:01:30.616564+0000","last_fullsized":"2024-09-13T07:01:30.616564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:15:57.439254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082396300000000004,"stat_sum":{"num_bytes":407719,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":429,"num_read_kb":370,"num_write":24,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"31'33","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.336226+0000","last_change":"2024-09-13T07:01:21.306234+0000","last_active":"2024-09-13T07:01:29.336226+0000","last_peered":"2024-09-13T07:01:29.336226+0000","last_clean":"2024-09-13T07:01:29.336226+0000","last_became_active":"2024-09-13T07:01:19.280106+0000","last_became_peered":"2024-09-13T07:01:19.280106+0000","last_unstale":"2024-09-13T07:01:29.336226+0000","last_undegraded":"2024-09-13T07:01:29.336226+0000","last_fullsized":"2024-09-13T07:01:29.336226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:08:18.619986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001182647,"stat_sum":{"num_bytes":786432,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":30,"num_read_kb":112,"num_write":23,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"30'51","reported_seq":79,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.125102+0000","last_change":"2024-09-13T07:01:21.293012+0000","last_active":"2024-09-13T07:01:29.125102+0000","last_peered":"2024-09-13T07:01:29.125102+0000","last_clean":"2024-09-13T07:01:29.125102+0000","last_became_active":"2024-09-13T07:01:19.277380+0000","last_became_peered":"2024-09-13T07:01:19.277380+0000","last_unstale":"2024-09-13T07:01:29.125102+0000","last_undegraded":"2024-09-13T07:01:29.125102+0000","last_fullsized":"2024-09-13T07:01:29.125102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:05:57.870409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010463009999999999,"stat_sum":{"num_bytes":991184,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":155,"num_read_kb":163,"num_write":63,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'11","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.094575+0000","last_change":"2024-09-13T07:01:21.314353+0000","last_active":"2024-09-13T07:01:29.094575+0000","last_peered":"2024-09-13T07:01:29.094575+0000","last_clean":"2024-09-13T07:01:29.094575+0000","last_became_active":"2024-09-13T07:01:19.280203+0000","last_became_peered":"2024-09-13T07:01:19.280203+0000","last_unstale":"2024-09-13T07:01:29.094575+0000","last_undegraded":"2024-09-13T07:01:29.094575+0000","last_fullsized":"2024-09-13T07:01:29.094575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:38.598517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024699230000000002,"stat_sum":{"num_bytes":933888,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7,"num_read_kb":1,"num_write":8,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"32'105","reported_seq":153,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.534196+0000","last_change":"2024-09-13T07:01:21.298190+0000","last_active":"2024-09-13T07:01:30.534196+0000","last_peered":"2024-09-13T07:01:30.534196+0000","last_clean":"2024-09-13T07:01:30.534196+0000","last_became_active":"2024-09-13T07:01:19.276431+0000","last_became_peered":"2024-09-13T07:01:19.276431+0000","last_unstale":"2024-09-13T07:01:30.534196+0000","last_undegraded":"2024-09-13T07:01:30.534196+0000","last_fullsized":"2024-09-13T07:01:30.534196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:50:28.618968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001937825,"stat_sum":{"num_bytes":2506798,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":5,"num_read":441,"num_read_kb":4428,"num_write":154,"num_write_kb":7475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.068267+0000","last_change":"2024-09-13T07:01:11.221939+0000","last_active":"2024-09-13T07:01:29.068267+0000","last_peered":"2024-09-13T07:01:29.068267+0000","last_clean":"2024-09-13T07:01:29.068267+0000","last_became_active":"2024-09-13T07:01:09.212302+0000","last_became_peered":"2024-09-13T07:01:09.212302+0000","last_unstale":"2024-09-13T07:01:29.068267+0000","last_undegraded":"2024-09-13T07:01:29.068267+0000","last_fullsized":"2024-09-13T07:01:29.068267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:49.220240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'15","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.111450+0000","last_change":"2024-09-13T07:01:21.305189+0000","last_active":"2024-09-13T07:01:29.111450+0000","last_peered":"2024-09-13T07:01:29.111450+0000","last_clean":"2024-09-13T07:01:29.111450+0000","last_became_active":"2024-09-13T07:01:19.278317+0000","last_became_peered":"2024-09-13T07:01:19.278317+0000","last_unstale":"2024-09-13T07:01:29.111450+0000","last_undegraded":"2024-09-13T07:01:29.111450+0000","last_fullsized":"2024-09-13T07:01:29.111450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:46.588082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038364399999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6,"num_read_kb":0,"num_write":9,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.125276+0000","last_change":"2024-09-13T07:01:11.232122+0000","last_active":"2024-09-13T07:01:29.125276+0000","last_peered":"2024-09-13T07:01:29.125276+0000","last_clean":"2024-09-13T07:01:29.125276+0000","last_became_active":"2024-09-13T07:01:09.213058+0000","last_became_peered":"2024-09-13T07:01:09.213058+0000","last_unstale":"2024-09-13T07:01:29.125276+0000","last_undegraded":"2024-09-13T07:01:29.125276+0000","last_fullsized":"2024-09-13T07:01:29.125276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:10.839167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067353900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"30'35","reported_seq":130,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.346526+0000","last_change":"2024-09-13T07:01:21.291928+0000","last_active":"2024-09-13T07:01:30.346526+0000","last_peered":"2024-09-13T07:01:30.346526+0000","last_clean":"2024-09-13T07:01:30.346526+0000","last_became_active":"2024-09-13T07:01:19.286391+0000","last_became_peered":"2024-09-13T07:01:19.286391+0000","last_unstale":"2024-09-13T07:01:30.346526+0000","last_undegraded":"2024-09-13T07:01:30.346526+0000","last_fullsized":"2024-09-13T07:01:30.346526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:58:51.082865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016908909999999999,"stat_sum":{"num_bytes":17000507,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":7,"num_read":99,"num_read_kb":262,"num_write":21,"num_write_kb":200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.095198+0000","last_change":"2024-09-13T07:01:11.860064+0000","last_active":"2024-09-13T07:01:29.095198+0000","last_peered":"2024-09-13T07:01:29.095198+0000","last_clean":"2024-09-13T07:01:29.095198+0000","last_became_active":"2024-09-13T07:01:09.521457+0000","last_became_peered":"2024-09-13T07:01:09.521457+0000","last_unstale":"2024-09-13T07:01:29.095198+0000","last_undegraded":"2024-09-13T07:01:29.095198+0000","last_fullsized":"2024-09-13T07:01:29.095198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:39.434445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"32'66","reported_seq":108,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.502901+0000","last_change":"2024-09-13T07:01:21.300698+0000","last_active":"2024-09-13T07:01:30.502901+0000","last_peered":"2024-09-13T07:01:30.502901+0000","last_clean":"2024-09-13T07:01:30.502901+0000","last_became_active":"2024-09-13T07:01:19.816695+0000","last_became_peered":"2024-09-13T07:01:19.816695+0000","last_unstale":"2024-09-13T07:01:30.502901+0000","last_undegraded":"2024-09-13T07:01:30.502901+0000","last_fullsized":"2024-09-13T07:01:30.502901+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:59:39.553337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085579199999999997,"stat_sum":{"num_bytes":10051758,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":5,"num_read":262,"num_read_kb":186,"num_write":93,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.123545+0000","last_change":"2024-09-13T07:01:11.232131+0000","last_active":"2024-09-13T07:01:29.123545+0000","last_peered":"2024-09-13T07:01:29.123545+0000","last_clean":"2024-09-13T07:01:29.123545+0000","last_became_active":"2024-09-13T07:01:09.523241+0000","last_became_peered":"2024-09-13T07:01:09.523241+0000","last_unstale":"2024-09-13T07:01:29.123545+0000","last_undegraded":"2024-09-13T07:01:29.123545+0000","last_fullsized":"2024-09-13T07:01:29.123545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:58:22.184544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'12","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.591314+0000","last_change":"2024-09-13T07:01:21.293697+0000","last_active":"2024-09-13T07:01:30.591314+0000","last_peered":"2024-09-13T07:01:30.591314+0000","last_clean":"2024-09-13T07:01:30.591314+0000","last_became_active":"2024-09-13T07:01:19.279719+0000","last_became_peered":"2024-09-13T07:01:19.279719+0000","last_unstale":"2024-09-13T07:01:30.591314+0000","last_undegraded":"2024-09-13T07:01:30.591314+0000","last_fullsized":"2024-09-13T07:01:30.591314+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:05:06.593984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085976499999999997,"stat_sum":{"num_bytes":1741414,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":29,"num_read_kb":27,"num_write":8,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"27'11","reported_seq":549,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.073626+0000","last_change":"2024-09-13T07:01:21.297796+0000","last_active":"2024-09-13T07:01:29.073626+0000","last_peered":"2024-09-13T07:01:29.073626+0000","last_clean":"2024-09-13T07:01:29.073626+0000","last_became_active":"2024-09-13T07:01:19.277941+0000","last_became_peered":"2024-09-13T07:01:19.277941+0000","last_unstale":"2024-09-13T07:01:29.073626+0000","last_undegraded":"2024-09-13T07:01:29.073626+0000","last_fullsized":"2024-09-13T07:01:29.073626+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:29:11.023237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017265959999999999,"stat_sum":{"num_bytes":1626247,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":3,"num_read":518,"num_read_kb":2171,"num_write":8,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.124691+0000","last_change":"2024-09-13T07:01:11.228726+0000","last_active":"2024-09-13T07:01:29.124691+0000","last_peered":"2024-09-13T07:01:29.124691+0000","last_clean":"2024-09-13T07:01:29.124691+0000","last_became_active":"2024-09-13T07:01:09.211518+0000","last_became_peered":"2024-09-13T07:01:09.211518+0000","last_unstale":"2024-09-13T07:01:29.124691+0000","last_undegraded":"2024-09-13T07:01:29.124691+0000","last_fullsized":"2024-09-13T07:01:29.124691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:21.453172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"32'48","reported_seq":200,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.672566+0000","last_change":"2024-09-13T07:01:21.300501+0000","last_active":"2024-09-13T07:01:30.672566+0000","last_peered":"2024-09-13T07:01:30.672566+0000","last_clean":"2024-09-13T07:01:30.672566+0000","last_became_active":"2024-09-13T07:01:19.816733+0000","last_became_peered":"2024-09-13T07:01:19.816733+0000","last_unstale":"2024-09-13T07:01:30.672566+0000","last_undegraded":"2024-09-13T07:01:30.672566+0000","last_fullsized":"2024-09-13T07:01:30.672566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:37.480143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065804299999999997,"stat_sum":{"num_bytes":17956880,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":164,"num_read_kb":128,"num_write":59,"num_write_kb":6761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.069919+0000","last_change":"2024-09-13T07:01:11.222462+0000","last_active":"2024-09-13T07:01:29.069919+0000","last_peered":"2024-09-13T07:01:29.069919+0000","last_clean":"2024-09-13T07:01:29.069919+0000","last_became_active":"2024-09-13T07:01:09.523133+0000","last_became_peered":"2024-09-13T07:01:09.523133+0000","last_unstale":"2024-09-13T07:01:29.069919+0000","last_undegraded":"2024-09-13T07:01:29.069919+0000","last_fullsized":"2024-09-13T07:01:29.069919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:12:38.585984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074093200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"30'10","reported_seq":97,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.110808+0000","last_change":"2024-09-13T07:01:21.305184+0000","last_active":"2024-09-13T07:01:29.110808+0000","last_peered":"2024-09-13T07:01:29.110808+0000","last_clean":"2024-09-13T07:01:29.110808+0000","last_became_active":"2024-09-13T07:01:19.286557+0000","last_became_peered":"2024-09-13T07:01:19.286557+0000","last_unstale":"2024-09-13T07:01:29.110808+0000","last_undegraded":"2024-09-13T07:01:29.110808+0000","last_fullsized":"2024-09-13T07:01:29.110808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:33:25.595462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044811299999999998,"stat_sum":{"num_bytes":12742703,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":71,"num_read_kb":95,"num_write":6,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.094242+0000","last_change":"2024-09-13T07:01:11.859918+0000","last_active":"2024-09-13T07:01:29.094242+0000","last_peered":"2024-09-13T07:01:29.094242+0000","last_clean":"2024-09-13T07:01:29.094242+0000","last_became_active":"2024-09-13T07:01:09.210971+0000","last_became_peered":"2024-09-13T07:01:09.210971+0000","last_unstale":"2024-09-13T07:01:29.094242+0000","last_undegraded":"2024-09-13T07:01:29.094242+0000","last_fullsized":"2024-09-13T07:01:29.094242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:30.775691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"30'76","reported_seq":610,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.612609+0000","last_change":"2024-09-13T07:01:21.290419+0000","last_active":"2024-09-13T07:01:30.612609+0000","last_peered":"2024-09-13T07:01:30.612609+0000","last_clean":"2024-09-13T07:01:30.612609+0000","last_became_active":"2024-09-13T07:01:19.282935+0000","last_became_peered":"2024-09-13T07:01:19.282935+0000","last_unstale":"2024-09-13T07:01:30.612609+0000","last_undegraded":"2024-09-13T07:01:30.612609+0000","last_fullsized":"2024-09-13T07:01:30.612609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:36:53.773908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064294199999999997,"stat_sum":{"num_bytes":409616,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":1538,"num_read_kb":1328,"num_write":71,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.124854+0000","last_change":"2024-09-13T07:01:11.227425+0000","last_active":"2024-09-13T07:01:29.124854+0000","last_peered":"2024-09-13T07:01:29.124854+0000","last_clean":"2024-09-13T07:01:29.124854+0000","last_became_active":"2024-09-13T07:01:09.207226+0000","last_became_peered":"2024-09-13T07:01:09.207226+0000","last_unstale":"2024-09-13T07:01:29.124854+0000","last_undegraded":"2024-09-13T07:01:29.124854+0000","last_fullsized":"2024-09-13T07:01:29.124854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:12.911230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.094428+0000","last_change":"2024-09-13T07:01:02.162365+0000","last_active":"2024-09-13T07:01:29.094428+0000","last_peered":"2024-09-13T07:01:29.094428+0000","last_clean":"2024-09-13T07:01:29.094428+0000","last_became_active":"2024-09-13T07:01:02.161407+0000","last_became_peered":"2024-09-13T07:01:02.161407+0000","last_unstale":"2024-09-13T07:01:29.094428+0000","last_undegraded":"2024-09-13T07:01:29.094428+0000","last_fullsized":"2024-09-13T07:01:29.094428+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:17.998339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.123912+0000","last_change":"2024-09-13T07:01:21.291091+0000","last_active":"2024-09-13T07:01:29.123912+0000","last_peered":"2024-09-13T07:01:29.123912+0000","last_clean":"2024-09-13T07:01:29.123912+0000","last_became_active":"2024-09-13T07:01:19.280796+0000","last_became_peered":"2024-09-13T07:01:19.280796+0000","last_unstale":"2024-09-13T07:01:29.123912+0000","last_undegraded":"2024-09-13T07:01:29.123912+0000","last_fullsized":"2024-09-13T07:01:29.123912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:20:10.332964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001264996,"stat_sum":{"num_bytes":933904,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":17,"num_read_kb":18,"num_write":7,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"27'4","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.073423+0000","last_change":"2024-09-13T07:01:21.298330+0000","last_active":"2024-09-13T07:01:29.073423+0000","last_peered":"2024-09-13T07:01:29.073423+0000","last_clean":"2024-09-13T07:01:29.073423+0000","last_became_active":"2024-09-13T07:01:19.816011+0000","last_became_peered":"2024-09-13T07:01:19.816011+0000","last_unstale":"2024-09-13T07:01:29.073423+0000","last_undegraded":"2024-09-13T07:01:29.073423+0000","last_fullsized":"2024-09-13T07:01:29.073423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:10:08.758669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022509729999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"32'50","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.467782+0000","last_change":"2024-09-13T07:01:22.025365+0000","last_active":"2024-09-13T07:01:30.467782+0000","last_peered":"2024-09-13T07:01:30.467782+0000","last_clean":"2024-09-13T07:01:30.467782+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:01:30.467782+0000","last_undegraded":"2024-09-13T07:01:30.467782+0000","last_fullsized":"2024-09-13T07:01:30.467782+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:11.537644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064478199999999999,"stat_sum":{"num_bytes":948634,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":3,"num_read":39,"num_read_kb":77,"num_write":44,"num_write_kb":10678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"30'11","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.423553+0000","last_change":"2024-09-13T07:01:21.291637+0000","last_active":"2024-09-13T07:01:30.423553+0000","last_peered":"2024-09-13T07:01:30.423553+0000","last_clean":"2024-09-13T07:01:30.423553+0000","last_became_active":"2024-09-13T07:01:19.282852+0000","last_became_peered":"2024-09-13T07:01:19.282852+0000","last_unstale":"2024-09-13T07:01:30.423553+0000","last_undegraded":"2024-09-13T07:01:30.423553+0000","last_fullsized":"2024-09-13T07:01:30.423553+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:51.391035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001533932,"stat_sum":{"num_bytes":1679641,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":22,"num_read_kb":30,"num_write":7,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"32'95","reported_seq":463,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.664365+0000","last_change":"2024-09-13T07:01:21.292633+0000","last_active":"2024-09-13T07:01:30.664365+0000","last_peered":"2024-09-13T07:01:30.664365+0000","last_clean":"2024-09-13T07:01:30.664365+0000","last_became_active":"2024-09-13T07:01:19.279813+0000","last_became_peered":"2024-09-13T07:01:19.279813+0000","last_unstale":"2024-09-13T07:01:30.664365+0000","last_undegraded":"2024-09-13T07:01:30.664365+0000","last_fullsized":"2024-09-13T07:01:30.664365+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:39.697691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072600099999999999,"stat_sum":{"num_bytes":22367453,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":806,"num_read_kb":693,"num_write":115,"num_write_kb":7105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"32'55","reported_seq":308,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.672462+0000","last_change":"2024-09-13T07:01:21.313310+0000","last_active":"2024-09-13T07:01:30.672462+0000","last_peered":"2024-09-13T07:01:30.672462+0000","last_clean":"2024-09-13T07:01:30.672462+0000","last_became_active":"2024-09-13T07:01:19.282520+0000","last_became_peered":"2024-09-13T07:01:19.282520+0000","last_unstale":"2024-09-13T07:01:30.672462+0000","last_undegraded":"2024-09-13T07:01:30.672462+0000","last_fullsized":"2024-09-13T07:01:30.672462+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:41:44.801487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018248170000000001,"stat_sum":{"num_bytes":19251200,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":483,"num_read_kb":419,"num_write":45,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"32'25","reported_seq":112,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.662849+0000","last_change":"2024-09-13T07:01:21.313674+0000","last_active":"2024-09-13T07:01:30.662849+0000","last_peered":"2024-09-13T07:01:30.662849+0000","last_clean":"2024-09-13T07:01:30.662849+0000","last_became_active":"2024-09-13T07:01:19.281215+0000","last_became_peered":"2024-09-13T07:01:19.281215+0000","last_unstale":"2024-09-13T07:01:30.662849+0000","last_undegraded":"2024-09-13T07:01:30.662849+0000","last_fullsized":"2024-09-13T07:01:30.662849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:22.161596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019878309999999998,"stat_sum":{"num_bytes":6832128,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":80,"num_read_kb":51,"num_write":20,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"30'14","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.363924+0000","last_change":"2024-09-13T07:01:21.314048+0000","last_active":"2024-09-13T07:01:30.363924+0000","last_peered":"2024-09-13T07:01:30.363924+0000","last_clean":"2024-09-13T07:01:30.363924+0000","last_became_active":"2024-09-13T07:01:19.281169+0000","last_became_peered":"2024-09-13T07:01:19.281169+0000","last_unstale":"2024-09-13T07:01:30.363924+0000","last_undegraded":"2024-09-13T07:01:30.363924+0000","last_fullsized":"2024-09-13T07:01:30.363924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:49:46.525500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023438180000000001,"stat_sum":{"num_bytes":4718592,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":74,"num_read_kb":7,"num_write":9,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"27'11","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.067959+0000","last_change":"2024-09-13T07:01:22.025282+0000","last_active":"2024-09-13T07:01:29.067959+0000","last_peered":"2024-09-13T07:01:29.067959+0000","last_clean":"2024-09-13T07:01:29.067959+0000","last_became_active":"2024-09-13T07:01:19.283407+0000","last_became_peered":"2024-09-13T07:01:19.283407+0000","last_unstale":"2024-09-13T07:01:29.067959+0000","last_undegraded":"2024-09-13T07:01:29.067959+0000","last_fullsized":"2024-09-13T07:01:29.067959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:33:56.343722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061928799999999996,"stat_sum":{"num_bytes":344064,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":8,"num_read_kb":19,"num_write":8,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"30'12","reported_seq":72,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.094404+0000","last_change":"2024-09-13T07:01:21.313825+0000","last_active":"2024-09-13T07:01:29.094404+0000","last_peered":"2024-09-13T07:01:29.094404+0000","last_clean":"2024-09-13T07:01:29.094404+0000","last_became_active":"2024-09-13T07:01:19.279714+0000","last_became_peered":"2024-09-13T07:01:19.279714+0000","last_unstale":"2024-09-13T07:01:29.094404+0000","last_undegraded":"2024-09-13T07:01:29.094404+0000","last_fullsized":"2024-09-13T07:01:29.094404+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:06:09.717433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002154132,"stat_sum":{"num_bytes":7685649,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":4,"num_read":43,"num_read_kb":17,"num_write":8,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"31'92","reported_seq":184,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.428310+0000","last_change":"2024-09-13T07:01:21.305471+0000","last_active":"2024-09-13T07:01:30.428310+0000","last_peered":"2024-09-13T07:01:30.428310+0000","last_clean":"2024-09-13T07:01:30.428310+0000","last_became_active":"2024-09-13T07:01:19.284230+0000","last_became_peered":"2024-09-13T07:01:19.284230+0000","last_unstale":"2024-09-13T07:01:30.428310+0000","last_undegraded":"2024-09-13T07:01:30.428310+0000","last_fullsized":"2024-09-13T07:01:30.428310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:47.692777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060210699999999995,"stat_sum":{"num_bytes":11563403,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":388,"num_read_kb":360,"num_write":121,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"28'77","reported_seq":103,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.094847+0000","last_change":"2024-09-13T07:01:21.313226+0000","last_active":"2024-09-13T07:01:29.094847+0000","last_peered":"2024-09-13T07:01:29.094847+0000","last_clean":"2024-09-13T07:01:29.094847+0000","last_became_active":"2024-09-13T07:01:19.279419+0000","last_became_peered":"2024-09-13T07:01:19.279419+0000","last_unstale":"2024-09-13T07:01:29.094847+0000","last_undegraded":"2024-09-13T07:01:29.094847+0000","last_fullsized":"2024-09-13T07:01:29.094847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:46:29.817757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001803165,"stat_sum":{"num_bytes":41,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":287,"num_read_kb":212,"num_write":132,"num_write_kb":7164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":197860452,"num_objects":211,"num_object_clones":46,"num_object_copies":422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":211,"num_whiteouts":125,"num_read":6715,"num_read_kb":11822,"num_write":1196,"num_write_kb":41905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6160384,"data_stored":5835067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1116,"ondisk_log_size":1116,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34416,"kb_used_data":3304,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337424,"statfs":{"total":96636764160,"available":96601522176,"internally_reserved":0,"allocated":3383296,"data_stored":3128926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":372736,"data_stored":166297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":708608,"data_stored":431495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29692,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342148,"statfs":{"total":96636764160,"available":96606359552,"internally_reserved":0,"allocated":2740224,"data_stored":2485383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":733184,"data_stored":471928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":348160,"data_stored":135213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":487424,"data_stored":255641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36608,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335232,"statfs":{"total":96636764160,"available":96599277568,"internally_reserved":0,"allocated":1277952,"data_stored":1022121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.643}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":296586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":120494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":336843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2404352,"data_stored":2350222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":296349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":31130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2453504,"data_stored":2403397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:01:31.762 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:01:31.770 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:01:31.773 INFO:teuthology.orchestra.run.smithi118.stdout:468 trunc from 0xe4a1cc8 to 0x3e6fe56 2024-09-13T07:01:31.813 INFO:teuthology.orchestra.run.smithi118.stdout:469 read 0x2b825f6 thru 0x2b8aaf8 (0x8503 bytes) 2024-09-13T07:01:31.814 INFO:teuthology.orchestra.run.smithi118.stdout:470 punch from 0x369d02d to 0x36a957e, (0xc551 bytes) 2024-09-13T07:01:31.815 INFO:teuthology.orchestra.run.smithi118.stdout:472 write 0xb1dabc thru 0xb2cc76 (0xf1bb bytes) 2024-09-13T07:01:31.817 INFO:teuthology.orchestra.run.smithi118.stdout:473 read 0x140021 thru 0x149793 (0x9773 bytes) 2024-09-13T07:01:31.835 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:31.832+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:31.846 INFO:teuthology.orchestra.run.smithi118.stdout:475 write 0xbda128f thru 0xbdafe28 (0xeb9a bytes) 2024-09-13T07:01:31.852 INFO:teuthology.orchestra.run.smithi118.stdout:476 clone 7 order 20 su 16384 sc 7 2024-09-13T07:01:31.897 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:31.897 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:31.910 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:01:31.950 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:31.948+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:32.037 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:32.032+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:32.057 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:32.057 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:32.067 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:01:32.074 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:32.077 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:01:32.088 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-13T07:01:31.358868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":198450839,"num_objects":215,"num_object_clones":46,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":125,"num_read":6831,"num_read_kb":11922,"num_write":1357,"num_write_kb":44323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1236,"ondisk_log_size":1236,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254792,"kb_used_data":9816,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754719928,"statfs":{"total":773094113280,"available":772833206272,"internally_reserved":0,"allocated":10051584,"data_stored":8097004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":250596920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":178453971,"num_objects":55,"num_object_clones":18,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":12,"num_read":4753,"num_read_kb":9426,"num_write":768,"num_write_kb":40375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033900"},"pg_stats":[{"pgid":"3.1f","version":"32'30","reported_seq":185,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.667027+0000","last_change":"2024-09-13T07:01:21.306378+0000","last_active":"2024-09-13T07:01:30.667027+0000","last_peered":"2024-09-13T07:01:30.667027+0000","last_clean":"2024-09-13T07:01:30.667027+0000","last_became_active":"2024-09-13T07:01:19.282082+0000","last_became_peered":"2024-09-13T07:01:19.282082+0000","last_unstale":"2024-09-13T07:01:30.667027+0000","last_undegraded":"2024-09-13T07:01:30.667027+0000","last_fullsized":"2024-09-13T07:01:30.667027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:41.783324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013321120000000001,"stat_sum":{"num_bytes":20080440,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":151,"num_read_kb":58,"num_write":25,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"32'18","reported_seq":99,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.670487+0000","last_change":"2024-09-13T07:01:21.291845+0000","last_active":"2024-09-13T07:01:30.670487+0000","last_peered":"2024-09-13T07:01:30.670487+0000","last_clean":"2024-09-13T07:01:30.670487+0000","last_became_active":"2024-09-13T07:01:19.281907+0000","last_became_peered":"2024-09-13T07:01:19.281907+0000","last_unstale":"2024-09-13T07:01:30.670487+0000","last_undegraded":"2024-09-13T07:01:30.670487+0000","last_fullsized":"2024-09-13T07:01:30.670487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:43.514663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001950586,"stat_sum":{"num_bytes":6029312,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":68,"num_read_kb":51,"num_write":14,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"27'5","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.125106+0000","last_change":"2024-09-13T07:01:21.293288+0000","last_active":"2024-09-13T07:01:29.125106+0000","last_peered":"2024-09-13T07:01:29.125106+0000","last_clean":"2024-09-13T07:01:29.125106+0000","last_became_active":"2024-09-13T07:01:19.276445+0000","last_became_peered":"2024-09-13T07:01:19.276445+0000","last_unstale":"2024-09-13T07:01:29.125106+0000","last_undegraded":"2024-09-13T07:01:29.125106+0000","last_fullsized":"2024-09-13T07:01:29.125106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:16:37.731881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076373900000000004,"stat_sum":{"num_bytes":86,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10,"num_read_kb":4,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"32'13","reported_seq":72,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.603140+0000","last_change":"2024-09-13T07:01:21.292234+0000","last_active":"2024-09-13T07:01:30.603140+0000","last_peered":"2024-09-13T07:01:30.603140+0000","last_clean":"2024-09-13T07:01:30.603140+0000","last_became_active":"2024-09-13T07:01:19.276459+0000","last_became_peered":"2024-09-13T07:01:19.276459+0000","last_unstale":"2024-09-13T07:01:30.603140+0000","last_undegraded":"2024-09-13T07:01:30.603140+0000","last_fullsized":"2024-09-13T07:01:30.603140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:50:50.173262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050866599999999996,"stat_sum":{"num_bytes":983078,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":61,"num_read_kb":71,"num_write":11,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"32'24","reported_seq":114,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.666284+0000","last_change":"2024-09-13T07:01:21.305771+0000","last_active":"2024-09-13T07:01:30.666284+0000","last_peered":"2024-09-13T07:01:30.666284+0000","last_clean":"2024-09-13T07:01:30.666284+0000","last_became_active":"2024-09-13T07:01:19.282549+0000","last_became_peered":"2024-09-13T07:01:19.282549+0000","last_unstale":"2024-09-13T07:01:30.666284+0000","last_undegraded":"2024-09-13T07:01:30.666284+0000","last_fullsized":"2024-09-13T07:01:30.666284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:37:00.384884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083111000000000003,"stat_sum":{"num_bytes":6832169,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":5,"num_read":84,"num_read_kb":99,"num_write":19,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"28'24","reported_seq":209,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.597989+0000","last_change":"2024-09-13T07:01:21.297776+0000","last_active":"2024-09-13T07:01:30.597989+0000","last_peered":"2024-09-13T07:01:30.597989+0000","last_clean":"2024-09-13T07:01:30.597989+0000","last_became_active":"2024-09-13T07:01:19.275470+0000","last_became_peered":"2024-09-13T07:01:19.275470+0000","last_unstale":"2024-09-13T07:01:30.597989+0000","last_undegraded":"2024-09-13T07:01:30.597989+0000","last_fullsized":"2024-09-13T07:01:30.597989+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:15.860822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001799911,"stat_sum":{"num_bytes":11910247,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":180,"num_read_kb":248,"num_write":20,"num_write_kb":228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"32'39","reported_seq":141,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.675697+0000","last_change":"2024-09-13T07:01:21.300421+0000","last_active":"2024-09-13T07:01:30.675697+0000","last_peered":"2024-09-13T07:01:30.675697+0000","last_clean":"2024-09-13T07:01:30.675697+0000","last_became_active":"2024-09-13T07:01:19.280734+0000","last_became_peered":"2024-09-13T07:01:19.280734+0000","last_unstale":"2024-09-13T07:01:30.675697+0000","last_undegraded":"2024-09-13T07:01:30.675697+0000","last_fullsized":"2024-09-13T07:01:30.675697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:02.332240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055727200000000004,"stat_sum":{"num_bytes":8545265,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":163,"num_read_kb":117,"num_write":39,"num_write_kb":228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"32'32","reported_seq":191,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.616564+0000","last_change":"2024-09-13T07:01:21.290482+0000","last_active":"2024-09-13T07:01:30.616564+0000","last_peered":"2024-09-13T07:01:30.616564+0000","last_clean":"2024-09-13T07:01:30.616564+0000","last_became_active":"2024-09-13T07:01:19.282690+0000","last_became_peered":"2024-09-13T07:01:19.282690+0000","last_unstale":"2024-09-13T07:01:30.616564+0000","last_undegraded":"2024-09-13T07:01:30.616564+0000","last_fullsized":"2024-09-13T07:01:30.616564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:15:57.439254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082396300000000004,"stat_sum":{"num_bytes":407719,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":429,"num_read_kb":370,"num_write":24,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"31'33","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.336226+0000","last_change":"2024-09-13T07:01:21.306234+0000","last_active":"2024-09-13T07:01:29.336226+0000","last_peered":"2024-09-13T07:01:29.336226+0000","last_clean":"2024-09-13T07:01:29.336226+0000","last_became_active":"2024-09-13T07:01:19.280106+0000","last_became_peered":"2024-09-13T07:01:19.280106+0000","last_unstale":"2024-09-13T07:01:29.336226+0000","last_undegraded":"2024-09-13T07:01:29.336226+0000","last_fullsized":"2024-09-13T07:01:29.336226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:08:18.619986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001182647,"stat_sum":{"num_bytes":786432,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":30,"num_read_kb":112,"num_write":23,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"30'51","reported_seq":79,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.125102+0000","last_change":"2024-09-13T07:01:21.293012+0000","last_active":"2024-09-13T07:01:29.125102+0000","last_peered":"2024-09-13T07:01:29.125102+0000","last_clean":"2024-09-13T07:01:29.125102+0000","last_became_active":"2024-09-13T07:01:19.277380+0000","last_became_peered":"2024-09-13T07:01:19.277380+0000","last_unstale":"2024-09-13T07:01:29.125102+0000","last_undegraded":"2024-09-13T07:01:29.125102+0000","last_fullsized":"2024-09-13T07:01:29.125102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:05:57.870409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010463009999999999,"stat_sum":{"num_bytes":991184,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":155,"num_read_kb":163,"num_write":63,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'11","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.094575+0000","last_change":"2024-09-13T07:01:21.314353+0000","last_active":"2024-09-13T07:01:29.094575+0000","last_peered":"2024-09-13T07:01:29.094575+0000","last_clean":"2024-09-13T07:01:29.094575+0000","last_became_active":"2024-09-13T07:01:19.280203+0000","last_became_peered":"2024-09-13T07:01:19.280203+0000","last_unstale":"2024-09-13T07:01:29.094575+0000","last_undegraded":"2024-09-13T07:01:29.094575+0000","last_fullsized":"2024-09-13T07:01:29.094575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:38.598517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024699230000000002,"stat_sum":{"num_bytes":933888,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7,"num_read_kb":1,"num_write":8,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"32'105","reported_seq":153,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.534196+0000","last_change":"2024-09-13T07:01:21.298190+0000","last_active":"2024-09-13T07:01:30.534196+0000","last_peered":"2024-09-13T07:01:30.534196+0000","last_clean":"2024-09-13T07:01:30.534196+0000","last_became_active":"2024-09-13T07:01:19.276431+0000","last_became_peered":"2024-09-13T07:01:19.276431+0000","last_unstale":"2024-09-13T07:01:30.534196+0000","last_undegraded":"2024-09-13T07:01:30.534196+0000","last_fullsized":"2024-09-13T07:01:30.534196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:50:28.618968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001937825,"stat_sum":{"num_bytes":2506798,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":5,"num_read":441,"num_read_kb":4428,"num_write":154,"num_write_kb":7475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.068267+0000","last_change":"2024-09-13T07:01:11.221939+0000","last_active":"2024-09-13T07:01:29.068267+0000","last_peered":"2024-09-13T07:01:29.068267+0000","last_clean":"2024-09-13T07:01:29.068267+0000","last_became_active":"2024-09-13T07:01:09.212302+0000","last_became_peered":"2024-09-13T07:01:09.212302+0000","last_unstale":"2024-09-13T07:01:29.068267+0000","last_undegraded":"2024-09-13T07:01:29.068267+0000","last_fullsized":"2024-09-13T07:01:29.068267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:49.220240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'15","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.111450+0000","last_change":"2024-09-13T07:01:21.305189+0000","last_active":"2024-09-13T07:01:29.111450+0000","last_peered":"2024-09-13T07:01:29.111450+0000","last_clean":"2024-09-13T07:01:29.111450+0000","last_became_active":"2024-09-13T07:01:19.278317+0000","last_became_peered":"2024-09-13T07:01:19.278317+0000","last_unstale":"2024-09-13T07:01:29.111450+0000","last_undegraded":"2024-09-13T07:01:29.111450+0000","last_fullsized":"2024-09-13T07:01:29.111450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:46.588082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038364399999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6,"num_read_kb":0,"num_write":9,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.125276+0000","last_change":"2024-09-13T07:01:11.232122+0000","last_active":"2024-09-13T07:01:29.125276+0000","last_peered":"2024-09-13T07:01:29.125276+0000","last_clean":"2024-09-13T07:01:29.125276+0000","last_became_active":"2024-09-13T07:01:09.213058+0000","last_became_peered":"2024-09-13T07:01:09.213058+0000","last_unstale":"2024-09-13T07:01:29.125276+0000","last_undegraded":"2024-09-13T07:01:29.125276+0000","last_fullsized":"2024-09-13T07:01:29.125276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:10.839167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067353900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"30'35","reported_seq":130,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.346526+0000","last_change":"2024-09-13T07:01:21.291928+0000","last_active":"2024-09-13T07:01:30.346526+0000","last_peered":"2024-09-13T07:01:30.346526+0000","last_clean":"2024-09-13T07:01:30.346526+0000","last_became_active":"2024-09-13T07:01:19.286391+0000","last_became_peered":"2024-09-13T07:01:19.286391+0000","last_unstale":"2024-09-13T07:01:30.346526+0000","last_undegraded":"2024-09-13T07:01:30.346526+0000","last_fullsized":"2024-09-13T07:01:30.346526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:58:51.082865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016908909999999999,"stat_sum":{"num_bytes":17000507,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":7,"num_read":99,"num_read_kb":262,"num_write":21,"num_write_kb":200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.095198+0000","last_change":"2024-09-13T07:01:11.860064+0000","last_active":"2024-09-13T07:01:29.095198+0000","last_peered":"2024-09-13T07:01:29.095198+0000","last_clean":"2024-09-13T07:01:29.095198+0000","last_became_active":"2024-09-13T07:01:09.521457+0000","last_became_peered":"2024-09-13T07:01:09.521457+0000","last_unstale":"2024-09-13T07:01:29.095198+0000","last_undegraded":"2024-09-13T07:01:29.095198+0000","last_fullsized":"2024-09-13T07:01:29.095198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:39.434445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"32'66","reported_seq":108,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.502901+0000","last_change":"2024-09-13T07:01:21.300698+0000","last_active":"2024-09-13T07:01:30.502901+0000","last_peered":"2024-09-13T07:01:30.502901+0000","last_clean":"2024-09-13T07:01:30.502901+0000","last_became_active":"2024-09-13T07:01:19.816695+0000","last_became_peered":"2024-09-13T07:01:19.816695+0000","last_unstale":"2024-09-13T07:01:30.502901+0000","last_undegraded":"2024-09-13T07:01:30.502901+0000","last_fullsized":"2024-09-13T07:01:30.502901+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:59:39.553337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085579199999999997,"stat_sum":{"num_bytes":10051758,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":5,"num_read":262,"num_read_kb":186,"num_write":93,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.123545+0000","last_change":"2024-09-13T07:01:11.232131+0000","last_active":"2024-09-13T07:01:29.123545+0000","last_peered":"2024-09-13T07:01:29.123545+0000","last_clean":"2024-09-13T07:01:29.123545+0000","last_became_active":"2024-09-13T07:01:09.523241+0000","last_became_peered":"2024-09-13T07:01:09.523241+0000","last_unstale":"2024-09-13T07:01:29.123545+0000","last_undegraded":"2024-09-13T07:01:29.123545+0000","last_fullsized":"2024-09-13T07:01:29.123545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:58:22.184544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'12","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.591314+0000","last_change":"2024-09-13T07:01:21.293697+0000","last_active":"2024-09-13T07:01:30.591314+0000","last_peered":"2024-09-13T07:01:30.591314+0000","last_clean":"2024-09-13T07:01:30.591314+0000","last_became_active":"2024-09-13T07:01:19.279719+0000","last_became_peered":"2024-09-13T07:01:19.279719+0000","last_unstale":"2024-09-13T07:01:30.591314+0000","last_undegraded":"2024-09-13T07:01:30.591314+0000","last_fullsized":"2024-09-13T07:01:30.591314+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:05:06.593984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085976499999999997,"stat_sum":{"num_bytes":1741414,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":29,"num_read_kb":27,"num_write":8,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"27'11","reported_seq":549,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.073626+0000","last_change":"2024-09-13T07:01:21.297796+0000","last_active":"2024-09-13T07:01:29.073626+0000","last_peered":"2024-09-13T07:01:29.073626+0000","last_clean":"2024-09-13T07:01:29.073626+0000","last_became_active":"2024-09-13T07:01:19.277941+0000","last_became_peered":"2024-09-13T07:01:19.277941+0000","last_unstale":"2024-09-13T07:01:29.073626+0000","last_undegraded":"2024-09-13T07:01:29.073626+0000","last_fullsized":"2024-09-13T07:01:29.073626+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:29:11.023237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017265959999999999,"stat_sum":{"num_bytes":1626247,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":3,"num_read":518,"num_read_kb":2171,"num_write":8,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.124691+0000","last_change":"2024-09-13T07:01:11.228726+0000","last_active":"2024-09-13T07:01:29.124691+0000","last_peered":"2024-09-13T07:01:29.124691+0000","last_clean":"2024-09-13T07:01:29.124691+0000","last_became_active":"2024-09-13T07:01:09.211518+0000","last_became_peered":"2024-09-13T07:01:09.211518+0000","last_unstale":"2024-09-13T07:01:29.124691+0000","last_undegraded":"2024-09-13T07:01:29.124691+0000","last_fullsized":"2024-09-13T07:01:29.124691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:21.453172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"32'48","reported_seq":200,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.672566+0000","last_change":"2024-09-13T07:01:21.300501+0000","last_active":"2024-09-13T07:01:30.672566+0000","last_peered":"2024-09-13T07:01:30.672566+0000","last_clean":"2024-09-13T07:01:30.672566+0000","last_became_active":"2024-09-13T07:01:19.816733+0000","last_became_peered":"2024-09-13T07:01:19.816733+0000","last_unstale":"2024-09-13T07:01:30.672566+0000","last_undegraded":"2024-09-13T07:01:30.672566+0000","last_fullsized":"2024-09-13T07:01:30.672566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:37.480143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065804299999999997,"stat_sum":{"num_bytes":17956880,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":164,"num_read_kb":128,"num_write":59,"num_write_kb":6761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.069919+0000","last_change":"2024-09-13T07:01:11.222462+0000","last_active":"2024-09-13T07:01:29.069919+0000","last_peered":"2024-09-13T07:01:29.069919+0000","last_clean":"2024-09-13T07:01:29.069919+0000","last_became_active":"2024-09-13T07:01:09.523133+0000","last_became_peered":"2024-09-13T07:01:09.523133+0000","last_unstale":"2024-09-13T07:01:29.069919+0000","last_undegraded":"2024-09-13T07:01:29.069919+0000","last_fullsized":"2024-09-13T07:01:29.069919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:12:38.585984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074093200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"30'10","reported_seq":97,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.110808+0000","last_change":"2024-09-13T07:01:21.305184+0000","last_active":"2024-09-13T07:01:29.110808+0000","last_peered":"2024-09-13T07:01:29.110808+0000","last_clean":"2024-09-13T07:01:29.110808+0000","last_became_active":"2024-09-13T07:01:19.286557+0000","last_became_peered":"2024-09-13T07:01:19.286557+0000","last_unstale":"2024-09-13T07:01:29.110808+0000","last_undegraded":"2024-09-13T07:01:29.110808+0000","last_fullsized":"2024-09-13T07:01:29.110808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:33:25.595462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044811299999999998,"stat_sum":{"num_bytes":12742703,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":71,"num_read_kb":95,"num_write":6,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.094242+0000","last_change":"2024-09-13T07:01:11.859918+0000","last_active":"2024-09-13T07:01:29.094242+0000","last_peered":"2024-09-13T07:01:29.094242+0000","last_clean":"2024-09-13T07:01:29.094242+0000","last_became_active":"2024-09-13T07:01:09.210971+0000","last_became_peered":"2024-09-13T07:01:09.210971+0000","last_unstale":"2024-09-13T07:01:29.094242+0000","last_undegraded":"2024-09-13T07:01:29.094242+0000","last_fullsized":"2024-09-13T07:01:29.094242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:30.775691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"30'76","reported_seq":610,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.612609+0000","last_change":"2024-09-13T07:01:21.290419+0000","last_active":"2024-09-13T07:01:30.612609+0000","last_peered":"2024-09-13T07:01:30.612609+0000","last_clean":"2024-09-13T07:01:30.612609+0000","last_became_active":"2024-09-13T07:01:19.282935+0000","last_became_peered":"2024-09-13T07:01:19.282935+0000","last_unstale":"2024-09-13T07:01:30.612609+0000","last_undegraded":"2024-09-13T07:01:30.612609+0000","last_fullsized":"2024-09-13T07:01:30.612609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:36:53.773908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064294199999999997,"stat_sum":{"num_bytes":409616,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":1538,"num_read_kb":1328,"num_write":71,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.124854+0000","last_change":"2024-09-13T07:01:11.227425+0000","last_active":"2024-09-13T07:01:29.124854+0000","last_peered":"2024-09-13T07:01:29.124854+0000","last_clean":"2024-09-13T07:01:29.124854+0000","last_became_active":"2024-09-13T07:01:09.207226+0000","last_became_peered":"2024-09-13T07:01:09.207226+0000","last_unstale":"2024-09-13T07:01:29.124854+0000","last_undegraded":"2024-09-13T07:01:29.124854+0000","last_fullsized":"2024-09-13T07:01:29.124854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:12.911230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.094428+0000","last_change":"2024-09-13T07:01:02.162365+0000","last_active":"2024-09-13T07:01:29.094428+0000","last_peered":"2024-09-13T07:01:29.094428+0000","last_clean":"2024-09-13T07:01:29.094428+0000","last_became_active":"2024-09-13T07:01:02.161407+0000","last_became_peered":"2024-09-13T07:01:02.161407+0000","last_unstale":"2024-09-13T07:01:29.094428+0000","last_undegraded":"2024-09-13T07:01:29.094428+0000","last_fullsized":"2024-09-13T07:01:29.094428+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:17.998339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.123912+0000","last_change":"2024-09-13T07:01:21.291091+0000","last_active":"2024-09-13T07:01:29.123912+0000","last_peered":"2024-09-13T07:01:29.123912+0000","last_clean":"2024-09-13T07:01:29.123912+0000","last_became_active":"2024-09-13T07:01:19.280796+0000","last_became_peered":"2024-09-13T07:01:19.280796+0000","last_unstale":"2024-09-13T07:01:29.123912+0000","last_undegraded":"2024-09-13T07:01:29.123912+0000","last_fullsized":"2024-09-13T07:01:29.123912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:20:10.332964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001264996,"stat_sum":{"num_bytes":933904,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":17,"num_read_kb":18,"num_write":7,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"27'4","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.073423+0000","last_change":"2024-09-13T07:01:21.298330+0000","last_active":"2024-09-13T07:01:29.073423+0000","last_peered":"2024-09-13T07:01:29.073423+0000","last_clean":"2024-09-13T07:01:29.073423+0000","last_became_active":"2024-09-13T07:01:19.816011+0000","last_became_peered":"2024-09-13T07:01:19.816011+0000","last_unstale":"2024-09-13T07:01:29.073423+0000","last_undegraded":"2024-09-13T07:01:29.073423+0000","last_fullsized":"2024-09-13T07:01:29.073423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:10:08.758669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022509729999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"32'50","reported_seq":79,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.467782+0000","last_change":"2024-09-13T07:01:22.025365+0000","last_active":"2024-09-13T07:01:30.467782+0000","last_peered":"2024-09-13T07:01:30.467782+0000","last_clean":"2024-09-13T07:01:30.467782+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:01:30.467782+0000","last_undegraded":"2024-09-13T07:01:30.467782+0000","last_fullsized":"2024-09-13T07:01:30.467782+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:11.537644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064478199999999999,"stat_sum":{"num_bytes":948634,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":3,"num_read":39,"num_read_kb":77,"num_write":44,"num_write_kb":10678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"30'11","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.423553+0000","last_change":"2024-09-13T07:01:21.291637+0000","last_active":"2024-09-13T07:01:30.423553+0000","last_peered":"2024-09-13T07:01:30.423553+0000","last_clean":"2024-09-13T07:01:30.423553+0000","last_became_active":"2024-09-13T07:01:19.282852+0000","last_became_peered":"2024-09-13T07:01:19.282852+0000","last_unstale":"2024-09-13T07:01:30.423553+0000","last_undegraded":"2024-09-13T07:01:30.423553+0000","last_fullsized":"2024-09-13T07:01:30.423553+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:51.391035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001533932,"stat_sum":{"num_bytes":1679641,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":22,"num_read_kb":30,"num_write":7,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"32'95","reported_seq":463,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.664365+0000","last_change":"2024-09-13T07:01:21.292633+0000","last_active":"2024-09-13T07:01:30.664365+0000","last_peered":"2024-09-13T07:01:30.664365+0000","last_clean":"2024-09-13T07:01:30.664365+0000","last_became_active":"2024-09-13T07:01:19.279813+0000","last_became_peered":"2024-09-13T07:01:19.279813+0000","last_unstale":"2024-09-13T07:01:30.664365+0000","last_undegraded":"2024-09-13T07:01:30.664365+0000","last_fullsized":"2024-09-13T07:01:30.664365+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:39.697691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072600099999999999,"stat_sum":{"num_bytes":22367453,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":806,"num_read_kb":693,"num_write":115,"num_write_kb":7105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"32'55","reported_seq":308,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.672462+0000","last_change":"2024-09-13T07:01:21.313310+0000","last_active":"2024-09-13T07:01:30.672462+0000","last_peered":"2024-09-13T07:01:30.672462+0000","last_clean":"2024-09-13T07:01:30.672462+0000","last_became_active":"2024-09-13T07:01:19.282520+0000","last_became_peered":"2024-09-13T07:01:19.282520+0000","last_unstale":"2024-09-13T07:01:30.672462+0000","last_undegraded":"2024-09-13T07:01:30.672462+0000","last_fullsized":"2024-09-13T07:01:30.672462+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:41:44.801487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018248170000000001,"stat_sum":{"num_bytes":19251200,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":483,"num_read_kb":419,"num_write":45,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"32'25","reported_seq":112,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.662849+0000","last_change":"2024-09-13T07:01:21.313674+0000","last_active":"2024-09-13T07:01:30.662849+0000","last_peered":"2024-09-13T07:01:30.662849+0000","last_clean":"2024-09-13T07:01:30.662849+0000","last_became_active":"2024-09-13T07:01:19.281215+0000","last_became_peered":"2024-09-13T07:01:19.281215+0000","last_unstale":"2024-09-13T07:01:30.662849+0000","last_undegraded":"2024-09-13T07:01:30.662849+0000","last_fullsized":"2024-09-13T07:01:30.662849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:22.161596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019878309999999998,"stat_sum":{"num_bytes":6832128,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":80,"num_read_kb":51,"num_write":20,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"30'14","reported_seq":104,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.363924+0000","last_change":"2024-09-13T07:01:21.314048+0000","last_active":"2024-09-13T07:01:30.363924+0000","last_peered":"2024-09-13T07:01:30.363924+0000","last_clean":"2024-09-13T07:01:30.363924+0000","last_became_active":"2024-09-13T07:01:19.281169+0000","last_became_peered":"2024-09-13T07:01:19.281169+0000","last_unstale":"2024-09-13T07:01:30.363924+0000","last_undegraded":"2024-09-13T07:01:30.363924+0000","last_fullsized":"2024-09-13T07:01:30.363924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:49:46.525500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023438180000000001,"stat_sum":{"num_bytes":4718592,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":74,"num_read_kb":7,"num_write":9,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"27'11","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.067959+0000","last_change":"2024-09-13T07:01:22.025282+0000","last_active":"2024-09-13T07:01:29.067959+0000","last_peered":"2024-09-13T07:01:29.067959+0000","last_clean":"2024-09-13T07:01:29.067959+0000","last_became_active":"2024-09-13T07:01:19.283407+0000","last_became_peered":"2024-09-13T07:01:19.283407+0000","last_unstale":"2024-09-13T07:01:29.067959+0000","last_undegraded":"2024-09-13T07:01:29.067959+0000","last_fullsized":"2024-09-13T07:01:29.067959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:33:56.343722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061928799999999996,"stat_sum":{"num_bytes":344064,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":8,"num_read_kb":19,"num_write":8,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"30'12","reported_seq":72,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.094404+0000","last_change":"2024-09-13T07:01:21.313825+0000","last_active":"2024-09-13T07:01:29.094404+0000","last_peered":"2024-09-13T07:01:29.094404+0000","last_clean":"2024-09-13T07:01:29.094404+0000","last_became_active":"2024-09-13T07:01:19.279714+0000","last_became_peered":"2024-09-13T07:01:19.279714+0000","last_unstale":"2024-09-13T07:01:29.094404+0000","last_undegraded":"2024-09-13T07:01:29.094404+0000","last_fullsized":"2024-09-13T07:01:29.094404+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:06:09.717433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002154132,"stat_sum":{"num_bytes":7685649,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":4,"num_read":43,"num_read_kb":17,"num_write":8,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"31'92","reported_seq":184,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T07:01:30.428310+0000","last_change":"2024-09-13T07:01:21.305471+0000","last_active":"2024-09-13T07:01:30.428310+0000","last_peered":"2024-09-13T07:01:30.428310+0000","last_clean":"2024-09-13T07:01:30.428310+0000","last_became_active":"2024-09-13T07:01:19.284230+0000","last_became_peered":"2024-09-13T07:01:19.284230+0000","last_unstale":"2024-09-13T07:01:30.428310+0000","last_undegraded":"2024-09-13T07:01:30.428310+0000","last_fullsized":"2024-09-13T07:01:30.428310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:47.692777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060210699999999995,"stat_sum":{"num_bytes":11563403,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":388,"num_read_kb":360,"num_write":121,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"28'77","reported_seq":103,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T07:01:29.094847+0000","last_change":"2024-09-13T07:01:21.313226+0000","last_active":"2024-09-13T07:01:29.094847+0000","last_peered":"2024-09-13T07:01:29.094847+0000","last_clean":"2024-09-13T07:01:29.094847+0000","last_became_active":"2024-09-13T07:01:19.279419+0000","last_became_peered":"2024-09-13T07:01:19.279419+0000","last_unstale":"2024-09-13T07:01:29.094847+0000","last_undegraded":"2024-09-13T07:01:29.094847+0000","last_fullsized":"2024-09-13T07:01:29.094847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:46:29.817757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001803165,"stat_sum":{"num_bytes":41,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":287,"num_read_kb":212,"num_write":132,"num_write_kb":7164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":197860452,"num_objects":211,"num_object_clones":46,"num_object_copies":422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":211,"num_whiteouts":125,"num_read":6715,"num_read_kb":11822,"num_write":1196,"num_write_kb":41905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6160384,"data_stored":5835067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1116,"ondisk_log_size":1116,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34416,"kb_used_data":3304,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337424,"statfs":{"total":96636764160,"available":96601522176,"internally_reserved":0,"allocated":3383296,"data_stored":3128926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":372736,"data_stored":166297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":708608,"data_stored":431495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29692,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342148,"statfs":{"total":96636764160,"available":96606359552,"internally_reserved":0,"allocated":2740224,"data_stored":2485383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":733184,"data_stored":471928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":348160,"data_stored":135213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":487424,"data_stored":255641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36608,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335232,"statfs":{"total":96636764160,"available":96599277568,"internally_reserved":0,"allocated":1277952,"data_stored":1022121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.643}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":296586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":120494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":336843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2404352,"data_stored":2350222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":296349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":31130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2453504,"data_stored":2403397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:01:32.090 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T07:01:32.090 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:01:32.091 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-13T07:01:32.091 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-13T07:01:32.139 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:32.133+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:32.177 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:32.177 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:32.187 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:01:32.240 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:32.233+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:32.302 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:32.303 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:32.313 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:01:32.341 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:32.337+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:32.382 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:01:32.386 INFO:teuthology.orchestra.run.smithi118.stdout:moved 0 / 82 (0%) 2024-09-13T07:01:32.386 INFO:teuthology.orchestra.run.smithi118.stdout:avg 10.25 2024-09-13T07:01:32.386 INFO:teuthology.orchestra.run.smithi118.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-13T07:01:32.386 INFO:teuthology.orchestra.run.smithi118.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-13T07:01:32.386 INFO:teuthology.orchestra.run.smithi118.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-13T07:01:32.387 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:32.387 INFO:teuthology.orchestra.run.smithi118.stdout:oload 1000 2024-09-13T07:01:32.387 INFO:teuthology.orchestra.run.smithi118.stdout:max_change 1 2024-09-13T07:01:32.387 INFO:teuthology.orchestra.run.smithi118.stdout:max_change_osds 4 2024-09-13T07:01:32.387 INFO:teuthology.orchestra.run.smithi118.stdout:average_utilization 0.0003 2024-09-13T07:01:32.388 INFO:teuthology.orchestra.run.smithi118.stdout:overload_utilization 0.0034 2024-09-13T07:01:32.389 INFO:teuthology.orchestra.run.smithi118.stderr:no change 2024-09-13T07:01:32.403 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-13T07:01:32.403 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-13T07:01:32.421 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:32.422 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:32.431 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:01:32.473 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:32.469+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:32.507 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone6 from 0xbdafe29 (overlap 0x3e6fe56) to 0x2bb4ba2 2024-09-13T07:01:32.544 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:32.536+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:32.593 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:32.593 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:32.603 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:01:32.642 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:32.640+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:32.706 INFO:teuthology.orchestra.run.smithi118.stdout:moved 0 / 82 (0%) 2024-09-13T07:01:32.706 INFO:teuthology.orchestra.run.smithi118.stdout:avg 10.25 2024-09-13T07:01:32.706 INFO:teuthology.orchestra.run.smithi118.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-13T07:01:32.706 INFO:teuthology.orchestra.run.smithi118.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-13T07:01:32.706 INFO:teuthology.orchestra.run.smithi118.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-13T07:01:32.707 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:32.707 INFO:teuthology.orchestra.run.smithi118.stdout:oload 1000 2024-09-13T07:01:32.707 INFO:teuthology.orchestra.run.smithi118.stdout:max_change 0.05 2024-09-13T07:01:32.707 INFO:teuthology.orchestra.run.smithi118.stdout:max_change_osds 4 2024-09-13T07:01:32.707 INFO:teuthology.orchestra.run.smithi118.stdout:average_utilization 0.0003 2024-09-13T07:01:32.707 INFO:teuthology.orchestra.run.smithi118.stdout:overload_utilization 0.0034 2024-09-13T07:01:32.710 INFO:teuthology.orchestra.run.smithi118.stderr:no change 2024-09-13T07:01:32.714 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:32.714 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:32.721 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-13T07:01:32.722 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-13T07:01:32.722 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-13T07:01:32.724 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:01:32.766 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:32.764+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:32.800 INFO:teuthology.orchestra.run.smithi118.stdout:478 writesame 0x8f63f22 thru 0x8f69910 data_size 0x17(0x59ef bytes) 2024-09-13T07:01:32.832 INFO:teuthology.orchestra.run.smithi118.stdout:479 write 0xe9537af thru 0xe95e1f0 (0xaa42 bytes) 2024-09-13T07:01:32.838 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:32.839 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:32.841 INFO:teuthology.orchestra.run.smithi118.stdout:480 writesame 0x13038eb thru 0x130554c data_size 0x15(0x1c62 bytes) 2024-09-13T07:01:32.844 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:32.837+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:32.854 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:01:32.856 INFO:teuthology.orchestra.run.smithi118.stdout:482 read 0x7155d6e thru 0x716005c (0xa2ef bytes) 2024-09-13T07:01:32.856 INFO:teuthology.orchestra.run.smithi118.stdout:483 write 0xacca6b4 thru 0xacd110f (0x6a5c bytes) 2024-09-13T07:01:32.873 INFO:teuthology.orchestra.run.smithi118.stdout:484 write 0x1ca4b5e thru 0x1ca7fc7 (0x346a bytes) 2024-09-13T07:01:32.897 INFO:teuthology.orchestra.run.smithi118.stdout:485 trunc from 0xe95e1f1 to 0x8a96ad3 2024-09-13T07:01:32.931 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:32.934 INFO:teuthology.orchestra.run.smithi118.stdout:487 trunc from 0x8a96ad3 to 0x6b081d8 2024-09-13T07:01:32.945 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:32.941+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:32.964 INFO:teuthology.orchestra.run.smithi118.stdout:488 punch from 0x324a428 to 0x324fc51, (0x5829 bytes) 2024-09-13T07:01:32.964 INFO:teuthology.orchestra.run.smithi118.stdout:491 read 0x3a67177 thru 0x3a691c1 (0x204b bytes) 2024-09-13T07:01:32.965 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:32.965 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:32.965 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:32.974 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:01:32.977 INFO:teuthology.orchestra.run.smithi118.stdout:494 trunc from 0x6b081d8 to 0xd9d05ce 2024-09-13T07:01:32.985 INFO:teuthology.orchestra.run.smithi118.stdout:498 punch from 0x36aa996 to 0x36af514, (0x4b7e bytes) 2024-09-13T07:01:32.985 INFO:teuthology.orchestra.run.smithi118.stdout:499 read 0xb1041dd thru 0xb108c1d (0x4a41 bytes) 2024-09-13T07:01:32.986 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:32.986 INFO:teuthology.orchestra.run.smithi118.stdout:502 punch from 0x29d82a4 to 0x29e7999, (0xf6f5 bytes) 2024-09-13T07:01:32.994 INFO:teuthology.orchestra.run.smithi118.stdout:503 write 0xe2e9be7 thru 0xe2f213a (0x8554 bytes) 2024-09-13T07:01:33.003 INFO:teuthology.orchestra.run.smithi118.stdout:505 read 0x9401845 thru 0x9409ca8 (0x8464 bytes) 2024-09-13T07:01:33.009 INFO:teuthology.orchestra.run.smithi118.stdout:506 trunc from 0xe2f213b to 0xa8e148a 2024-09-13T07:01:33.042 INFO:teuthology.orchestra.run.smithi118.stdout:507 writesame 0xdc09408 thru 0xdc0fcc1 data_size 0x23(0x68ba bytes) 2024-09-13T07:01:33.042 INFO:teuthology.orchestra.run.smithi118.stdout:509 write 0x812e77e thru 0x81356cf (0x6f52 bytes) 2024-09-13T07:01:33.046 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:33.045+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:33.053 INFO:teuthology.orchestra.run.smithi118.stdout:510 punch from 0xc53e957 to 0xc54419c, (0x5845 bytes) 2024-09-13T07:01:33.073 INFO:teuthology.orchestra.run.smithi118.stdout:511 trunc from 0xdc0fcc2 to 0x1c16aa1 2024-09-13T07:01:33.084 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:33.085 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:33.097 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:01:33.135 INFO:teuthology.orchestra.run.smithi118.stdout:513 punch from 0x1ad2248 to 0x1ad55b7, (0x336f bytes) 2024-09-13T07:01:33.135 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:33.135 INFO:teuthology.orchestra.run.smithi118.stdout:518 read 0xcc0593 thru 0xccb1aa (0xac18 bytes) 2024-09-13T07:01:33.138 INFO:teuthology.orchestra.run.smithi118.stdout:519 trunc from 0x1c16aa1 to 0x3bbe53a 2024-09-13T07:01:33.147 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:33.141+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:33.151 INFO:teuthology.orchestra.run.smithi118.stdout:520 trunc from 0x3bbe53a to 0x6ae3def 2024-09-13T07:01:33.164 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:33.164 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:33.164 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:33.165 INFO:teuthology.orchestra.run.smithi118.stdout:526 punch from 0x550ca0e to 0x550e2d6, (0x18c8 bytes) 2024-09-13T07:01:33.165 INFO:teuthology.orchestra.run.smithi118.stdout:530 writesame 0x185582f thru 0x185895a data_size 0x6(0x312c bytes) 2024-09-13T07:01:33.176 INFO:teuthology.orchestra.run.smithi118.stdout:531 punch from 0x3ee79c2 to 0x3eee0d9, (0x6717 bytes) 2024-09-13T07:01:33.176 INFO:teuthology.orchestra.run.smithi118.stdout:532 punch from 0x2893a95 to 0x289efb3, (0xb51e bytes) 2024-09-13T07:01:33.176 INFO:teuthology.orchestra.run.smithi118.stdout:533 punch from 0x2b50f7e to 0x2b540d6, (0x3158 bytes) 2024-09-13T07:01:33.181 INFO:teuthology.orchestra.run.smithi118.stdout:534 trunc from 0x6ae3def to 0x365110f 2024-09-13T07:01:33.189 INFO:teuthology.orchestra.run.smithi118.stdout:537 write 0x21c8008 thru 0x21cdf63 (0x5f5c bytes) 2024-09-13T07:01:33.193 INFO:teuthology.orchestra.run.smithi118.stdout:539 read 0x1169ae8 thru 0x116e6b3 (0x4bcc bytes) 2024-09-13T07:01:33.193 INFO:teuthology.orchestra.run.smithi118.stdout:540 read 0x2e1558b thru 0x2e1fe0f (0xa885 bytes) 2024-09-13T07:01:33.194 INFO:teuthology.orchestra.run.smithi118.stdout:542 read 0x35243a1 thru 0x3529075 (0x4cd5 bytes) 2024-09-13T07:01:33.201 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:33.201 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:33.205 INFO:teuthology.orchestra.run.smithi118.stdout:543 trunc from 0x365110f to 0x9d84211 2024-09-13T07:01:33.211 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:01:33.229 INFO:teuthology.orchestra.run.smithi118.stdout:544 write 0xbc13ab5 thru 0xbc16d56 (0x32a2 bytes) 2024-09-13T07:01:33.238 INFO:teuthology.orchestra.run.smithi118.stdout:545 write 0x66c3e46 thru 0x66ca295 (0x6450 bytes) 2024-09-13T07:01:33.248 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:33.241+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:33.249 INFO:teuthology.orchestra.run.smithi118.stdout:546 read 0x17fe963 thru 0x180e576 (0xfc14 bytes) 2024-09-13T07:01:33.252 INFO:teuthology.orchestra.run.smithi118.stdout:547 trunc from 0xbc16d57 to 0xc17fdef 2024-09-13T07:01:33.263 INFO:teuthology.orchestra.run.smithi118.stdout:550 read 0x982dca3 thru 0x983bc29 (0xdf87 bytes) 2024-09-13T07:01:33.263 INFO:teuthology.orchestra.run.smithi118.stdout:551 trunc from 0xc17fdef to 0xbedbba3 2024-09-13T07:01:33.273 INFO:teuthology.orchestra.run.smithi118.stdout:552 write 0x9cde383 thru 0x9ceccc6 (0xe944 bytes) 2024-09-13T07:01:33.299 INFO:teuthology.orchestra.run.smithi118.stdout:553 trunc from 0xbedbba3 to 0x6c1aeea 2024-09-13T07:01:33.322 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:33.322 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:33.326 INFO:teuthology.orchestra.run.smithi118.stdout:554 trunc from 0x6c1aeea to 0x1fce1b8 2024-09-13T07:01:33.332 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:01:33.347 INFO:teuthology.orchestra.run.smithi118.stdout:555 read 0x125d833 thru 0x126a538 (0xcd06 bytes) 2024-09-13T07:01:33.348 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:33.345+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:33.370 INFO:teuthology.orchestra.run.smithi118.stdout:556 trunc from 0x1fce1b8 to 0xd986d17 2024-09-13T07:01:33.375 INFO:teuthology.orchestra.run.smithi118.stdout:moved 1 / 82 (1.21951%) 2024-09-13T07:01:33.375 INFO:teuthology.orchestra.run.smithi118.stdout:avg 10.25 2024-09-13T07:01:33.375 INFO:teuthology.orchestra.run.smithi118.stdout:stddev 3.26917 -> 3.34477 (expected baseline 2.99479) 2024-09-13T07:01:33.375 INFO:teuthology.orchestra.run.smithi118.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-13T07:01:33.375 INFO:teuthology.orchestra.run.smithi118.stdout:max osd.7 with 14 -> 15 pgs (1.36585 -> 1.46341 * mean) 2024-09-13T07:01:33.376 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:33.376 INFO:teuthology.orchestra.run.smithi118.stdout:oload 110 2024-09-13T07:01:33.376 INFO:teuthology.orchestra.run.smithi118.stdout:max_change 0.05 2024-09-13T07:01:33.376 INFO:teuthology.orchestra.run.smithi118.stdout:max_change_osds 4 2024-09-13T07:01:33.376 INFO:teuthology.orchestra.run.smithi118.stdout:average_utilization 0.0003 2024-09-13T07:01:33.376 INFO:teuthology.orchestra.run.smithi118.stdout:overload_utilization 0.0004 2024-09-13T07:01:33.376 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 weight 1.0000 -> 0.9500 2024-09-13T07:01:33.380 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:01:33.385 INFO:teuthology.orchestra.run.smithi118.stdout:558 read 0xd3f13e7 thru 0xd3fa08b (0x8ca5 bytes) 2024-09-13T07:01:33.385 INFO:teuthology.orchestra.run.smithi118.stdout:561 write 0xa7c1b8c thru 0xa7cd205 (0xb67a bytes) 2024-09-13T07:01:33.395 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-13T07:01:33.395 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-13T07:01:33.396 INFO:teuthology.orchestra.run.smithi118.stdout:562 write 0x1bb5760 thru 0x1bb7a55 (0x22f6 bytes) 2024-09-13T07:01:33.405 INFO:teuthology.orchestra.run.smithi118.stdout:563 punch from 0xa1d4e7 to 0xa1de2d, (0x946 bytes) 2024-09-13T07:01:33.405 INFO:teuthology.orchestra.run.smithi118.stdout:564 punch from 0xb58e667 to 0xb59b0b5, (0xca4e bytes) 2024-09-13T07:01:33.405 INFO:teuthology.orchestra.run.smithi118.stdout:565 read 0x776a5c1 thru 0x776ed55 (0x4795 bytes) 2024-09-13T07:01:33.405 INFO:teuthology.orchestra.run.smithi118.stdout:566 read 0x7509701 thru 0x7516dfc (0xd6fc bytes) 2024-09-13T07:01:33.405 INFO:teuthology.orchestra.run.smithi118.stdout:568 read 0xb49cca3 thru 0xb49ff3b (0x3299 bytes) 2024-09-13T07:01:33.405 INFO:teuthology.orchestra.run.smithi118.stdout:570 writesame 0xc419984 thru 0xc4257d6 data_size 0x3(0xbe53 bytes) 2024-09-13T07:01:33.420 INFO:teuthology.orchestra.run.smithi118.stdout:573 write 0x81f6014 thru 0x81f705c (0x1049 bytes) 2024-09-13T07:01:33.425 INFO:teuthology.orchestra.run.smithi118.stdout:575 writesame 0x598378b thru 0x5987417 data_size 0x1(0x3c8d bytes) 2024-09-13T07:01:33.440 INFO:teuthology.orchestra.run.smithi118.stdout:576 read 0x5388d26 thru 0x538e790 (0x5a6b bytes) 2024-09-13T07:01:33.440 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:33.440 INFO:teuthology.orchestra.run.smithi118.stdout:579 read 0x197c245 thru 0x197f9a7 (0x3763 bytes) 2024-09-13T07:01:33.441 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:33.441 INFO:teuthology.orchestra.run.smithi118.stdout:581 write 0x374b681 thru 0x3757cce (0xc64e bytes) 2024-09-13T07:01:33.449 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:33.445+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:33.451 INFO:teuthology.orchestra.run.smithi118.stdout:582 trunc from 0xd986d17 to 0xd7abc80 2024-09-13T07:01:33.475 INFO:teuthology.orchestra.run.smithi118.stdout:583 trunc from 0xd7abc80 to 0x2e9a8d8 2024-09-13T07:01:33.488 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:33.488 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:33.498 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:01:33.511 INFO:teuthology.orchestra.run.smithi118.stdout:586 writesame 0x740bc4a thru 0x74156c1 data_size 0x4(0x9a78 bytes) 2024-09-13T07:01:33.522 INFO:teuthology.orchestra.run.smithi118.stdout:588 trunc from 0x74156c2 to 0xe14c68a 2024-09-13T07:01:33.529 INFO:teuthology.orchestra.run.smithi118.stdout:589 write 0x3b4247e thru 0x3b49110 (0x6c93 bytes) 2024-09-13T07:01:33.537 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:01:33.540 INFO:teuthology.orchestra.run.smithi118.stdout:591 write 0x68b7b20 thru 0x68bfad7 (0x7fb8 bytes) 2024-09-13T07:01:33.550 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:33.545+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:33.553 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:33.553 INFO:teuthology.orchestra.run.smithi118.stdout:594 read 0x6bf2552 thru 0x6bff754 (0xd203 bytes) 2024-09-13T07:01:33.554 INFO:teuthology.orchestra.run.smithi118.stdout:595 write 0x329a1cd thru 0x32a366b (0x949f bytes) 2024-09-13T07:01:33.560 INFO:teuthology.orchestra.run.smithi118.stdout:596 read 0xd858e70 thru 0xd866793 (0xd924 bytes) 2024-09-13T07:01:33.561 INFO:teuthology.orchestra.run.smithi118.stdout:597 write 0x97a3174 thru 0x97aa0ac (0x6f39 bytes) 2024-09-13T07:01:33.569 INFO:teuthology.orchestra.run.smithi118.stdout:598 read 0xa32adb9 thru 0xa32c961 (0x1ba9 bytes) 2024-09-13T07:01:33.569 INFO:teuthology.orchestra.run.smithi118.stdout:600 write 0x96543c4 thru 0x965ec11 (0xa84e bytes) 2024-09-13T07:01:33.596 INFO:teuthology.orchestra.run.smithi118.stdout:602 trunc from 0xe14c68a to 0x78b6c8 2024-09-13T07:01:33.606 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:33.606 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:33.617 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:01:33.650 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:33.649+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:33.664 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:33.664 INFO:teuthology.orchestra.run.smithi118.stdout:605 read 0x47cb19 thru 0x47fbf3 (0x30db bytes) 2024-09-13T07:01:33.665 INFO:teuthology.orchestra.run.smithi118.stdout:606 read 0x3cc6b3 thru 0x3db73a (0xf088 bytes) 2024-09-13T07:01:33.673 INFO:teuthology.orchestra.run.smithi118.stdout:607 write 0x8b3a79 thru 0x8c31f2 (0xf77a bytes) 2024-09-13T07:01:33.677 INFO:teuthology.orchestra.run.smithi118.stdout:608 trunc from 0x8c31f3 to 0xbb7683 2024-09-13T07:01:33.681 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:33.685 INFO:teuthology.orchestra.run.smithi118.stdout:609 read 0x8c0c58 thru 0x8c9c4c (0x8ff5 bytes) 2024-09-13T07:01:33.686 INFO:teuthology.orchestra.run.smithi118.stdout:611 read 0x5afa2d thru 0x5bf75a (0xfd2e bytes) 2024-09-13T07:01:33.700 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:33.718 INFO:teuthology.orchestra.run.smithi118.stdout:612 writesame 0xbf502bd thru 0xbf5f540 data_size 0xa6(0xf284 bytes) 2024-09-13T07:01:33.718 INFO:teuthology.orchestra.run.smithi118.stdout:613 read 0x5f67979 thru 0x5f6ebe9 (0x7271 bytes) 2024-09-13T07:01:33.718 INFO:teuthology.orchestra.run.smithi118.stdout:614 read 0x6605029 thru 0x660935b (0x4333 bytes) 2024-09-13T07:01:33.718 INFO:teuthology.orchestra.run.smithi118.stdout:615 writesame 0x96f91c4 thru 0x96ff159 data_size 0x5(0x5f96 bytes) 2024-09-13T07:01:33.720 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:33.720 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:33.720 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:33.726 INFO:teuthology.orchestra.run.smithi118.stdout:617 write 0x5d87ff5 thru 0x5d9418a (0xc196 bytes) 2024-09-13T07:01:33.731 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:01:33.736 INFO:teuthology.orchestra.run.smithi118.stdout:618 writesame 0x3c9db8 thru 0x3d0522 data_size 0x19(0x676b bytes) 2024-09-13T07:01:33.743 INFO:teuthology.orchestra.run.smithi118.stdout:moved 0 / 82 (0%) 2024-09-13T07:01:33.743 INFO:teuthology.orchestra.run.smithi118.stdout:avg 10.25 2024-09-13T07:01:33.744 INFO:teuthology.orchestra.run.smithi118.stdout:stddev 3.34477 -> 3.34477 (expected baseline 2.99479) 2024-09-13T07:01:33.744 INFO:teuthology.orchestra.run.smithi118.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-13T07:01:33.744 INFO:teuthology.orchestra.run.smithi118.stdout:max osd.7 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-13T07:01:33.744 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:33.744 INFO:teuthology.orchestra.run.smithi118.stdout:oload 1000 2024-09-13T07:01:33.744 INFO:teuthology.orchestra.run.smithi118.stdout:max_change 1 2024-09-13T07:01:33.744 INFO:teuthology.orchestra.run.smithi118.stdout:max_change_osds 4 2024-09-13T07:01:33.744 INFO:teuthology.orchestra.run.smithi118.stdout:average_utilization 0.0003 2024-09-13T07:01:33.744 INFO:teuthology.orchestra.run.smithi118.stdout:overload_utilization 0.0034 2024-09-13T07:01:33.746 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:33.747 INFO:teuthology.orchestra.run.smithi118.stderr:no change 2024-09-13T07:01:33.751 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:33.745+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:33.759 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-13T07:01:33.760 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-13T07:01:33.764 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:33.789 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:33.826 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:33.831 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:33.832 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:33.842 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:01:33.852 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:33.845+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:33.862 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:33.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:33.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:33.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:33.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:33.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:33.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:33.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:33.865 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:33.954 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:33.949+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:33.963 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:33.963 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:33.963 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:33.963 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:33.963 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:33.973 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:01:34.055 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:34.049+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:34.077 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:34.077 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:34.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:34.078 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:34.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.078 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:34.078 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:34.078 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.5703391539999991, 2024-09-13T07:01:34.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:01:34.078 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.078 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.078 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.078 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.078 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.078 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:01:34.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:01:34.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:01:34.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:34.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:01:34.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:01:34.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:01:34.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:01:34.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:01:34.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:01:34.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.083 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.5455114440000006, 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.0068396199999992, 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.089 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:34.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:01:34.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:34.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:01:34.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:01:34.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:01:34.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:01:34.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:01:34.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:34.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:01:34.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:01:34.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.9772144269999998, 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.095 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:01:34.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:01:34.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.100 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.4445007959999998, 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.101 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:34.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:01:34.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:01:34.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:01:34.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:01:34.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:01:34.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:01:34.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.4444183400000004, 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.4443865300000001, 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.112 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:34.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:01:34.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:01:34.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:01:34.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:01:34.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:01:34.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:01:34.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:01:34.117 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.117 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.117 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.4443313719999997, 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:01:34.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:01:34.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:01:34.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.4442860929999997, 2024-09-13T07:01:34.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:01:34.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:01:34.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:01:34.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:01:34.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.4442475589999999, 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.129 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:01:34.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:01:34.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:01:34.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:34.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:01:34.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:01:34.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:01:34.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:01:34.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.134 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.4441859399999997, 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.135 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:01:34.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:01:34.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:01:34.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:01:34.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.4299632569999998, 2024-09-13T07:01:34.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:01:34.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:01:34.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3951 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13369344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657045+0000", 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.4181047210000002, 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090047159999999994, 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.146 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3951 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657045+0000", 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657045+0000", 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657044+0000", 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657046+0000", 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3099999999999999e-06 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657052+0000", 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8980000000000001e-06 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657057+0000", 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7790000000000003e-06 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665904+0000", 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0088473179999999998 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665974+0000", 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0038999999999998e-05 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666049+0000", 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4777999999999995e-05 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.418084737, 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.150 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:01:34.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:01:34.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:01:34.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:01:34.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:01:34.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:01:34.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:01:34.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.154 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.154 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:01:34.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:01:34.154 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.154 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.154 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.154 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4000 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.154 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657605+0000", 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.4175446229999999, 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089452449999999992, 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4000 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657605+0000", 2024-09-13T07:01:34.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657605+0000", 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657604+0000", 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657606+0000", 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.018e-06 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657612+0000", 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9519999999999999e-06 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657617+0000", 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8069999999999999e-06 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666410+0000", 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0087930149999999995 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666478+0000", 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8023999999999998e-05 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666550+0000", 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1796000000000002e-05 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.158 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.4175276760000002, 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:01:34.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:01:34.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:01:34.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:01:34.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:01:34.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:01:34.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:01:34.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.4175055209999998, 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:01:34.163 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.417485804, 2024-09-13T07:01:34.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:01:34.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.4174684740000001, 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:34.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:34.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:01:34.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:01:34.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:01:34.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:01:34.176 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.176 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.176 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.176 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.176 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:34.176 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.417172812, 2024-09-13T07:01:34.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:01:34.176 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.176 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.176 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:01:34.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:01:34.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:01:34.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:01:34.180 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.180 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.180 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.180 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.180 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.180 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:34.185 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:34.177+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:34.185 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:01:34.282 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:34.281+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:34.296 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:34.297 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:34.307 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:01:34.378 INFO:teuthology.orchestra.run.smithi118.stdout:moved 2 / 82 (2.43902%) 2024-09-13T07:01:34.378 INFO:teuthology.orchestra.run.smithi118.stdout:avg 10.25 2024-09-13T07:01:34.378 INFO:teuthology.orchestra.run.smithi118.stdout:stddev 3.34477 -> 3.11247 (expected baseline 2.99479) 2024-09-13T07:01:34.378 INFO:teuthology.orchestra.run.smithi118.stdout:min osd.2 with 5 -> 6 pgs (0.487805 -> 0.585366 * mean) 2024-09-13T07:01:34.378 INFO:teuthology.orchestra.run.smithi118.stdout:max osd.7 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-13T07:01:34.379 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:34.379 INFO:teuthology.orchestra.run.smithi118.stdout:oload 110 2024-09-13T07:01:34.379 INFO:teuthology.orchestra.run.smithi118.stdout:max_change 1 2024-09-13T07:01:34.379 INFO:teuthology.orchestra.run.smithi118.stdout:max_change_osds 4 2024-09-13T07:01:34.379 INFO:teuthology.orchestra.run.smithi118.stdout:average_utilization 0.0003 2024-09-13T07:01:34.379 INFO:teuthology.orchestra.run.smithi118.stdout:overload_utilization 0.0004 2024-09-13T07:01:34.379 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 weight 0.9500 -> 0.8265 2024-09-13T07:01:34.382 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:34.381+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:34.461 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:34.461 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:34.461 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:34.461 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:34.461 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:34.470 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:01:34.483 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:34.477+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:34.577 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:34.577 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:34.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:34.577 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:34.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.577 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1528 3.0 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:34.577 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.049314+0000", 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.525364111, 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026695681999999998, 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1528 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.049314+0000", 2024-09-13T07:01:34.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.049314+0000", 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.049311+0000", 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055045+0000", 2024-09-13T07:01:34.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005733106 2024-09-13T07:01:34.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055075+0000", 2024-09-13T07:01:34.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0540999999999998e-05 2024-09-13T07:01:34.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055090+0000", 2024-09-13T07:01:34.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4827999999999999e-05 2024-09-13T07:01:34.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055215+0000", 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000124657 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055392+0000", 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017693900000000001 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056063+0000", 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00067108999999999995 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075194+0000", 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019131756999999999 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075871+0000", 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00067684799999999997 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075945+0000", 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3468999999999997e-05 2024-09-13T07:01:34.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.076010+0000", 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4911000000000002e-05 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3121 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.260057+0000", 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.3146207180000005, 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028240511999999999, 2024-09-13T07:01:34.583 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3121 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260057+0000", 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260057+0000", 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260056+0000", 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260066+0000", 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0787e-05 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260091+0000", 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4947e-05 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260103+0000", 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1435e-05 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260188+0000", 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5352000000000005e-05 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260310+0000", 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122309 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.267876+0000", 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0075653990000000004 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.288015+0000", 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020139221999999998 2024-09-13T07:01:34.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.288112+0000", 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6731000000000006e-05 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.288214+0000", 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010206599999999999 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.288298+0000", 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4054000000000005e-05 2024-09-13T07:01:34.588 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3827 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.651790+0000", 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.9228877020000001, 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017344287999999999, 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3827 2024-09-13T07:01:34.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651790+0000", 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651790+0000", 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651790+0000", 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651792+0000", 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651798+0000", 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8799999999999996e-06 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651803+0000", 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.515e-06 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665057+0000", 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01325455 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665447+0000", 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00039010200000000003 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669135+0000", 2024-09-13T07:01:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036872580000000001 2024-09-13T07:01:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3933 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652199+0000", 2024-09-13T07:01:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.9224792109999997, 2024-09-13T07:01:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017341579999999999, 2024-09-13T07:01:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.594 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.594 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.594 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.594 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3933 2024-09-13T07:01:34.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.594 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652199+0000", 2024-09-13T07:01:34.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652199+0000", 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652198+0000", 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652201+0000", 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2120000000000002e-06 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652206+0000", 2024-09-13T07:01:34.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3279999999999996e-06 2024-09-13T07:01:34.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652210+0000", 2024-09-13T07:01:34.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8879999999999999e-06 2024-09-13T07:01:34.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669349+0000", 2024-09-13T07:01:34.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017139560000000002 2024-09-13T07:01:34.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669428+0000", 2024-09-13T07:01:34.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8485000000000005e-05 2024-09-13T07:01:34.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669540+0000", 2024-09-13T07:01:34.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112456 2024-09-13T07:01:34.597 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.597 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.597 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.598 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3934 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 14155776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.598 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652214+0000", 2024-09-13T07:01:34.598 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.922464459, 2024-09-13T07:01:34.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017524746000000001, 2024-09-13T07:01:34.598 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.598 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.598 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.598 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.598 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.598 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3934 2024-09-13T07:01:34.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652214+0000", 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652214+0000", 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652213+0000", 2024-09-13T07:01:34.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652216+0000", 2024-09-13T07:01:34.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2790000000000001e-06 2024-09-13T07:01:34.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652221+0000", 2024-09-13T07:01:34.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4859999999999997e-06 2024-09-13T07:01:34.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652225+0000", 2024-09-13T07:01:34.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0090000000000001e-06 2024-09-13T07:01:34.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669598+0000", 2024-09-13T07:01:34.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017373119999999999 2024-09-13T07:01:34.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669679+0000", 2024-09-13T07:01:34.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0636000000000005e-05 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669738+0000", 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9534000000000003e-05 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3959 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 8650752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652229+0000", 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.922449029, 2024-09-13T07:01:34.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017820598, 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3959 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652229+0000", 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652229+0000", 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652229+0000", 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652231+0000", 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2800000000000002e-06 2024-09-13T07:01:34.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652236+0000", 2024-09-13T07:01:34.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3709999999999999e-06 2024-09-13T07:01:34.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652241+0000", 2024-09-13T07:01:34.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2300000000000002e-06 2024-09-13T07:01:34.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669762+0000", 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017521473999999999 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669889+0000", 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012696199999999999 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670050+0000", 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000160602 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3960 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 9961472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652245+0000", 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.92243327, 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018033285999999999, 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3960 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652245+0000", 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652245+0000", 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652244+0000", 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652255+0000", 2024-09-13T07:01:34.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:01:34.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652261+0000", 2024-09-13T07:01:34.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1310000000000003e-06 2024-09-13T07:01:34.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652266+0000", 2024-09-13T07:01:34.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3859999999999998e-06 2024-09-13T07:01:34.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670080+0000", 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017813780000000001 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670166+0000", 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5586000000000004e-05 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670278+0000", 2024-09-13T07:01:34.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011231 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3961 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652271+0000", 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.9224070210000002, 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018223255000000001, 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.611 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3961 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652271+0000", 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652271+0000", 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652271+0000", 2024-09-13T07:01:34.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652274+0000", 2024-09-13T07:01:34.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.016e-06 2024-09-13T07:01:34.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652279+0000", 2024-09-13T07:01:34.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4639999999999997e-06 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652283+0000", 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3150000000000002e-06 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670331+0000", 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018047210000000001 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670404+0000", 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3588000000000001e-05 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670494+0000", 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0085000000000002e-05 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3962 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652288+0000", 2024-09-13T07:01:34.615 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.9223902009999998, 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018397475999999999, 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3962 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652288+0000", 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652288+0000", 2024-09-13T07:01:34.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652287+0000", 2024-09-13T07:01:34.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652290+0000", 2024-09-13T07:01:34.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4849999999999999e-06 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652295+0000", 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4489999999999998e-06 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652300+0000", 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4719999999999997e-06 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670571+0000", 2024-09-13T07:01:34.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018270967999999999 2024-09-13T07:01:34.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670627+0000", 2024-09-13T07:01:34.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6128000000000002e-05 2024-09-13T07:01:34.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670685+0000", 2024-09-13T07:01:34.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8341000000000003e-05 2024-09-13T07:01:34.619 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.619 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3963 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652304+0000", 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.9223735610000001, 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018517675000000001, 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3963 2024-09-13T07:01:34.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.621 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652304+0000", 2024-09-13T07:01:34.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652304+0000", 2024-09-13T07:01:34.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652304+0000", 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652307+0000", 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5790000000000002e-06 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652312+0000", 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7259999999999999e-06 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652317+0000", 2024-09-13T07:01:34.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0300000000000001e-06 2024-09-13T07:01:34.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670712+0000", 2024-09-13T07:01:34.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018394534000000001 2024-09-13T07:01:34.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670770+0000", 2024-09-13T07:01:34.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8041999999999998e-05 2024-09-13T07:01:34.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670822+0000", 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2139999999999999e-05 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3982 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652322+0000", 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.9223557370000002, 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018630451999999999, 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.624 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3982 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652322+0000", 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652322+0000", 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652322+0000", 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652324+0000", 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3300000000000001e-06 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652329+0000", 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7940000000000002e-06 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652333+0000", 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3930000000000001e-06 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670845+0000", 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018511933000000001 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670901+0000", 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.541e-05 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670953+0000", 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1943000000000001e-05 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3983 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.628 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652338+0000", 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.9223400870000003, 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018767628000000001, 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3983 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652338+0000", 2024-09-13T07:01:34.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652338+0000", 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652338+0000", 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652340+0000", 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5339999999999998e-06 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652346+0000", 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7769999999999999e-06 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652350+0000", 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4920000000000004e-06 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670978+0000", 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018627212000000001 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671025+0000", 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7604e-05 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671106+0000", 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0365999999999996e-05 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.632 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3984 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 11665408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652355+0000", 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.922323413, 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020621576999999999, 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3984 2024-09-13T07:01:34.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.634 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652355+0000", 2024-09-13T07:01:34.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652355+0000", 2024-09-13T07:01:34.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652354+0000", 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652357+0000", 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2680000000000001e-06 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652362+0000", 2024-09-13T07:01:34.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8289999999999996e-06 2024-09-13T07:01:34.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652367+0000", 2024-09-13T07:01:34.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4139999999999996e-06 2024-09-13T07:01:34.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671142+0000", 2024-09-13T07:01:34.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018775066 2024-09-13T07:01:34.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671194+0000", 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1983e-05 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672976+0000", 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001782393 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.637 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3985 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652371+0000", 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.922306936, 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019123712000000001, 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3985 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652371+0000", 2024-09-13T07:01:34.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652371+0000", 2024-09-13T07:01:34.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652371+0000", 2024-09-13T07:01:34.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652373+0000", 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.199e-06 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652378+0000", 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2360000000000004e-06 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652382+0000", 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2359999999999998e-06 2024-09-13T07:01:34.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671331+0000", 2024-09-13T07:01:34.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018948606 2024-09-13T07:01:34.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671406+0000", 2024-09-13T07:01:34.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5009e-05 2024-09-13T07:01:34.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671495+0000", 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8769000000000003e-05 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3986 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652387+0000", 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.9222912480000001, 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019262962000000002, 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.642 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.643 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3986 2024-09-13T07:01:34.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.643 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652387+0000", 2024-09-13T07:01:34.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652387+0000", 2024-09-13T07:01:34.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652386+0000", 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652389+0000", 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.306e-06 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652394+0000", 2024-09-13T07:01:34.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6760000000000004e-06 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652399+0000", 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671515+0000", 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019116024999999998 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671581+0000", 2024-09-13T07:01:34.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6105999999999994e-05 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671650+0000", 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8577000000000004e-05 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4011 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652403+0000", 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.9222747560000002, 2024-09-13T07:01:34.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019398288999999999, 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4011 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652403+0000", 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652403+0000", 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652403+0000", 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652405+0000", 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2479999999999999e-06 2024-09-13T07:01:34.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652410+0000", 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2240000000000003e-06 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652415+0000", 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3510000000000004e-06 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671669+0000", 2024-09-13T07:01:34.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019253849999999999 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671732+0000", 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3474999999999998e-05 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671802+0000", 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9522000000000003e-05 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4012 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652419+0000", 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.9222593259999998, 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019542509, 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4012 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652419+0000", 2024-09-13T07:01:34.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652419+0000", 2024-09-13T07:01:34.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652418+0000", 2024-09-13T07:01:34.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652421+0000", 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2970000000000002e-06 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652426+0000", 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7379999999999999e-06 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652431+0000", 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5580000000000002e-06 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671823+0000", 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019391731999999998 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671890+0000", 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7563000000000003e-05 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671961+0000", 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0955000000000001e-05 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4013 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 11403264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652435+0000", 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.9222430739999998, 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019849087000000001, 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.655 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4013 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652435+0000", 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652435+0000", 2024-09-13T07:01:34.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652435+0000", 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652437+0000", 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0810000000000001e-06 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652442+0000", 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2889999999999997e-06 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652446+0000", 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4449999999999998e-06 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671982+0000", 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01953531 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672051+0000", 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9488999999999998e-05 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672284+0000", 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023278799999999999 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4014 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652450+0000", 2024-09-13T07:01:34.659 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.9222277170000002, 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020000851, 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4014 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652450+0000", 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652450+0000", 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652450+0000", 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652452+0000", 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0339999999999999e-06 2024-09-13T07:01:34.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652457+0000", 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3739999999999997e-06 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652462+0000", 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7180000000000004e-06 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672307+0000", 2024-09-13T07:01:34.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019844653 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672380+0000", 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3485000000000005e-05 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672451+0000", 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0901000000000004e-05 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4015 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652470+0000", 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.922208216, 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020116328999999999, 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4015 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652470+0000", 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652470+0000", 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652469+0000", 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652478+0000", 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1990000000000006e-06 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652484+0000", 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9800000000000003e-06 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652488+0000", 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3880000000000002e-06 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672465+0000", 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019976694999999999 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672511+0000", 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6272e-05 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672586+0000", 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5143999999999997e-05 2024-09-13T07:01:34.667 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.668 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.668 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.668 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:34.668 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:34.668 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:34.669 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:01:34.670 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:34.665+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:34.703 INFO:teuthology.orchestra.run.smithi118.stdout:620 write 0x32b113b thru 0x32b1769 (0x62f bytes) 2024-09-13T07:01:34.720 INFO:teuthology.orchestra.run.smithi118.stdout:621 trunc from 0xbf5f541 to 0x6ac6be6 2024-09-13T07:01:34.736 INFO:teuthology.orchestra.run.smithi118.stdout:623 punch from 0x1a29e3b to 0x1a33666, (0x982b bytes) 2024-09-13T07:01:34.746 INFO:teuthology.orchestra.run.smithi118.stdout:624 trunc from 0x6ac6be6 to 0xcd1b645 2024-09-13T07:01:34.756 INFO:teuthology.orchestra.run.smithi118.stdout:625 read 0x51502a6 thru 0x5150399 (0xf4 bytes) 2024-09-13T07:01:34.756 INFO:teuthology.orchestra.run.smithi118.stdout:626 read 0x47f7dd9 thru 0x480353c (0xb764 bytes) 2024-09-13T07:01:34.756 INFO:teuthology.orchestra.run.smithi118.stdout:627 punch from 0x31561a1 to 0x315b136, (0x4f95 bytes) 2024-09-13T07:01:34.756 INFO:teuthology.orchestra.run.smithi118.stdout:628 writesame 0x22e5024 thru 0x22f0953 data_size 0x8(0xb930 bytes) 2024-09-13T07:01:34.770 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:34.765+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:34.789 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:34.790 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:34.799 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:01:34.871 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:34.869+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:34.950 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:34.951 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:34.951 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:34.951 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:34.951 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:34.960 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:01:34.972 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:34.965+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:35.073 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:35.069+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:35.074 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:35.074 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:35.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:35.074 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:35.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.074 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:434 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'4, mlcod=26'2)", 2024-09-13T07:01:35.074 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:01:35.074 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.536637434999999, 2024-09-13T07:01:35.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001931945, 2024-09-13T07:01:35.075 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.075 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.075 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:01:35.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:01:35.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535695+0000", 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535709+0000", 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3577999999999999e-05 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535732+0000", 2024-09-13T07:01:35.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3085e-05 2024-09-13T07:01:35.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535742+0000", 2024-09-13T07:01:35.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.048e-05 2024-09-13T07:01:35.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535840+0000", 2024-09-13T07:01:35.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7159999999999998e-05 2024-09-13T07:01:35.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535871+0000", 2024-09-13T07:01:35.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1510999999999997e-05 2024-09-13T07:01:35.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537572+0000", 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017008279999999999 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537629+0000", 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6520000000000001e-05 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:435 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'5, mlcod=26'2)", 2024-09-13T07:01:35.079 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:35.080 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.535505729, 2024-09-13T07:01:35.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017126769999999999, 2024-09-13T07:01:35.080 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.080 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.080 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:35.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:35.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:35.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:35.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536835+0000", 2024-09-13T07:01:35.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4939999999999997e-06 2024-09-13T07:01:35.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536854+0000", 2024-09-13T07:01:35.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8655999999999999e-05 2024-09-13T07:01:35.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536862+0000", 2024-09-13T07:01:35.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6570000000000003e-06 2024-09-13T07:01:35.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536926+0000", 2024-09-13T07:01:35.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4037999999999999e-05 2024-09-13T07:01:35.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536958+0000", 2024-09-13T07:01:35.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1584000000000001e-05 2024-09-13T07:01:35.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538493+0000", 2024-09-13T07:01:35.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00153513 2024-09-13T07:01:35.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538541+0000", 2024-09-13T07:01:35.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7883999999999998e-05 2024-09-13T07:01:35.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.084 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.085 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:483 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'24, mlcod=26'21)", 2024-09-13T07:01:35.085 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:01:35.085 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.457027099999999, 2024-09-13T07:01:35.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018371780000000001, 2024-09-13T07:01:35.085 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.085 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.085 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:01:35.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615306+0000", 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615322+0000", 2024-09-13T07:01:35.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5519000000000002e-05 2024-09-13T07:01:35.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615335+0000", 2024-09-13T07:01:35.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3854e-05 2024-09-13T07:01:35.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615345+0000", 2024-09-13T07:01:35.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4930000000000004e-06 2024-09-13T07:01:35.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615431+0000", 2024-09-13T07:01:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6346000000000003e-05 2024-09-13T07:01:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615480+0000", 2024-09-13T07:01:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8303999999999997e-05 2024-09-13T07:01:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617102+0000", 2024-09-13T07:01:35.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016225250000000001 2024-09-13T07:01:35.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617144+0000", 2024-09-13T07:01:35.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1936e-05 2024-09-13T07:01:35.089 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.089 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.089 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.089 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:488 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'26, mlcod=26'24)", 2024-09-13T07:01:35.090 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.624174+0000", 2024-09-13T07:01:35.090 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.448159909999999, 2024-09-13T07:01:35.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001630374, 2024-09-13T07:01:35.090 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.090 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.090 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624174+0000", 2024-09-13T07:01:35.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624174+0000", 2024-09-13T07:01:35.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624172+0000", 2024-09-13T07:01:35.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:35.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624193+0000", 2024-09-13T07:01:35.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0475999999999998e-05 2024-09-13T07:01:35.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624210+0000", 2024-09-13T07:01:35.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7521999999999999e-05 2024-09-13T07:01:35.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624218+0000", 2024-09-13T07:01:35.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9580000000000007e-06 2024-09-13T07:01:35.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624289+0000", 2024-09-13T07:01:35.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0482999999999996e-05 2024-09-13T07:01:35.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624323+0000", 2024-09-13T07:01:35.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3643000000000003e-05 2024-09-13T07:01:35.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.625746+0000", 2024-09-13T07:01:35.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014238009999999999 2024-09-13T07:01:35.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.625804+0000", 2024-09-13T07:01:35.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8087999999999997e-05 2024-09-13T07:01:35.095 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.095 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.095 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.096 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:494 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'29, mlcod=26'27)", 2024-09-13T07:01:35.096 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.636753+0000", 2024-09-13T07:01:35.096 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.435581486, 2024-09-13T07:01:35.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001628083, 2024-09-13T07:01:35.096 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.096 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.096 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636753+0000", 2024-09-13T07:01:35.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636753+0000", 2024-09-13T07:01:35.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636751+0000", 2024-09-13T07:01:35.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:35.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636761+0000", 2024-09-13T07:01:35.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0118e-05 2024-09-13T07:01:35.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636780+0000", 2024-09-13T07:01:35.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9313e-05 2024-09-13T07:01:35.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636792+0000", 2024-09-13T07:01:35.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1674e-05 2024-09-13T07:01:35.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636872+0000", 2024-09-13T07:01:35.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0098000000000005e-05 2024-09-13T07:01:35.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636915+0000", 2024-09-13T07:01:35.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3037000000000001e-05 2024-09-13T07:01:35.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.638323+0000", 2024-09-13T07:01:35.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001407752 2024-09-13T07:01:35.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.638381+0000", 2024-09-13T07:01:35.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7578999999999999e-05 2024-09-13T07:01:35.104 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.104 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.104 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.104 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:518 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'38, mlcod=26'36)", 2024-09-13T07:01:35.104 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:01:35.104 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7474503620000004, 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018201179999999999, 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324883+0000", 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324891+0000", 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1499999999999999e-06 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324924+0000", 2024-09-13T07:01:35.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2601999999999997e-05 2024-09-13T07:01:35.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324942+0000", 2024-09-13T07:01:35.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7796999999999999e-05 2024-09-13T07:01:35.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325060+0000", 2024-09-13T07:01:35.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011786900000000001 2024-09-13T07:01:35.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325123+0000", 2024-09-13T07:01:35.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3474999999999998e-05 2024-09-13T07:01:35.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326620+0000", 2024-09-13T07:01:35.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014968830000000001 2024-09-13T07:01:35.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326704+0000", 2024-09-13T07:01:35.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3716999999999998e-05 2024-09-13T07:01:35.109 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.109 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.109 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.110 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:597 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'39, mlcod=26'37)", 2024-09-13T07:01:35.110 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:01:35.110 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.6306230339999992, 2024-09-13T07:01:35.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017959829999999999, 2024-09-13T07:01:35.110 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.110 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.110 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:01:35.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:01:35.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441709+0000", 2024-09-13T07:01:35.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:35.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441728+0000", 2024-09-13T07:01:35.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9627999999999999e-05 2024-09-13T07:01:35.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441765+0000", 2024-09-13T07:01:35.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6559e-05 2024-09-13T07:01:35.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441783+0000", 2024-09-13T07:01:35.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7691000000000001e-05 2024-09-13T07:01:35.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441904+0000", 2024-09-13T07:01:35.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121184 2024-09-13T07:01:35.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441966+0000", 2024-09-13T07:01:35.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2428999999999995e-05 2024-09-13T07:01:35.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443448+0000", 2024-09-13T07:01:35.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00148197 2024-09-13T07:01:35.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443507+0000", 2024-09-13T07:01:35.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8699000000000002e-05 2024-09-13T07:01:35.115 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.115 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.115 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.115 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:01:35.115 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:01:35.115 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.0157427400000003, 2024-09-13T07:01:35.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:01:35.116 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.116 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.116 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:01:35.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:01:35.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:01:35.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:35.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:01:35.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:01:35.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:01:35.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:01:35.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:01:35.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:01:35.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:01:35.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:01:35.121 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.121 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.121 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.121 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1548 3.4 e27/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 27'52, mlcod=27'50)", 2024-09-13T07:01:35.121 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:01:35.121 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.9301212040000006, 2024-09-13T07:01:35.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017825779999999999, 2024-09-13T07:01:35.121 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142211+0000", 2024-09-13T07:01:35.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:35.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142232+0000", 2024-09-13T07:01:35.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0888000000000001e-05 2024-09-13T07:01:35.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142268+0000", 2024-09-13T07:01:35.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6261999999999999e-05 2024-09-13T07:01:35.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142285+0000", 2024-09-13T07:01:35.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7640000000000001e-05 2024-09-13T07:01:35.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142381+0000", 2024-09-13T07:01:35.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5171000000000001e-05 2024-09-13T07:01:35.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142450+0000", 2024-09-13T07:01:35.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9778999999999997e-05 2024-09-13T07:01:35.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143953+0000", 2024-09-13T07:01:35.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001503129 2024-09-13T07:01:35.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143995+0000", 2024-09-13T07:01:35.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1878000000000001e-05 2024-09-13T07:01:35.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.126 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.126 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1550 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'14, mlcod=27'12)", 2024-09-13T07:01:35.126 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.9294734929999997, 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002750452, 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:01:35.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142859+0000", 2024-09-13T07:01:35.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:35.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142874+0000", 2024-09-13T07:01:35.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5149e-05 2024-09-13T07:01:35.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142911+0000", 2024-09-13T07:01:35.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6439000000000001e-05 2024-09-13T07:01:35.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142926+0000", 2024-09-13T07:01:35.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5079000000000001e-05 2024-09-13T07:01:35.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143012+0000", 2024-09-13T07:01:35.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6230000000000006e-05 2024-09-13T07:01:35.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143063+0000", 2024-09-13T07:01:35.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1369000000000002e-05 2024-09-13T07:01:35.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145539+0000", 2024-09-13T07:01:35.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024753420000000002 2024-09-13T07:01:35.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145611+0000", 2024-09-13T07:01:35.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2392999999999996e-05 2024-09-13T07:01:35.131 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.131 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.131 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.132 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1668 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'17, mlcod=27'15)", 2024-09-13T07:01:35.132 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:35.132 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.7310580309999999, 2024-09-13T07:01:35.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033976620000000001, 2024-09-13T07:01:35.132 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.132 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.132 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:35.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:35.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:35.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341281+0000", 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8109999999999999e-06 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341298+0000", 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6575e-05 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341308+0000", 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0369e-05 2024-09-13T07:01:35.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341434+0000", 2024-09-13T07:01:35.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125454 2024-09-13T07:01:35.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341453+0000", 2024-09-13T07:01:35.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9159000000000001e-05 2024-09-13T07:01:35.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344638+0000", 2024-09-13T07:01:35.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031851750000000002 2024-09-13T07:01:35.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344674+0000", 2024-09-13T07:01:35.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5410999999999997e-05 2024-09-13T07:01:35.136 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.136 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.136 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.137 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1666 3.4 e28/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 28'55, mlcod=27'53)", 2024-09-13T07:01:35.137 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:01:35.137 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.7310043589999999, 2024-09-13T07:01:35.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001805809, 2024-09-13T07:01:35.137 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.137 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.137 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:01:35.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:01:35.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341329+0000", 2024-09-13T07:01:35.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:01:35.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341349+0000", 2024-09-13T07:01:35.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.999e-05 2024-09-13T07:01:35.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341357+0000", 2024-09-13T07:01:35.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0620000000000003e-06 2024-09-13T07:01:35.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341361+0000", 2024-09-13T07:01:35.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:01:35.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341710+0000", 2024-09-13T07:01:35.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034873400000000001 2024-09-13T07:01:35.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341736+0000", 2024-09-13T07:01:35.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5772e-05 2024-09-13T07:01:35.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343097+0000", 2024-09-13T07:01:35.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013615319999999999 2024-09-13T07:01:35.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343135+0000", 2024-09-13T07:01:35.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8111999999999997e-05 2024-09-13T07:01:35.141 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.141 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.141 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1800 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'20, mlcod=28'18)", 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.5286338930000003, 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001874398, 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:01:35.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543699+0000", 2024-09-13T07:01:35.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:35.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543713+0000", 2024-09-13T07:01:35.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4786e-05 2024-09-13T07:01:35.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543737+0000", 2024-09-13T07:01:35.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3626e-05 2024-09-13T07:01:35.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543752+0000", 2024-09-13T07:01:35.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.489e-05 2024-09-13T07:01:35.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543832+0000", 2024-09-13T07:01:35.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0396999999999996e-05 2024-09-13T07:01:35.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543994+0000", 2024-09-13T07:01:35.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016129100000000001 2024-09-13T07:01:35.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545471+0000", 2024-09-13T07:01:35.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014771960000000001 2024-09-13T07:01:35.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545575+0000", 2024-09-13T07:01:35.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103732 2024-09-13T07:01:35.146 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.146 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.146 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.146 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1882 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'23, mlcod=28'21)", 2024-09-13T07:01:35.146 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:01:35.146 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.4330009120000007, 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001767172, 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639332+0000", 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639343+0000", 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1436e-05 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639361+0000", 2024-09-13T07:01:35.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7915e-05 2024-09-13T07:01:35.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639370+0000", 2024-09-13T07:01:35.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3189999999999995e-06 2024-09-13T07:01:35.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639425+0000", 2024-09-13T07:01:35.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4917999999999998e-05 2024-09-13T07:01:35.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639454+0000", 2024-09-13T07:01:35.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9862999999999999e-05 2024-09-13T07:01:35.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641059+0000", 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016048950000000001 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641100+0000", 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0883999999999997e-05 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3069 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'33, mlcod=31'31)", 2024-09-13T07:01:35.151 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:01:35.152 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9098611959999996, 2024-09-13T07:01:35.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018597010000000001, 2024-09-13T07:01:35.152 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.152 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.152 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:01:35.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162471+0000", 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162520+0000", 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8466e-05 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.154 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162546+0000", 2024-09-13T07:01:35.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6089999999999999e-05 2024-09-13T07:01:35.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.154 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.154 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162562+0000", 2024-09-13T07:01:35.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5962000000000001e-05 2024-09-13T07:01:35.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.154 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.154 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162646+0000", 2024-09-13T07:01:35.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3985000000000003e-05 2024-09-13T07:01:35.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162691+0000", 2024-09-13T07:01:35.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5785999999999998e-05 2024-09-13T07:01:35.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164265+0000", 2024-09-13T07:01:35.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015740890000000001 2024-09-13T07:01:35.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164333+0000", 2024-09-13T07:01:35.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7014999999999994e-05 2024-09-13T07:01:35.156 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.156 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.156 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.156 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3121 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'36, mlcod=31'34)", 2024-09-13T07:01:35.156 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:01:35.157 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.7959148090000001, 2024-09-13T07:01:35.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019657780000000001, 2024-09-13T07:01:35.157 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.157 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.157 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:01:35.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276417+0000", 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276497+0000", 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9729999999999997e-05 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276537+0000", 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0102999999999999e-05 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276553+0000", 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6518999999999999e-05 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276662+0000", 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109252 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276729+0000", 2024-09-13T07:01:35.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7056000000000003e-05 2024-09-13T07:01:35.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278270+0000", 2024-09-13T07:01:35.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015408819999999999 2024-09-13T07:01:35.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278385+0000", 2024-09-13T07:01:35.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114613 2024-09-13T07:01:35.160 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.161 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.161 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.161 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3390 3.0 e32/22 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:head v 32'45, mlcod=31'42)", 2024-09-13T07:01:35.161 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:01:35.161 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.5768264619999997, 2024-09-13T07:01:35.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002246587, 2024-09-13T07:01:35.161 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.161 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.162 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:01:35.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:01:35.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495507+0000", 2024-09-13T07:01:35.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495530+0000", 2024-09-13T07:01:35.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2592999999999999e-05 2024-09-13T07:01:35.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495550+0000", 2024-09-13T07:01:35.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0152999999999998e-05 2024-09-13T07:01:35.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495562+0000", 2024-09-13T07:01:35.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2336e-05 2024-09-13T07:01:35.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495662+0000", 2024-09-13T07:01:35.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100317 2024-09-13T07:01:35.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495717+0000", 2024-09-13T07:01:35.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4304e-05 2024-09-13T07:01:35.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497707+0000", 2024-09-13T07:01:35.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019906849999999999 2024-09-13T07:01:35.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497754+0000", 2024-09-13T07:01:35.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6715999999999998e-05 2024-09-13T07:01:35.166 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.166 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.166 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.166 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3399 3.4 e32/22 3:22863a52:::rbd_data.11945d91e99c.0000000000000013:head v 32'65, mlcod=31'62)", 2024-09-13T07:01:35.166 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:01:35.166 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.5728897899999996, 2024-09-13T07:01:35.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028830779999999999, 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499442+0000", 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499459+0000", 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7042e-05 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499493+0000", 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3788000000000002e-05 2024-09-13T07:01:35.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499512+0000", 2024-09-13T07:01:35.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9055999999999998e-05 2024-09-13T07:01:35.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499608+0000", 2024-09-13T07:01:35.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6168000000000005e-05 2024-09-13T07:01:35.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499658+0000", 2024-09-13T07:01:35.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9469999999999999e-05 2024-09-13T07:01:35.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502277+0000", 2024-09-13T07:01:35.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026188639999999998 2024-09-13T07:01:35.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502327+0000", 2024-09-13T07:01:35.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0578999999999998e-05 2024-09-13T07:01:35.171 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.171 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.171 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.171 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3395 3.0 e32/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 32'47, mlcod=31'43)", 2024-09-13T07:01:35.171 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:35.171 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.5727917050000002, 2024-09-13T07:01:35.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027505870000000001, 2024-09-13T07:01:35.172 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.172 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.172 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:35.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:35.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:35.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499560+0000", 2024-09-13T07:01:35.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8416000000000001e-05 2024-09-13T07:01:35.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499576+0000", 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5602000000000001e-05 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499587+0000", 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1112e-05 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499654+0000", 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7164999999999998e-05 2024-09-13T07:01:35.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499674+0000", 2024-09-13T07:01:35.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9638000000000001e-05 2024-09-13T07:01:35.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502210+0000", 2024-09-13T07:01:35.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025367100000000002 2024-09-13T07:01:35.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502293+0000", 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2607000000000004e-05 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:4534 3.0 e32/22 3:00392e9e:::rbd_data.11944cdc0656.000000000000000c:head v 32'51, mlcod=32'49)", 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.161090519, 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017597330000000001, 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:01:35.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:01:35.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911242+0000", 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911306+0000", 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4480999999999995e-05 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911338+0000", 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1733000000000003e-05 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911351+0000", 2024-09-13T07:01:35.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3241000000000001e-05 2024-09-13T07:01:35.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911455+0000", 2024-09-13T07:01:35.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103723 2024-09-13T07:01:35.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911495+0000", 2024-09-13T07:01:35.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0683000000000003e-05 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.912933+0000", 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001437219 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.913003+0000", 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0629000000000004e-05 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.180 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.181 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.181 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:35.182 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:01:35.183 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:35.181+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:35.283 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:35.277+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:35.300 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:35.301 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:35.310 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:01:35.384 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:35.381+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:35.393 INFO:teuthology.orchestra.run.smithi118.stdout:629 write 0x6a24248 thru 0x6a2c0fc (0x7eb5 bytes) 2024-09-13T07:01:35.395 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:01:35.406 INFO:teuthology.orchestra.run.smithi118.stdout:631 writesame 0x2574cfc thru 0x25794a7 data_size 0x21(0x47ac bytes) 2024-09-13T07:01:35.414 INFO:teuthology.orchestra.run.smithi118.stdout:632 read 0x40afb29 thru 0x40b4482 (0x495a bytes) 2024-09-13T07:01:35.414 INFO:teuthology.orchestra.run.smithi118.stdout:633 write 0x6899b55 thru 0x689fa30 (0x5edc bytes) 2024-09-13T07:01:35.417 INFO:teuthology.orchestra.run.smithi118.stdout:634 read 0x5871be6 thru 0x587fb43 (0xdf5e bytes) 2024-09-13T07:01:35.417 INFO:teuthology.orchestra.run.smithi118.stdout:635 clone 8 order 25 su 131072 sc 8 2024-09-13T07:01:35.435 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:35.435 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:35.435 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:35.435 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:35.435 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:35.446 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:01:35.490 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:35.489+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:35.586 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:35.585+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:35.591 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:35.591 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:35.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:35.591 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:35.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.591 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:441 3.b 3:d465e8df:::rbd_data.1194d22ad098.0000000000000036:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:35.591 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:35.592 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 11.047968615, 2024-09-13T07:01:35.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012043093, 2024-09-13T07:01:35.592 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.592 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.592 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.592 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.592 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.592 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 441 2024-09-13T07:01:35.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.592 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541122+0000", 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541136+0000", 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3899e-05 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541163+0000", 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.694e-05 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541176+0000", 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2802e-05 2024-09-13T07:01:35.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541257+0000", 2024-09-13T07:01:35.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1018000000000003e-05 2024-09-13T07:01:35.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541414+0000", 2024-09-13T07:01:35.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015745000000000001 2024-09-13T07:01:35.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:35.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.544450+0000", 2024-09-13T07:01:35.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030357209999999999 2024-09-13T07:01:35.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:35.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553002+0000", 2024-09-13T07:01:35.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085521020000000007 2024-09-13T07:01:35.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:35.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553022+0000", 2024-09-13T07:01:35.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9694000000000002e-05 2024-09-13T07:01:35.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553103+0000", 2024-09-13T07:01:35.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1008999999999996e-05 2024-09-13T07:01:35.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553168+0000", 2024-09-13T07:01:35.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5010000000000003e-05 2024-09-13T07:01:35.597 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.597 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.597 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1155 3.7 3:e095a6bd:::rbd_data.11943a018f34.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7563293519999998, 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022798902999999999, 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1155 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832762+0000", 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832776+0000", 2024-09-13T07:01:35.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3828e-05 2024-09-13T07:01:35.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832806+0000", 2024-09-13T07:01:35.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0159e-05 2024-09-13T07:01:35.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832820+0000", 2024-09-13T07:01:35.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.384e-05 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832904+0000", 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4506999999999996e-05 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.833121+0000", 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000217223 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:35.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.839072+0000", 2024-09-13T07:01:35.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0059509849999999998 2024-09-13T07:01:35.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:35.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855489+0000", 2024-09-13T07:01:35.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016416907000000001 2024-09-13T07:01:35.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:35.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855503+0000", 2024-09-13T07:01:35.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3658999999999999e-05 2024-09-13T07:01:35.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855535+0000", 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1696000000000003e-05 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855563+0000", 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8578999999999999e-05 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.603 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1200 3.7 3:e095a6bd:::rbd_data.11943a018f34.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:35.604 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:01:35.604 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.6099904019999993, 2024-09-13T07:01:35.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018736263, 2024-09-13T07:01:35.604 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.604 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.604 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.604 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.604 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.604 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1200 2024-09-13T07:01:35.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.604 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979100+0000", 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979115+0000", 2024-09-13T07:01:35.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4823e-05 2024-09-13T07:01:35.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979149+0000", 2024-09-13T07:01:35.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3772999999999997e-05 2024-09-13T07:01:35.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979163+0000", 2024-09-13T07:01:35.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3845999999999999e-05 2024-09-13T07:01:35.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979333+0000", 2024-09-13T07:01:35.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000169622 2024-09-13T07:01:35.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979603+0000", 2024-09-13T07:01:35.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00027078999999999997 2024-09-13T07:01:35.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:35.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.986017+0000", 2024-09-13T07:01:35.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064139590000000003 2024-09-13T07:01:35.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:35.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997476+0000", 2024-09-13T07:01:35.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011458582 2024-09-13T07:01:35.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:35.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997588+0000", 2024-09-13T07:01:35.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011161 2024-09-13T07:01:35.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997732+0000", 2024-09-13T07:01:35.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000144 2024-09-13T07:01:35.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997839+0000", 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107892 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1531 3.c e27/22 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head v 27'9, mlcod=27'7)", 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.5145622450000005, 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039127038000000003, 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074530+0000", 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:35.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099434+0000", 2024-09-13T07:01:35.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.024903926999999999 2024-09-13T07:01:35.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099459+0000", 2024-09-13T07:01:35.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5384999999999998e-05 2024-09-13T07:01:35.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099469+0000", 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.024e-05 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099566+0000", 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6533999999999994e-05 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099600+0000", 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4007000000000002e-05 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113477+0000", 2024-09-13T07:01:35.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01387741 2024-09-13T07:01:35.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113658+0000", 2024-09-13T07:01:35.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000180953 2024-09-13T07:01:35.615 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.615 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.615 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.615 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3187 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:01:35.615 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.410269+0000", 2024-09-13T07:01:35.615 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 6.17882482, 2024-09-13T07:01:35.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012395160000000001, 2024-09-13T07:01:35.615 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.615 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:35.615 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.615 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3187 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410269+0000", 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410269+0000", 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410266+0000", 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410283+0000", 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7527999999999999e-05 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410317+0000", 2024-09-13T07:01:35.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3726000000000002e-05 2024-09-13T07:01:35.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410340+0000", 2024-09-13T07:01:35.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3252e-05 2024-09-13T07:01:35.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410425+0000", 2024-09-13T07:01:35.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4338999999999993e-05 2024-09-13T07:01:35.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410528+0000", 2024-09-13T07:01:35.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103058 2024-09-13T07:01:35.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:01:35.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.412544+0000", 2024-09-13T07:01:35.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020164639999999999 2024-09-13T07:01:35.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:35.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422499+0000", 2024-09-13T07:01:35.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099549960000000007 2024-09-13T07:01:35.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:35.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422521+0000", 2024-09-13T07:01:35.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1433e-05 2024-09-13T07:01:35.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:35.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422604+0000", 2024-09-13T07:01:35.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2820999999999999e-05 2024-09-13T07:01:35.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422664+0000", 2024-09-13T07:01:35.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0362999999999998e-05 2024-09-13T07:01:35.627 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.627 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.628 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.628 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3762 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.628 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.653608+0000", 2024-09-13T07:01:35.628 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9354853350000001, 2024-09-13T07:01:35.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009039765, 2024-09-13T07:01:35.629 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.629 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.630 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3762 2024-09-13T07:01:35.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.630 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.653608+0000", 2024-09-13T07:01:35.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.653608+0000", 2024-09-13T07:01:35.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.653608+0000", 2024-09-13T07:01:35.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.653610+0000", 2024-09-13T07:01:35.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4140000000000001e-06 2024-09-13T07:01:35.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.653615+0000", 2024-09-13T07:01:35.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1869999999999996e-06 2024-09-13T07:01:35.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.653620+0000", 2024-09-13T07:01:35.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4460000000000003e-06 2024-09-13T07:01:35.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.659177+0000", 2024-09-13T07:01:35.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055576289999999997 2024-09-13T07:01:35.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.659247+0000", 2024-09-13T07:01:35.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9740999999999995e-05 2024-09-13T07:01:35.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.662648+0000", 2024-09-13T07:01:35.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003400756 2024-09-13T07:01:35.637 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.638 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.638 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3923 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.638 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654168+0000", 2024-09-13T07:01:35.639 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9349254079999998, 2024-09-13T07:01:35.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010121198, 2024-09-13T07:01:35.639 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.639 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.640 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.640 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.640 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.640 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3923 2024-09-13T07:01:35.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.640 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654168+0000", 2024-09-13T07:01:35.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654168+0000", 2024-09-13T07:01:35.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654168+0000", 2024-09-13T07:01:35.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654171+0000", 2024-09-13T07:01:35.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5899999999999999e-06 2024-09-13T07:01:35.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654181+0000", 2024-09-13T07:01:35.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.781e-06 2024-09-13T07:01:35.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654199+0000", 2024-09-13T07:01:35.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7977000000000001e-05 2024-09-13T07:01:35.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.662544+0000", 2024-09-13T07:01:35.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0083447049999999991 2024-09-13T07:01:35.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.664193+0000", 2024-09-13T07:01:35.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016489129999999999 2024-09-13T07:01:35.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.664289+0000", 2024-09-13T07:01:35.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6781999999999996e-05 2024-09-13T07:01:35.648 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.649 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.649 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.649 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3924 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.649 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654207+0000", 2024-09-13T07:01:35.650 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9348866310000004, 2024-09-13T07:01:35.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010809559, 2024-09-13T07:01:35.650 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.650 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.650 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.650 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.650 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.651 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3924 2024-09-13T07:01:35.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.651 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654207+0000", 2024-09-13T07:01:35.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654207+0000", 2024-09-13T07:01:35.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654206+0000", 2024-09-13T07:01:35.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654211+0000", 2024-09-13T07:01:35.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:01:35.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654221+0000", 2024-09-13T07:01:35.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0351e-05 2024-09-13T07:01:35.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654230+0000", 2024-09-13T07:01:35.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2749999999999995e-06 2024-09-13T07:01:35.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.664829+0000", 2024-09-13T07:01:35.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010599641999999999 2024-09-13T07:01:35.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.664909+0000", 2024-09-13T07:01:35.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9536999999999994e-05 2024-09-13T07:01:35.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665016+0000", 2024-09-13T07:01:35.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010778200000000001 2024-09-13T07:01:35.659 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.659 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.659 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.660 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3970 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.660 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654237+0000", 2024-09-13T07:01:35.660 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9348565029999998, 2024-09-13T07:01:35.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010974097, 2024-09-13T07:01:35.661 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.661 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.661 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.661 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.661 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.661 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3970 2024-09-13T07:01:35.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.662 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654237+0000", 2024-09-13T07:01:35.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654237+0000", 2024-09-13T07:01:35.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654236+0000", 2024-09-13T07:01:35.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654240+0000", 2024-09-13T07:01:35.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.997e-06 2024-09-13T07:01:35.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654250+0000", 2024-09-13T07:01:35.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7569999999999999e-06 2024-09-13T07:01:35.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654258+0000", 2024-09-13T07:01:35.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2379999999999997e-06 2024-09-13T07:01:35.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665043+0000", 2024-09-13T07:01:35.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010784956 2024-09-13T07:01:35.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665127+0000", 2024-09-13T07:01:35.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3856000000000005e-05 2024-09-13T07:01:35.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665211+0000", 2024-09-13T07:01:35.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3923000000000003e-05 2024-09-13T07:01:35.669 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.670 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.670 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.670 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3971 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.670 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654265+0000", 2024-09-13T07:01:35.671 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9348281739999997, 2024-09-13T07:01:35.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011136940999999999, 2024-09-13T07:01:35.671 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.671 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.671 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.671 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.672 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.672 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3971 2024-09-13T07:01:35.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.672 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654265+0000", 2024-09-13T07:01:35.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654265+0000", 2024-09-13T07:01:35.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654265+0000", 2024-09-13T07:01:35.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654277+0000", 2024-09-13T07:01:35.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1673e-05 2024-09-13T07:01:35.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654287+0000", 2024-09-13T07:01:35.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0137e-05 2024-09-13T07:01:35.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654295+0000", 2024-09-13T07:01:35.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2700000000000004e-06 2024-09-13T07:01:35.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665237+0000", 2024-09-13T07:01:35.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010941973000000001 2024-09-13T07:01:35.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665316+0000", 2024-09-13T07:01:35.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9200000000000001e-05 2024-09-13T07:01:35.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665402+0000", 2024-09-13T07:01:35.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6193999999999995e-05 2024-09-13T07:01:35.679 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.679 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.679 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.680 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3972 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 10485760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.680 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654303+0000", 2024-09-13T07:01:35.680 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9347909850000002, 2024-09-13T07:01:35.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011322693, 2024-09-13T07:01:35.681 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.681 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.681 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.681 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.681 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.682 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3972 2024-09-13T07:01:35.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.682 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654303+0000", 2024-09-13T07:01:35.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654303+0000", 2024-09-13T07:01:35.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654302+0000", 2024-09-13T07:01:35.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654312+0000", 2024-09-13T07:01:35.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9250000000000007e-06 2024-09-13T07:01:35.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654322+0000", 2024-09-13T07:01:35.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0431000000000001e-05 2024-09-13T07:01:35.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654330+0000", 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.072e-06 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665429+0000", 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011098182 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665534+0000", 2024-09-13T07:01:35.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105561 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665625+0000", 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1166000000000001e-05 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3973 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654338+0000", 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9347556140000002, 2024-09-13T07:01:35.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011466931, 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3973 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654338+0000", 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654338+0000", 2024-09-13T07:01:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654337+0000", 2024-09-13T07:01:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654341+0000", 2024-09-13T07:01:35.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7500000000000001e-06 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654351+0000", 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7489999999999993e-06 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654359+0000", 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0830000000000004e-06 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665650+0000", 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011291066000000001 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665722+0000", 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2442999999999997e-05 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665805+0000", 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2464000000000002e-05 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.691 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3974 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.691 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654366+0000", 2024-09-13T07:01:35.691 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9347272440000003, 2024-09-13T07:01:35.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011612753, 2024-09-13T07:01:35.691 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.691 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.691 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.691 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.691 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.691 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3974 2024-09-13T07:01:35.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654366+0000", 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654366+0000", 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654366+0000", 2024-09-13T07:01:35.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654369+0000", 2024-09-13T07:01:35.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7019999999999999e-06 2024-09-13T07:01:35.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654379+0000", 2024-09-13T07:01:35.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9510000000000001e-06 2024-09-13T07:01:35.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654387+0000", 2024-09-13T07:01:35.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0169999999999996e-06 2024-09-13T07:01:35.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665829+0000", 2024-09-13T07:01:35.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011441251 2024-09-13T07:01:35.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665895+0000", 2024-09-13T07:01:35.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5969000000000005e-05 2024-09-13T07:01:35.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665979+0000", 2024-09-13T07:01:35.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4464999999999999e-05 2024-09-13T07:01:35.695 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.695 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.695 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.695 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3975 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.695 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:35.695 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9346982969999997, 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011993248, 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3975 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654398+0000", 2024-09-13T07:01:35.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8619999999999996e-06 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654408+0000", 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8209999999999997e-06 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654416+0000", 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0180000000000001e-06 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666002+0000", 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011586008 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666074+0000", 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1500000000000003e-05 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666388+0000", 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00031467300000000001 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.699 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.700 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4022 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.700 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:01:35.700 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9346698010000001, 2024-09-13T07:01:35.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014992557, 2024-09-13T07:01:35.700 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.700 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.700 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4022 2024-09-13T07:01:35.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.701 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:01:35.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:01:35.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654423+0000", 2024-09-13T07:01:35.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654427+0000", 2024-09-13T07:01:35.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0960000000000003e-06 2024-09-13T07:01:35.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654437+0000", 2024-09-13T07:01:35.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7070000000000004e-06 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654445+0000", 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9049999999999997e-06 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666423+0000", 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011978223 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666521+0000", 2024-09-13T07:01:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7832999999999997e-05 2024-09-13T07:01:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669416+0000", 2024-09-13T07:01:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002895393 2024-09-13T07:01:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.704 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4023 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9346409150000001, 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015452399, 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4023 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654452+0000", 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654456+0000", 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7570000000000002e-06 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654465+0000", 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7140000000000005e-06 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654486+0000", 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0290000000000001e-05 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669455+0000", 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014969651 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669722+0000", 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026677899999999997 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669905+0000", 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018285999999999999 2024-09-13T07:01:35.708 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4024 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9346001150000003, 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015682979, 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.709 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4024 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654497+0000", 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4800000000000003e-06 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654507+0000", 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8749999999999995e-06 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654515+0000", 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0350000000000001e-06 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669936+0000", 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015420916 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670030+0000", 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3712e-05 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670176+0000", 2024-09-13T07:01:35.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000146613 2024-09-13T07:01:35.713 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.713 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.713 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.713 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4025 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.713 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:35.713 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.934571107, 2024-09-13T07:01:35.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016683559000000001, 2024-09-13T07:01:35.713 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.713 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.714 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.714 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.714 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.714 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4025 2024-09-13T07:01:35.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.714 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:35.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654526+0000", 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654535+0000", 2024-09-13T07:01:35.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4199999999999996e-06 2024-09-13T07:01:35.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654543+0000", 2024-09-13T07:01:35.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:01:35.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670307+0000", 2024-09-13T07:01:35.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015763482999999998 2024-09-13T07:01:35.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670399+0000", 2024-09-13T07:01:35.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2068e-05 2024-09-13T07:01:35.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671206+0000", 2024-09-13T07:01:35.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000807352 2024-09-13T07:01:35.717 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.717 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.717 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4026 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 13238272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9345433779999999, 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016876344000000001, 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4026 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654554+0000", 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1119999999999998e-06 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654564+0000", 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0327e-05 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654571+0000", 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3440000000000003e-06 2024-09-13T07:01:35.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671239+0000", 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016668039999999999 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671327+0000", 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7818999999999994e-05 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671427+0000", 2024-09-13T07:01:35.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9324e-05 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4027 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9345146159999995, 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017835597000000002, 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:35.722 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4027 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:35.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654578+0000", 2024-09-13T07:01:35.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:35.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:35.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654590+0000", 2024-09-13T07:01:35.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1595e-05 2024-09-13T07:01:35.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:35.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654601+0000", 2024-09-13T07:01:35.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0827e-05 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654609+0000", 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4919999999999993e-06 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672157+0000", 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017547770000000001 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672234+0000", 2024-09-13T07:01:35.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7509999999999995e-05 2024-09-13T07:01:35.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:35.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:35.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:35.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672415+0000", 2024-09-13T07:01:35.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018005500000000001 2024-09-13T07:01:35.726 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.726 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.726 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.726 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:35.726 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:35.726 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:35.728 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:01:35.728 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:35.725+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:35.829 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:35.825+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:35.846 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:35.846 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:35.846 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:35.846 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:35.856 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:01:35.881 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:01:35.930 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:35.929+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:35.965 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:35.965 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:35.965 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:35.965 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:35.965 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:35.975 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:01:36.030 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:36.029+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:36.033 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:36.056 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:36.078 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:36.084 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:36.084 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:36.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:36.084 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:36.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.084 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1501 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.084 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023150+0000", 2024-09-13T07:01:36.084 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056499345000001, 2024-09-13T07:01:36.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037513666000000001, 2024-09-13T07:01:36.084 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.084 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.085 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.085 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.085 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1501 2024-09-13T07:01:36.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.085 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023150+0000", 2024-09-13T07:01:36.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023150+0000", 2024-09-13T07:01:36.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023149+0000", 2024-09-13T07:01:36.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023153+0000", 2024-09-13T07:01:36.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.58e-06 2024-09-13T07:01:36.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023161+0000", 2024-09-13T07:01:36.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0730000000000005e-06 2024-09-13T07:01:36.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023167+0000", 2024-09-13T07:01:36.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4300000000000003e-06 2024-09-13T07:01:36.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060527+0000", 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037360021 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060586+0000", 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8891999999999998e-05 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060664+0000", 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7210000000000001e-05 2024-09-13T07:01:36.088 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.089 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.089 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1502 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.089 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023194+0000", 2024-09-13T07:01:36.089 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056455281, 2024-09-13T07:01:36.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037692165999999999, 2024-09-13T07:01:36.089 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.089 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.090 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1502 2024-09-13T07:01:36.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.090 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023194+0000", 2024-09-13T07:01:36.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023194+0000", 2024-09-13T07:01:36.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023193+0000", 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023197+0000", 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0040000000000001e-06 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023206+0000", 2024-09-13T07:01:36.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.969e-06 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023213+0000", 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2250000000000003e-06 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060721+0000", 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037508138000000003 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060812+0000", 2024-09-13T07:01:36.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0074000000000005e-05 2024-09-13T07:01:36.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060886+0000", 2024-09-13T07:01:36.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4541000000000003e-05 2024-09-13T07:01:36.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.093 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.093 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1503 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.093 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023219+0000", 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056429751, 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037827616000000001, 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1503 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023219+0000", 2024-09-13T07:01:36.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023219+0000", 2024-09-13T07:01:36.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023219+0000", 2024-09-13T07:01:36.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023222+0000", 2024-09-13T07:01:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1980000000000001e-06 2024-09-13T07:01:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023230+0000", 2024-09-13T07:01:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9650000000000008e-06 2024-09-13T07:01:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023239+0000", 2024-09-13T07:01:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8759999999999994e-06 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060923+0000", 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037683662 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060965+0000", 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2001e-05 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061047+0000", 2024-09-13T07:01:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2400999999999999e-05 2024-09-13T07:01:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1504 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023246+0000", 2024-09-13T07:01:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056403674, 2024-09-13T07:01:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038032028000000002, 2024-09-13T07:01:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1504 2024-09-13T07:01:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023246+0000", 2024-09-13T07:01:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023246+0000", 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023245+0000", 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023249+0000", 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.216e-06 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023259+0000", 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4269999999999997e-06 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023265+0000", 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.72e-06 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061077+0000", 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037811895999999998 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061186+0000", 2024-09-13T07:01:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010862900000000001 2024-09-13T07:01:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061278+0000", 2024-09-13T07:01:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1656000000000005e-05 2024-09-13T07:01:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1505 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023272+0000", 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056377615000001, 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038242703000000003, 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1505 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023272+0000", 2024-09-13T07:01:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023272+0000", 2024-09-13T07:01:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023271+0000", 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023274+0000", 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1860000000000001e-06 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023282+0000", 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023289+0000", 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.285e-06 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061345+0000", 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038056363000000003 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061409+0000", 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4004000000000006e-05 2024-09-13T07:01:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061514+0000", 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105324 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1506 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3162112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023297+0000", 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056351722, 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038402842, 2024-09-13T07:01:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1506 2024-09-13T07:01:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023297+0000", 2024-09-13T07:01:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023297+0000", 2024-09-13T07:01:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023297+0000", 2024-09-13T07:01:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023300+0000", 2024-09-13T07:01:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023309+0000", 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2349999999999999e-06 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023315+0000", 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7020000000000004e-06 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061539+0000", 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038223752999999999 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061618+0000", 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9331000000000003e-05 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061700+0000", 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1896000000000003e-05 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1507 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023321+0000", 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056327941999999, 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038581179, 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1507 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023321+0000", 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023321+0000", 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023321+0000", 2024-09-13T07:01:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023325+0000", 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6140000000000002e-06 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023333+0000", 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1780000000000003e-06 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023340+0000", 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6789999999999999e-06 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061730+0000", 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038389853000000002 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061830+0000", 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9810999999999997e-05 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061902+0000", 2024-09-13T07:01:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2702999999999998e-05 2024-09-13T07:01:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1508 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3325952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023348+0000", 2024-09-13T07:01:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056301076, 2024-09-13T07:01:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038783950999999997, 2024-09-13T07:01:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1508 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023348+0000", 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023348+0000", 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023348+0000", 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023351+0000", 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3249999999999999e-06 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023359+0000", 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4910000000000005e-06 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023366+0000", 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.443e-06 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061943+0000", 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038577144000000001 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062027+0000", 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3467999999999994e-05 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062132+0000", 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105553 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1509 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023372+0000", 2024-09-13T07:01:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056277443000001, 2024-09-13T07:01:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039003355000000003, 2024-09-13T07:01:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1509 2024-09-13T07:01:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023372+0000", 2024-09-13T07:01:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023372+0000", 2024-09-13T07:01:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023371+0000", 2024-09-13T07:01:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023375+0000", 2024-09-13T07:01:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4570000000000001e-06 2024-09-13T07:01:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023382+0000", 2024-09-13T07:01:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7409999999999992e-06 2024-09-13T07:01:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023389+0000", 2024-09-13T07:01:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1310000000000003e-06 2024-09-13T07:01:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062165+0000", 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038776418 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062284+0000", 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119491 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062375+0000", 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0607999999999997e-05 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1510 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023395+0000", 2024-09-13T07:01:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056254619000001, 2024-09-13T07:01:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039253549999999998, 2024-09-13T07:01:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1510 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023395+0000", 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023395+0000", 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023394+0000", 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023398+0000", 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5370000000000002e-06 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023406+0000", 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9980000000000003e-06 2024-09-13T07:01:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023412+0000", 2024-09-13T07:01:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5060000000000001e-06 2024-09-13T07:01:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062423+0000", 2024-09-13T07:01:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039010421000000003 2024-09-13T07:01:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062524+0000", 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101282 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062648+0000", 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012431399999999999 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1511 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023418+0000", 2024-09-13T07:01:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056231242999999, 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039437184, 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1511 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023418+0000", 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023418+0000", 2024-09-13T07:01:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023417+0000", 2024-09-13T07:01:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023421+0000", 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2830000000000002e-06 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023429+0000", 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7929999999999998e-06 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023435+0000", 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5080000000000002e-06 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062679+0000", 2024-09-13T07:01:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039243553 2024-09-13T07:01:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062760+0000", 2024-09-13T07:01:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1570999999999995e-05 2024-09-13T07:01:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062855+0000", 2024-09-13T07:01:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4946999999999997e-05 2024-09-13T07:01:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1512 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023441+0000", 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056207691999999, 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039615015000000003, 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1512 2024-09-13T07:01:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023441+0000", 2024-09-13T07:01:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023441+0000", 2024-09-13T07:01:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023441+0000", 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023449+0000", 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700000000000007e-06 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023457+0000", 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2789999999999998e-06 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023465+0000", 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0590000000000003e-06 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062914+0000", 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039448762999999998 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062994+0000", 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9559000000000002e-05 2024-09-13T07:01:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063057+0000", 2024-09-13T07:01:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2786000000000005e-05 2024-09-13T07:01:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1513 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023472+0000", 2024-09-13T07:01:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056177492, 2024-09-13T07:01:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039796658999999998, 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1513 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023472+0000", 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023472+0000", 2024-09-13T07:01:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023471+0000", 2024-09-13T07:01:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023474+0000", 2024-09-13T07:01:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:01:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023483+0000", 2024-09-13T07:01:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6379999999999993e-06 2024-09-13T07:01:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023490+0000", 2024-09-13T07:01:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5930000000000002e-06 2024-09-13T07:01:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063095+0000", 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039605156000000002 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063181+0000", 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6166000000000001e-05 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063268+0000", 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7337999999999996e-05 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1514 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3817472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023496+0000", 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056153205999999, 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039930146, 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1514 2024-09-13T07:01:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023496+0000", 2024-09-13T07:01:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023496+0000", 2024-09-13T07:01:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023495+0000", 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023499+0000", 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4379999999999999e-06 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023507+0000", 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023514+0000", 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4239999999999998e-06 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063286+0000", 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039772620000000002 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063342+0000", 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5770000000000003e-05 2024-09-13T07:01:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063426+0000", 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4011999999999995e-05 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1515 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023520+0000", 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056129647000001, 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040093238000000003, 2024-09-13T07:01:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1515 2024-09-13T07:01:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023520+0000", 2024-09-13T07:01:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023520+0000", 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023519+0000", 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023522+0000", 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9809999999999999e-06 2024-09-13T07:01:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023533+0000", 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0852e-05 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023539+0000", 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.511e-06 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063464+0000", 2024-09-13T07:01:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039924266 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063539+0000", 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5813000000000001e-05 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063613+0000", 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3291e-05 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1516 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023548+0000", 2024-09-13T07:01:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.0561016, 2024-09-13T07:01:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040191618999999998, 2024-09-13T07:01:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1516 2024-09-13T07:01:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023548+0000", 2024-09-13T07:01:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023548+0000", 2024-09-13T07:01:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023547+0000", 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023551+0000", 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1840000000000001e-06 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023559+0000", 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0709999999999995e-06 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023567+0000", 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.306e-06 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063634+0000", 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040067047000000001 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063673+0000", 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8788000000000002e-05 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063739+0000", 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6705000000000006e-05 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1517 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023573+0000", 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056076278000001, 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040350010999999998, 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1517 2024-09-13T07:01:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023573+0000", 2024-09-13T07:01:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023573+0000", 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023572+0000", 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023576+0000", 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8940000000000003e-06 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023584+0000", 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1529999999999997e-06 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023593+0000", 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8589999999999994e-06 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063778+0000", 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040184894999999998 2024-09-13T07:01:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063851+0000", 2024-09-13T07:01:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3114000000000005e-05 2024-09-13T07:01:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063923+0000", 2024-09-13T07:01:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2084999999999998e-05 2024-09-13T07:01:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1518 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 4145152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023602+0000", 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.056047512999999, 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040449005000000003, 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1518 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023602+0000", 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023602+0000", 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023601+0000", 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023604+0000", 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3739999999999998e-06 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023614+0000", 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8230000000000006e-06 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023621+0000", 2024-09-13T07:01:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9870000000000002e-06 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063942+0000", 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040320644000000003 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063985+0000", 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3114000000000001e-05 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.064051+0000", 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5784999999999994e-05 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1524 3.14 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.03488061, 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061190880000000003, 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1524 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044767+0000", 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052670+0000", 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079022450000000004 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052682+0000", 2024-09-13T07:01:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2336e-05 2024-09-13T07:01:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052696+0000", 2024-09-13T07:01:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-13T07:01:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052755+0000", 2024-09-13T07:01:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9298e-05 2024-09-13T07:01:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052947+0000", 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019153900000000001 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.053694+0000", 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074663099999999999 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.102148+0000", 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048454329999999997 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105816+0000", 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036675950000000001 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105921+0000", 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105874 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105959+0000", 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7991000000000003e-05 2024-09-13T07:01:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1533 3.a e27/22 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head v 27'21, mlcod=27'19)", 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.066960+0000", 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.012689494, 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038507995000000003, 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066960+0000", 2024-09-13T07:01:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066960+0000", 2024-09-13T07:01:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066958+0000", 2024-09-13T07:01:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069912+0000", 2024-09-13T07:01:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002953893 2024-09-13T07:01:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069937+0000", 2024-09-13T07:01:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4771000000000001e-05 2024-09-13T07:01:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069950+0000", 2024-09-13T07:01:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3028e-05 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070009+0000", 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9484000000000002e-05 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070042+0000", 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3025000000000003e-05 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105378+0000", 2024-09-13T07:01:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035335706000000001 2024-09-13T07:01:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105468+0000", 2024-09-13T07:01:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9587999999999997e-05 2024-09-13T07:01:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:36.176 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:36.178 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:36.178 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:36.178 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:36.178 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:36.178 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:36.177+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:36.179 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:01:36.193 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:36.196 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:36.196 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:36.196 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:36.196 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:36.196 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:36.196 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:36.196 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:36.197 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:36.278 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:36.273+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:36.295 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:36.295 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:36.305 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:01:36.379 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:36.377+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:36.393 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:01:36.415 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:36.415 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:36.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:01:36.479 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:36.477+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:36.533 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:36.533 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:36.546 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:01:36.567 INFO:teuthology.orchestra.run.smithi118.stdout:leaving image image_client.0-clone7 intact 2024-09-13T07:01:36.580 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:36.573+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:36.653 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:36.654 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:36.663 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:01:36.705 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:36.701+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:36.782 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:36.777+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:36.812 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:36.812 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:36.822 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:01:36.883 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:36.881+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:36.891 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-13T07:01:36.930 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:36.930 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:36.940 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:01:36.984 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:36.981+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:36.999 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:36.999 INFO:teuthology.orchestra.run.smithi118.stdout:639 read 0x3f94329 thru 0x3f99021 (0x4cf9 bytes) 2024-09-13T07:01:37.017 INFO:teuthology.orchestra.run.smithi118.stdout:640 trunc from 0xcd1b645 to 0x2918716 2024-09-13T07:01:37.060 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:37.060 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:37.075 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:01:37.085 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:37.081+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:37.175 INFO:teuthology.orchestra.run.smithi118.stdout:641 write 0xc5dbf35 thru 0xc5dddce (0x1e9a bytes) 2024-09-13T07:01:37.177 INFO:teuthology.orchestra.run.smithi118.stdout:642 write 0x5ea1f44 thru 0x5eafd3b (0xddf8 bytes) 2024-09-13T07:01:37.181 INFO:teuthology.orchestra.run.smithi118.stdout:644 write 0x6a5e14d thru 0x6a6b502 (0xd3b6 bytes) 2024-09-13T07:01:37.184 INFO:teuthology.orchestra.run.smithi118.stdout:645 punch from 0x4f12ffa to 0x4f2016f, (0xd175 bytes) 2024-09-13T07:01:37.184 INFO:teuthology.orchestra.run.smithi118.stdout:648 writesame 0x147fbd2 thru 0x1483432 data_size 0x11(0x3861 bytes) 2024-09-13T07:01:37.185 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:37.181+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:37.187 INFO:teuthology.orchestra.run.smithi118.stdout:649 write 0x5bab999 thru 0x5bb525d (0x98c5 bytes) 2024-09-13T07:01:37.191 INFO:teuthology.orchestra.run.smithi118.stdout:650 read 0xb6dfe6 thru 0xb7957a (0xb595 bytes) 2024-09-13T07:01:37.192 INFO:teuthology.orchestra.run.smithi118.stdout:651 punch from 0x8fd13b3 to 0x8fda78d, (0x93da bytes) 2024-09-13T07:01:37.192 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:37.192 INFO:teuthology.orchestra.run.smithi118.stdout:654 write 0x77b9a06 thru 0x77be4b7 (0x4ab2 bytes) 2024-09-13T07:01:37.195 INFO:teuthology.orchestra.run.smithi118.stdout:655 write 0xb24ba2f thru 0xb24ec41 (0x3213 bytes) 2024-09-13T07:01:37.198 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:37.198 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:37.208 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:01:37.213 INFO:teuthology.orchestra.run.smithi118.stdout:656 writesame 0xe7396e6 thru 0xe741e84 data_size 0x3(0x879f bytes) 2024-09-13T07:01:37.215 INFO:teuthology.orchestra.run.smithi118.stdout:658 read 0xb09361b thru 0xb093d1b (0x701 bytes) 2024-09-13T07:01:37.216 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:37.216 INFO:teuthology.orchestra.run.smithi118.stdout:660 write 0x4365070 thru 0x436d827 (0x87b8 bytes) 2024-09-13T07:01:37.218 INFO:teuthology.orchestra.run.smithi118.stdout:662 write 0x723a300 thru 0x7245b8e (0xb88f bytes) 2024-09-13T07:01:37.222 INFO:teuthology.orchestra.run.smithi118.stdout:665 write 0x168de52 thru 0x16996ac (0xb85b bytes) 2024-09-13T07:01:37.225 INFO:teuthology.orchestra.run.smithi118.stdout:666 write 0x1dbf7dd thru 0x1dc3039 (0x385d bytes) 2024-09-13T07:01:37.228 INFO:teuthology.orchestra.run.smithi118.stdout:667 write 0xe1f3807 thru 0xe1fbf23 (0x871d bytes) 2024-09-13T07:01:37.231 INFO:teuthology.orchestra.run.smithi118.stdout:668 writesame 0x2ebcb0b thru 0x2ec1332 data_size 0x4(0x4828 bytes) 2024-09-13T07:01:37.237 INFO:teuthology.orchestra.run.smithi118.stdout:669 read 0x4104982 thru 0x410a605 (0x5c84 bytes) 2024-09-13T07:01:37.238 INFO:teuthology.orchestra.run.smithi118.stdout:671 writesame 0x6e4b3e8 thru 0x6e50e12 data_size 0x1(0x5a2b bytes) 2024-09-13T07:01:37.269 INFO:teuthology.orchestra.run.smithi118.stdout:673 write 0xeb994e5 thru 0xeba68b0 (0xd3cc bytes) 2024-09-13T07:01:37.273 INFO:teuthology.orchestra.run.smithi118.stdout:675 writesame 0x1193a91 thru 0x119a46d data_size 0x1(0x69dd bytes) 2024-09-13T07:01:37.286 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:37.285+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:37.291 INFO:teuthology.orchestra.run.smithi118.stdout:676 read 0x5365da1 thru 0x5368873 (0x2ad3 bytes) 2024-09-13T07:01:37.292 INFO:teuthology.orchestra.run.smithi118.stdout:677 read 0xa63ae31 thru 0xa63f7dc (0x49ac bytes) 2024-09-13T07:01:37.293 INFO:teuthology.orchestra.run.smithi118.stdout:679 write 0xe1d6ba3 thru 0xe1dc32f (0x578d bytes) 2024-09-13T07:01:37.296 INFO:teuthology.orchestra.run.smithi118.stdout:680 write 0x12f9f82 thru 0x12fd458 (0x34d7 bytes) 2024-09-13T07:01:37.299 INFO:teuthology.orchestra.run.smithi118.stdout:681 writesame 0xb30c13c thru 0xb316826 data_size 0x13(0xa6eb bytes) 2024-09-13T07:01:37.304 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:37.304 INFO:teuthology.orchestra.run.smithi118.stdout:684 write 0x4ef8c82 thru 0x4f0799d (0xed1c bytes) 2024-09-13T07:01:37.307 INFO:teuthology.orchestra.run.smithi118.stdout:688 writesame 0xe7a206a thru 0xe7b01ae data_size 0x2f(0xe145 bytes) 2024-09-13T07:01:37.311 INFO:teuthology.orchestra.run.smithi118.stdout:689 read 0x566ef4f thru 0x5673ec5 (0x4f77 bytes) 2024-09-13T07:01:37.313 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:37.313 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:37.323 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:01:37.334 INFO:teuthology.orchestra.run.smithi118.stdout:691 trunc from 0xeba68b1 to 0x1e4ba5e 2024-09-13T07:01:37.347 INFO:teuthology.orchestra.run.smithi118.stdout:692 clone 9 order 23 su 262144 sc 11 2024-09-13T07:01:37.387 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:37.381+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:37.431 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:37.431 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:37.440 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:01:37.487 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:37.481+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:37.585 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone8 from 0x1e4ba5e (overlap 0x1e4ba5e) to 0x0 2024-09-13T07:01:37.588 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:37.585+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:37.599 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:37.599 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:37.610 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:01:37.689 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:37.689+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:37.712 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:37.712 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:37.723 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:01:37.790 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:37.785+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:37.824 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-13T07:01:37.882 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:37.882 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:37.891 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:37.885+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:37.893 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:01:37.992 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:37.989+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:38.006 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:38.006 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:38.017 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:01:38.093 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:38.089+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:38.169 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:38.170 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:38.179 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:01:38.194 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:38.193+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:38.208 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:01:38.222 INFO:teuthology.orchestra.run.smithi118.stdout:693 punch from 0x11fc5a7 to 0x1207ab2, (0xb50b bytes) 2024-09-13T07:01:38.227 INFO:teuthology.orchestra.run.smithi118.stdout:696 read 0xf7c0ab thru 0xf8114d (0x50a3 bytes) 2024-09-13T07:01:38.228 INFO:teuthology.orchestra.run.smithi118.stdout:697 read 0xf87b73 thru 0xf9150d (0x999b bytes) 2024-09-13T07:01:38.229 INFO:teuthology.orchestra.run.smithi118.stdout:698 clone 10 order 23 su 2097152 sc 14 2024-09-13T07:01:38.282 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:38.282 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:38.291 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:01:38.334 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:38.333+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:38.349 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:38.368 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:38.385 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:38.395 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:38.393+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:38.405 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:38.409 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:01:38.431 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:38.445 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:38.445 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:38.445 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:38.445 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:38.445 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:38.451 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:38.455 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:01:38.475 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:38.510 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:38.511 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:38.509+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:38.512 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:38.512 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:38.512 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:38.513 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:38.513 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:38.513 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:38.513 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:38.513 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:38.593 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:38.594 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:38.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:38.594 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:38.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.594 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:434 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'4, mlcod=26'2)", 2024-09-13T07:01:38.594 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:01:38.594 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.056326016, 2024-09-13T07:01:38.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001931945, 2024-09-13T07:01:38.594 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.594 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.594 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535695+0000", 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:38.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535709+0000", 2024-09-13T07:01:38.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3577999999999999e-05 2024-09-13T07:01:38.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535732+0000", 2024-09-13T07:01:38.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3085e-05 2024-09-13T07:01:38.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535742+0000", 2024-09-13T07:01:38.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.048e-05 2024-09-13T07:01:38.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535840+0000", 2024-09-13T07:01:38.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7159999999999998e-05 2024-09-13T07:01:38.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535871+0000", 2024-09-13T07:01:38.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1510999999999997e-05 2024-09-13T07:01:38.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537572+0000", 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017008279999999999 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537629+0000", 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6520000000000001e-05 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:435 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'5, mlcod=26'2)", 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:38.598 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.055194309999999, 2024-09-13T07:01:38.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017126769999999999, 2024-09-13T07:01:38.599 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.599 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.599 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:38.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:38.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536835+0000", 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4939999999999997e-06 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536854+0000", 2024-09-13T07:01:38.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8655999999999999e-05 2024-09-13T07:01:38.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536862+0000", 2024-09-13T07:01:38.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6570000000000003e-06 2024-09-13T07:01:38.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536926+0000", 2024-09-13T07:01:38.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4037999999999999e-05 2024-09-13T07:01:38.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536958+0000", 2024-09-13T07:01:38.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1584000000000001e-05 2024-09-13T07:01:38.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538493+0000", 2024-09-13T07:01:38.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00153513 2024-09-13T07:01:38.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538541+0000", 2024-09-13T07:01:38.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7883999999999998e-05 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:483 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'24, mlcod=26'21)", 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.976715681, 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018371780000000001, 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615306+0000", 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615322+0000", 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5519000000000002e-05 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615335+0000", 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3854e-05 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615345+0000", 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4930000000000004e-06 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615431+0000", 2024-09-13T07:01:38.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6346000000000003e-05 2024-09-13T07:01:38.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615480+0000", 2024-09-13T07:01:38.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8303999999999997e-05 2024-09-13T07:01:38.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617102+0000", 2024-09-13T07:01:38.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016225250000000001 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617144+0000", 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1936e-05 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:488 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'26, mlcod=26'24)", 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.624174+0000", 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.967848491, 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001630374, 2024-09-13T07:01:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.608 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.608 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624174+0000", 2024-09-13T07:01:38.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624174+0000", 2024-09-13T07:01:38.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624172+0000", 2024-09-13T07:01:38.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:38.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624193+0000", 2024-09-13T07:01:38.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0475999999999998e-05 2024-09-13T07:01:38.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624210+0000", 2024-09-13T07:01:38.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7521999999999999e-05 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624218+0000", 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9580000000000007e-06 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624289+0000", 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0482999999999996e-05 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.624323+0000", 2024-09-13T07:01:38.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3643000000000003e-05 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.625746+0000", 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014238009999999999 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.625804+0000", 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8087999999999997e-05 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:494 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'29, mlcod=26'27)", 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.636753+0000", 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.955270067000001, 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001628083, 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636753+0000", 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636753+0000", 2024-09-13T07:01:38.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636751+0000", 2024-09-13T07:01:38.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:38.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636761+0000", 2024-09-13T07:01:38.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0118e-05 2024-09-13T07:01:38.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636780+0000", 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9313e-05 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636792+0000", 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1674e-05 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636872+0000", 2024-09-13T07:01:38.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0098000000000005e-05 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.636915+0000", 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3037000000000001e-05 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.638323+0000", 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001407752 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.638381+0000", 2024-09-13T07:01:38.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7578999999999999e-05 2024-09-13T07:01:38.616 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.616 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.616 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.616 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:518 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'38, mlcod=26'36)", 2024-09-13T07:01:38.616 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:01:38.616 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.267138943000001, 2024-09-13T07:01:38.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018201179999999999, 2024-09-13T07:01:38.616 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.616 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.616 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:01:38.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:01:38.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324883+0000", 2024-09-13T07:01:38.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324891+0000", 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1499999999999999e-06 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324924+0000", 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2601999999999997e-05 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324942+0000", 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7796999999999999e-05 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325060+0000", 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011786900000000001 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325123+0000", 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3474999999999998e-05 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326620+0000", 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014968830000000001 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326704+0000", 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3716999999999998e-05 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:597 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'39, mlcod=26'37)", 2024-09-13T07:01:38.620 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.150311615, 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017959829999999999, 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:01:38.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441709+0000", 2024-09-13T07:01:38.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:38.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441728+0000", 2024-09-13T07:01:38.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9627999999999999e-05 2024-09-13T07:01:38.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441765+0000", 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6559e-05 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441783+0000", 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7691000000000001e-05 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441904+0000", 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121184 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441966+0000", 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2428999999999995e-05 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443448+0000", 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00148197 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443507+0000", 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8699000000000002e-05 2024-09-13T07:01:38.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.535431321000001, 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:01:38.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:01:38.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:01:38.629 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.629 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.629 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.629 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1548 3.4 e27/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 27'52, mlcod=27'50)", 2024-09-13T07:01:38.629 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:01:38.629 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.449809784999999, 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017825779999999999, 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142211+0000", 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142232+0000", 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0888000000000001e-05 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142268+0000", 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6261999999999999e-05 2024-09-13T07:01:38.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142285+0000", 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7640000000000001e-05 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142381+0000", 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5171000000000001e-05 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142450+0000", 2024-09-13T07:01:38.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9778999999999997e-05 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143953+0000", 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001503129 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143995+0000", 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1878000000000001e-05 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1550 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'14, mlcod=27'12)", 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.449162074, 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002750452, 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142859+0000", 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142874+0000", 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5149e-05 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142911+0000", 2024-09-13T07:01:38.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6439000000000001e-05 2024-09-13T07:01:38.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142926+0000", 2024-09-13T07:01:38.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5079000000000001e-05 2024-09-13T07:01:38.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143012+0000", 2024-09-13T07:01:38.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6230000000000006e-05 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143063+0000", 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1369000000000002e-05 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145539+0000", 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024753420000000002 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145611+0000", 2024-09-13T07:01:38.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2392999999999996e-05 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1668 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'17, mlcod=27'15)", 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.250746612, 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033976620000000001, 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:38.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341281+0000", 2024-09-13T07:01:38.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8109999999999999e-06 2024-09-13T07:01:38.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341298+0000", 2024-09-13T07:01:38.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6575e-05 2024-09-13T07:01:38.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341308+0000", 2024-09-13T07:01:38.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0369e-05 2024-09-13T07:01:38.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341434+0000", 2024-09-13T07:01:38.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125454 2024-09-13T07:01:38.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341453+0000", 2024-09-13T07:01:38.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9159000000000001e-05 2024-09-13T07:01:38.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344638+0000", 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031851750000000002 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344674+0000", 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5410999999999997e-05 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.642 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1666 3.4 e28/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 28'55, mlcod=27'53)", 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.25069294, 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001805809, 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341329+0000", 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341349+0000", 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.999e-05 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341357+0000", 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0620000000000003e-06 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341361+0000", 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341710+0000", 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034873400000000001 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341736+0000", 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5772e-05 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343097+0000", 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013615319999999999 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343135+0000", 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8111999999999997e-05 2024-09-13T07:01:38.646 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1800 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'20, mlcod=28'18)", 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.048322474000001, 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001874398, 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:01:38.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:01:38.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543699+0000", 2024-09-13T07:01:38.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:38.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543713+0000", 2024-09-13T07:01:38.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4786e-05 2024-09-13T07:01:38.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543737+0000", 2024-09-13T07:01:38.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3626e-05 2024-09-13T07:01:38.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543752+0000", 2024-09-13T07:01:38.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.489e-05 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543832+0000", 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0396999999999996e-05 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543994+0000", 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016129100000000001 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545471+0000", 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014771960000000001 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545575+0000", 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103732 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1882 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'23, mlcod=28'21)", 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:01:38.651 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 11.952689492999999, 2024-09-13T07:01:38.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001767172, 2024-09-13T07:01:38.652 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.652 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.652 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:01:38.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639332+0000", 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639343+0000", 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1436e-05 2024-09-13T07:01:38.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639361+0000", 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7915e-05 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639370+0000", 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3189999999999995e-06 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639425+0000", 2024-09-13T07:01:38.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4917999999999998e-05 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639454+0000", 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9862999999999999e-05 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641059+0000", 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016048950000000001 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641100+0000", 2024-09-13T07:01:38.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0883999999999997e-05 2024-09-13T07:01:38.656 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.656 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.656 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.656 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3069 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'33, mlcod=31'31)", 2024-09-13T07:01:38.656 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:01:38.656 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.4295497770000001, 2024-09-13T07:01:38.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018597010000000001, 2024-09-13T07:01:38.656 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.656 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.656 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162471+0000", 2024-09-13T07:01:38.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162520+0000", 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8466e-05 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162546+0000", 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6089999999999999e-05 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162562+0000", 2024-09-13T07:01:38.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5962000000000001e-05 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162646+0000", 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3985000000000003e-05 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162691+0000", 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5785999999999998e-05 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164265+0000", 2024-09-13T07:01:38.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015740890000000001 2024-09-13T07:01:38.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164333+0000", 2024-09-13T07:01:38.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7014999999999994e-05 2024-09-13T07:01:38.660 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.660 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.660 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.660 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3121 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'36, mlcod=31'34)", 2024-09-13T07:01:38.660 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.3156033899999997, 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019657780000000001, 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276417+0000", 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276497+0000", 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9729999999999997e-05 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276537+0000", 2024-09-13T07:01:38.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0102999999999999e-05 2024-09-13T07:01:38.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276553+0000", 2024-09-13T07:01:38.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6518999999999999e-05 2024-09-13T07:01:38.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276662+0000", 2024-09-13T07:01:38.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109252 2024-09-13T07:01:38.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276729+0000", 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7056000000000003e-05 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278270+0000", 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015408819999999999 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278385+0000", 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114613 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.664 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3390 3.0 e32/22 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:head v 32'45, mlcod=31'42)", 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.0965150430000001, 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002246587, 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:01:38.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:01:38.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495507+0000", 2024-09-13T07:01:38.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:38.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495530+0000", 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2592999999999999e-05 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495550+0000", 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0152999999999998e-05 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495562+0000", 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2336e-05 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495662+0000", 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100317 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495717+0000", 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4304e-05 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497707+0000", 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019906849999999999 2024-09-13T07:01:38.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497754+0000", 2024-09-13T07:01:38.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6715999999999998e-05 2024-09-13T07:01:38.669 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.669 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.669 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.669 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3399 3.4 e32/22 3:22863a52:::rbd_data.11945d91e99c.0000000000000013:head v 32'65, mlcod=31'62)", 2024-09-13T07:01:38.669 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:01:38.669 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.0925783710000001, 2024-09-13T07:01:38.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028830779999999999, 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499442+0000", 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499459+0000", 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7042e-05 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499493+0000", 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3788000000000002e-05 2024-09-13T07:01:38.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499512+0000", 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9055999999999998e-05 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499608+0000", 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6168000000000005e-05 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499658+0000", 2024-09-13T07:01:38.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9469999999999999e-05 2024-09-13T07:01:38.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502277+0000", 2024-09-13T07:01:38.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026188639999999998 2024-09-13T07:01:38.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502327+0000", 2024-09-13T07:01:38.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0578999999999998e-05 2024-09-13T07:01:38.673 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.673 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.673 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.674 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3395 3.0 e32/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 32'47, mlcod=31'43)", 2024-09-13T07:01:38.674 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:38.674 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.0924802860000007, 2024-09-13T07:01:38.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027505870000000001, 2024-09-13T07:01:38.674 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.674 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.674 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:38.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499560+0000", 2024-09-13T07:01:38.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8416000000000001e-05 2024-09-13T07:01:38.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499576+0000", 2024-09-13T07:01:38.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5602000000000001e-05 2024-09-13T07:01:38.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499587+0000", 2024-09-13T07:01:38.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1112e-05 2024-09-13T07:01:38.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499654+0000", 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7164999999999998e-05 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499674+0000", 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9638000000000001e-05 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502210+0000", 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025367100000000002 2024-09-13T07:01:38.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502293+0000", 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2607000000000004e-05 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:4534 3.0 e32/22 3:00392e9e:::rbd_data.11944cdc0656.000000000000000c:head v 32'51, mlcod=32'49)", 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.6807790999999996, 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017597330000000001, 2024-09-13T07:01:38.678 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:38.679 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:38.679 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:38.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:38.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:01:38.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:38.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:01:38.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:38.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:38.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911242+0000", 2024-09-13T07:01:38.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:38.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:38.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911306+0000", 2024-09-13T07:01:38.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4480999999999995e-05 2024-09-13T07:01:38.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:38.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911338+0000", 2024-09-13T07:01:38.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1733000000000003e-05 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911351+0000", 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3241000000000001e-05 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911455+0000", 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103723 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:38.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911495+0000", 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0683000000000003e-05 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.912933+0000", 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001437219 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.913003+0000", 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0629000000000004e-05 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.682 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.683 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:38.683 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:38.683 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:38.684 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone9 from 0x1e4ba5e (overlap 0x1e4ba5e) to 0x1b44154 2024-09-13T07:01:38.685 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:01:38.685 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:38.681+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:38.785 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:38.781+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:38.794 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:38.794 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:38.804 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:01:38.820 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-13T07:01:38.826 INFO:teuthology.orchestra.run.smithi118.stdout:699 trunc from 0x1e4ba5e to 0xcfaf3 2024-09-13T07:01:38.850 INFO:teuthology.orchestra.run.smithi118.stdout:700 read 0x308fe thru 0x3a200 (0x9903 bytes) 2024-09-13T07:01:38.850 INFO:teuthology.orchestra.run.smithi118.stdout:701 read 0x1dee3 thru 0x285e1 (0xa6ff bytes) 2024-09-13T07:01:38.872 INFO:teuthology.orchestra.run.smithi118.stdout:702 write 0x6b24b33 thru 0x6b2eaa8 (0x9f76 bytes) 2024-09-13T07:01:38.879 INFO:teuthology.orchestra.run.smithi118.stdout:703 punch from 0x71c800 to 0x729426, (0xcc26 bytes) 2024-09-13T07:01:38.879 INFO:teuthology.orchestra.run.smithi118.stdout:706 write 0x3a8aeff thru 0x3a8b598 (0x69a bytes) 2024-09-13T07:01:38.886 INFO:teuthology.orchestra.run.smithi118.stdout:707 trunc from 0x6b2eaa9 to 0x6e32d69 2024-09-13T07:01:38.886 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:38.885+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:38.896 INFO:teuthology.orchestra.run.smithi118.stdout:709 writesame 0x39db967 thru 0x39e8fd5 data_size 0x1(0xd66f bytes) 2024-09-13T07:01:38.908 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:38.908 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:38.908 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:38.908 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:38.908 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:38.918 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:01:38.940 INFO:teuthology.orchestra.run.smithi118.stdout:710 writesame 0x49a2c99 thru 0x49af4ca data_size 0x7d(0xc832 bytes) 2024-09-13T07:01:38.946 INFO:teuthology.orchestra.run.smithi118.stdout:711 read 0x5e6bf0b thru 0x5e711ac (0x52a2 bytes) 2024-09-13T07:01:38.959 INFO:teuthology.orchestra.run.smithi118.stdout:713 write 0x820255b thru 0x8205bfc (0x36a2 bytes) 2024-09-13T07:01:38.965 INFO:teuthology.orchestra.run.smithi118.stdout:714 punch from 0x1f1662b to 0x1f22e59, (0xc82e bytes) 2024-09-13T07:01:38.965 INFO:teuthology.orchestra.run.smithi118.stdout:715 write 0x6866d03 thru 0x686f958 (0x8c56 bytes) 2024-09-13T07:01:38.971 INFO:teuthology.orchestra.run.smithi118.stdout:717 read 0x4f62d2f thru 0x4f6900f (0x62e1 bytes) 2024-09-13T07:01:38.972 INFO:teuthology.orchestra.run.smithi118.stdout:718 write 0x7bdb47b thru 0x7bdcf6f (0x1af5 bytes) 2024-09-13T07:01:38.977 INFO:teuthology.orchestra.run.smithi118.stdout:719 write 0x7db4b38 thru 0x7dc23ae (0xd877 bytes) 2024-09-13T07:01:38.983 INFO:teuthology.orchestra.run.smithi118.stdout:720 read 0x494549b thru 0x49541cb (0xed31 bytes) 2024-09-13T07:01:38.984 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:38.984 INFO:teuthology.orchestra.run.smithi118.stdout:724 trunc from 0x8205bfd to 0x84b7f27 2024-09-13T07:01:38.986 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:38.981+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:38.997 INFO:teuthology.orchestra.run.smithi118.stdout:725 clone 11 order 25 su 131072 sc 6 2024-09-13T07:01:39.053 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:39.053 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:39.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:39.053 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:39.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.053 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:441 3.b 3:d465e8df:::rbd_data.1194d22ad098.0000000000000036:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:39.053 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:39.053 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.51024228, 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012043093, 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 441 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:39.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541122+0000", 2024-09-13T07:01:39.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541136+0000", 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3899e-05 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541163+0000", 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.694e-05 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541176+0000", 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2802e-05 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541257+0000", 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1018000000000003e-05 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541414+0000", 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015745000000000001 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.544450+0000", 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030357209999999999 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553002+0000", 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085521020000000007 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553022+0000", 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9694000000000002e-05 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553103+0000", 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1008999999999996e-05 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553168+0000", 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5010000000000003e-05 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1155 3.7 3:e095a6bd:::rbd_data.11943a018f34.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.218603017, 2024-09-13T07:01:39.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022798902999999999, 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1155 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:01:39.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832762+0000", 2024-09-13T07:01:39.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:39.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832776+0000", 2024-09-13T07:01:39.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3828e-05 2024-09-13T07:01:39.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832806+0000", 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0159e-05 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832820+0000", 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.384e-05 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832904+0000", 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4506999999999996e-05 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.833121+0000", 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000217223 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.839072+0000", 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0059509849999999998 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855489+0000", 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016416907000000001 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855503+0000", 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3658999999999999e-05 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855535+0000", 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1696000000000003e-05 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855563+0000", 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8578999999999999e-05 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.064 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1200 3.7 3:e095a6bd:::rbd_data.11943a018f34.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.072264067000001, 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018736263, 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1200 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.065 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:01:39.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:01:39.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979100+0000", 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979115+0000", 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4823e-05 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979149+0000", 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3772999999999997e-05 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979163+0000", 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3845999999999999e-05 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979333+0000", 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000169622 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979603+0000", 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00027078999999999997 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:39.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.986017+0000", 2024-09-13T07:01:39.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064139590000000003 2024-09-13T07:01:39.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:39.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997476+0000", 2024-09-13T07:01:39.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011458582 2024-09-13T07:01:39.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:39.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997588+0000", 2024-09-13T07:01:39.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011161 2024-09-13T07:01:39.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997732+0000", 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000144 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997839+0000", 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107892 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.070 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1531 3.c e27/22 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head v 27'9, mlcod=27'7)", 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.97683591, 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039127038000000003, 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:01:39.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074530+0000", 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099434+0000", 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.024903926999999999 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099459+0000", 2024-09-13T07:01:39.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5384999999999998e-05 2024-09-13T07:01:39.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099469+0000", 2024-09-13T07:01:39.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.024e-05 2024-09-13T07:01:39.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099566+0000", 2024-09-13T07:01:39.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6533999999999994e-05 2024-09-13T07:01:39.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099600+0000", 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4007000000000002e-05 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113477+0000", 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01387741 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113658+0000", 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000180953 2024-09-13T07:01:39.074 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3187 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.410269+0000", 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.6410984850000006, 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012395160000000001, 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.075 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3187 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410269+0000", 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410269+0000", 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410266+0000", 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410283+0000", 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7527999999999999e-05 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410317+0000", 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3726000000000002e-05 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410340+0000", 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3252e-05 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410425+0000", 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4338999999999993e-05 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410528+0000", 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103058 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.412544+0000", 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020164639999999999 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422499+0000", 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099549960000000007 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422521+0000", 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1433e-05 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422604+0000", 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2820999999999999e-05 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422664+0000", 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0362999999999998e-05 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3975 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:39.080 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.3969719620000003, 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011993248, 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3975 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654398+0000", 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8619999999999996e-06 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654408+0000", 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8209999999999997e-06 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654416+0000", 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0180000000000001e-06 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666002+0000", 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011586008 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666074+0000", 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1500000000000003e-05 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666388+0000", 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00031467300000000001 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4022 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:01:39.084 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.3969434659999997, 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014992557, 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4022 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654423+0000", 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654427+0000", 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0960000000000003e-06 2024-09-13T07:01:39.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654437+0000", 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7070000000000004e-06 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654445+0000", 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9049999999999997e-06 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666423+0000", 2024-09-13T07:01:39.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011978223 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666521+0000", 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7832999999999997e-05 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669416+0000", 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002895393 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4023 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.3969145800000007, 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015452399, 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4023 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654452+0000", 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654456+0000", 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7570000000000002e-06 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654465+0000", 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7140000000000005e-06 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654486+0000", 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0290000000000001e-05 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669455+0000", 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014969651 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669722+0000", 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026677899999999997 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669905+0000", 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018285999999999999 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.092 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4024 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.39687378, 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015682979, 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4024 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.093 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654497+0000", 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4800000000000003e-06 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654507+0000", 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8749999999999995e-06 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654515+0000", 2024-09-13T07:01:39.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0350000000000001e-06 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669936+0000", 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015420916 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670030+0000", 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3712e-05 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670176+0000", 2024-09-13T07:01:39.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000146613 2024-09-13T07:01:39.097 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.097 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.097 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.097 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4025 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:39.097 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:39.097 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.3968447719999997, 2024-09-13T07:01:39.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016683559000000001, 2024-09-13T07:01:39.097 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.097 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.097 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4025 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654526+0000", 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654535+0000", 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4199999999999996e-06 2024-09-13T07:01:39.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654543+0000", 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670307+0000", 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015763482999999998 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670399+0000", 2024-09-13T07:01:39.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2068e-05 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671206+0000", 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000807352 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4026 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 13238272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.3968170430000004, 2024-09-13T07:01:39.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016876344000000001, 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4026 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654554+0000", 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1119999999999998e-06 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654564+0000", 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0327e-05 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654571+0000", 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3440000000000003e-06 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671239+0000", 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016668039999999999 2024-09-13T07:01:39.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671327+0000", 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7818999999999994e-05 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671427+0000", 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9324e-05 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.105 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4027 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.3967882809999992, 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017835597000000002, 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4027 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:01:39.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654578+0000", 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654590+0000", 2024-09-13T07:01:39.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1595e-05 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654601+0000", 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0827e-05 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654609+0000", 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4919999999999993e-06 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672157+0000", 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017547770000000001 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672234+0000", 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7509999999999995e-05 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672415+0000", 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018005500000000001 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11064 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.607882+0000", 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.44348568399999999, 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012098661, 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11064 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607882+0000", 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607882+0000", 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607881+0000", 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607885+0000", 2024-09-13T07:01:39.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5530000000000003e-06 2024-09-13T07:01:39.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607896+0000", 2024-09-13T07:01:39.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1073999999999999e-05 2024-09-13T07:01:39.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607905+0000", 2024-09-13T07:01:39.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8310000000000007e-06 2024-09-13T07:01:39.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.617863+0000", 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099584400000000007 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.617953+0000", 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0050999999999996e-05 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.619980+0000", 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002026721 2024-09-13T07:01:39.113 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11130 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608440+0000", 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.442927497, 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01196526, 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.114 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11130 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608440+0000", 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608440+0000", 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608439+0000", 2024-09-13T07:01:39.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608443+0000", 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4499999999999997e-06 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608454+0000", 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0383999999999999e-05 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608462+0000", 2024-09-13T07:01:39.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0199999999999994e-06 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620250+0000", 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011788361000000001 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620322+0000", 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2193999999999994e-05 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620405+0000", 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2566999999999997e-05 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11151 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608582+0000", 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.44278513000000003, 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012261311, 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11151 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608582+0000", 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608582+0000", 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608581+0000", 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608586+0000", 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1289999999999998e-06 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608596+0000", 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0787e-05 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608618+0000", 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1971e-05 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620453+0000", 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011834829999999999 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620542+0000", 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8947000000000001e-05 2024-09-13T07:01:39.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620843+0000", 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00030128199999999999 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11152 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608627+0000", 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.44274028900000001, 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012436806, 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.122 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11152 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608627+0000", 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608627+0000", 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608626+0000", 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608631+0000", 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4109999999999998e-06 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608642+0000", 2024-09-13T07:01:39.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1158000000000001e-05 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608650+0000", 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5110000000000003e-06 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620871+0000", 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012221019 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620956+0000", 2024-09-13T07:01:39.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5223999999999996e-05 2024-09-13T07:01:39.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621064+0000", 2024-09-13T07:01:39.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010727599999999999 2024-09-13T07:01:39.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.126 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.126 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11153 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:39.126 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608658+0000", 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.44270917399999998, 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012691595, 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11153 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608658+0000", 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608658+0000", 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608657+0000", 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608661+0000", 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7629999999999998e-06 2024-09-13T07:01:39.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608672+0000", 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0732e-05 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608680+0000", 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8429999999999993e-06 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621106+0000", 2024-09-13T07:01:39.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01242594 2024-09-13T07:01:39.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621190+0000", 2024-09-13T07:01:39.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4005000000000006e-05 2024-09-13T07:01:39.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621350+0000", 2024-09-13T07:01:39.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000699999999999 2024-09-13T07:01:39.130 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.130 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.130 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.131 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11173 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:39.131 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608785+0000", 2024-09-13T07:01:39.131 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.44258262799999998, 2024-09-13T07:01:39.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012946599, 2024-09-13T07:01:39.131 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.131 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.131 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.131 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.131 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.131 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11173 2024-09-13T07:01:39.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608785+0000", 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608785+0000", 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608784+0000", 2024-09-13T07:01:39.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608788+0000", 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5070000000000002e-06 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608799+0000", 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0431000000000001e-05 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608807+0000", 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5220000000000007e-06 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621397+0000", 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01258975 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621478+0000", 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0408000000000007e-05 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621731+0000", 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025365899999999999 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11174 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.44255170999999999, 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01318603, 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.135 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11174 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608819+0000", 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9740000000000004e-06 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608829+0000", 2024-09-13T07:01:39.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0383999999999999e-05 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608838+0000", 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6719999999999993e-06 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621751+0000", 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012913017000000001 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621809+0000", 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8505000000000003e-05 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622002+0000", 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192141 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11175 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:01:39.139 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.44252184999999999, 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013984661000000001, 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11175 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608849+0000", 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8449999999999996e-06 2024-09-13T07:01:39.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:39.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608859+0000", 2024-09-13T07:01:39.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0335e-05 2024-09-13T07:01:39.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608868+0000", 2024-09-13T07:01:39.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6070000000000008e-06 2024-09-13T07:01:39.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622108+0000", 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013240676 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622223+0000", 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115277 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622830+0000", 2024-09-13T07:01:39.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00060653900000000001 2024-09-13T07:01:39.144 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.144 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.144 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.144 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:39.144 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:39.144 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:39.145 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:01:39.146 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:39.141+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:39.246 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:39.245+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:39.262 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:39.262 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:39.262 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:39.263 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:39.273 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:01:39.348 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:39.345+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:39.382 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:39.382 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:39.382 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:39.383 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:39.383 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:39.392 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:01:39.395 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:01:39.395 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:01:39.395 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:01:39.396 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-13T07:01:39.426 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:01:39.486 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:39.485+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:39.503 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:39.504 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:39.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:39.504 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:39.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.504 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1501 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.504 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023150+0000", 2024-09-13T07:01:39.504 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.477144843, 2024-09-13T07:01:39.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037513666000000001, 2024-09-13T07:01:39.504 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.504 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1501 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023150+0000", 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023150+0000", 2024-09-13T07:01:39.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023149+0000", 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023153+0000", 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.58e-06 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023161+0000", 2024-09-13T07:01:39.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0730000000000005e-06 2024-09-13T07:01:39.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023167+0000", 2024-09-13T07:01:39.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4300000000000003e-06 2024-09-13T07:01:39.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060527+0000", 2024-09-13T07:01:39.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037360021 2024-09-13T07:01:39.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060586+0000", 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8891999999999998e-05 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060664+0000", 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7210000000000001e-05 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1502 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023194+0000", 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.477100779000001, 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037692165999999999, 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1502 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023194+0000", 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023194+0000", 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023193+0000", 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023197+0000", 2024-09-13T07:01:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0040000000000001e-06 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023206+0000", 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.969e-06 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023213+0000", 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2250000000000003e-06 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060721+0000", 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037508138000000003 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060812+0000", 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0074000000000005e-05 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060886+0000", 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4541000000000003e-05 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1503 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023219+0000", 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.477075249, 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037827616000000001, 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1503 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023219+0000", 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023219+0000", 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023219+0000", 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023222+0000", 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1980000000000001e-06 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023230+0000", 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9650000000000008e-06 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023239+0000", 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8759999999999994e-06 2024-09-13T07:01:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060923+0000", 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037683662 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.060965+0000", 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2001e-05 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061047+0000", 2024-09-13T07:01:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2400999999999999e-05 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1504 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023246+0000", 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.477049171999999, 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038032028000000002, 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1504 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023246+0000", 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023246+0000", 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023245+0000", 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023249+0000", 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.216e-06 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023259+0000", 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4269999999999997e-06 2024-09-13T07:01:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023265+0000", 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.72e-06 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061077+0000", 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037811895999999998 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061186+0000", 2024-09-13T07:01:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010862900000000001 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061278+0000", 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1656000000000005e-05 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1505 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023272+0000", 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.477023113, 2024-09-13T07:01:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038242703000000003, 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1505 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023272+0000", 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023272+0000", 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023271+0000", 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023274+0000", 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1860000000000001e-06 2024-09-13T07:01:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023282+0000", 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023289+0000", 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.285e-06 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061345+0000", 2024-09-13T07:01:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038056363000000003 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061409+0000", 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4004000000000006e-05 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061514+0000", 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105324 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1506 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3162112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023297+0000", 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.476997219999999, 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038402842, 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1506 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023297+0000", 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023297+0000", 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023297+0000", 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023300+0000", 2024-09-13T07:01:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-13T07:01:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023309+0000", 2024-09-13T07:01:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2349999999999999e-06 2024-09-13T07:01:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023315+0000", 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7020000000000004e-06 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061539+0000", 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038223752999999999 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061618+0000", 2024-09-13T07:01:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9331000000000003e-05 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061700+0000", 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1896000000000003e-05 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1507 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023321+0000", 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.47697344, 2024-09-13T07:01:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038581179, 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1507 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023321+0000", 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023321+0000", 2024-09-13T07:01:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023321+0000", 2024-09-13T07:01:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023325+0000", 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6140000000000002e-06 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023333+0000", 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1780000000000003e-06 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023340+0000", 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6789999999999999e-06 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061730+0000", 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038389853000000002 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061830+0000", 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9810999999999997e-05 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061902+0000", 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2702999999999998e-05 2024-09-13T07:01:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1508 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3325952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023348+0000", 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.476946573999999, 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038783950999999997, 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1508 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023348+0000", 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023348+0000", 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023348+0000", 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023351+0000", 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3249999999999999e-06 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023359+0000", 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4910000000000005e-06 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023366+0000", 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.443e-06 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.061943+0000", 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038577144000000001 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062027+0000", 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3467999999999994e-05 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062132+0000", 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105553 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1509 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023372+0000", 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.476922941, 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039003355000000003, 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1509 2024-09-13T07:01:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023372+0000", 2024-09-13T07:01:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023372+0000", 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023371+0000", 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023375+0000", 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4570000000000001e-06 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023382+0000", 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7409999999999992e-06 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023389+0000", 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1310000000000003e-06 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062165+0000", 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038776418 2024-09-13T07:01:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062284+0000", 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119491 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062375+0000", 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0607999999999997e-05 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1510 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3489792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023395+0000", 2024-09-13T07:01:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.476900117, 2024-09-13T07:01:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039253549999999998, 2024-09-13T07:01:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1510 2024-09-13T07:01:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023395+0000", 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023395+0000", 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023394+0000", 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023398+0000", 2024-09-13T07:01:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5370000000000002e-06 2024-09-13T07:01:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023406+0000", 2024-09-13T07:01:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9980000000000003e-06 2024-09-13T07:01:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023412+0000", 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5060000000000001e-06 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062423+0000", 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039010421000000003 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062524+0000", 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101282 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062648+0000", 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012431399999999999 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1511 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023418+0000", 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.476876741, 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039437184, 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1511 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023418+0000", 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023418+0000", 2024-09-13T07:01:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023417+0000", 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023421+0000", 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2830000000000002e-06 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023429+0000", 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7929999999999998e-06 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023435+0000", 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5080000000000002e-06 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062679+0000", 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039243553 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062760+0000", 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1570999999999995e-05 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062855+0000", 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4946999999999997e-05 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1512 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023441+0000", 2024-09-13T07:01:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.47685319, 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039615015000000003, 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1512 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023441+0000", 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023441+0000", 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023441+0000", 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023449+0000", 2024-09-13T07:01:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700000000000007e-06 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023457+0000", 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2789999999999998e-06 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023465+0000", 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0590000000000003e-06 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062914+0000", 2024-09-13T07:01:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039448762999999998 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.062994+0000", 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9559000000000002e-05 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063057+0000", 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2786000000000005e-05 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1513 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023472+0000", 2024-09-13T07:01:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.476822990000001, 2024-09-13T07:01:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039796658999999998, 2024-09-13T07:01:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1513 2024-09-13T07:01:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023472+0000", 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023472+0000", 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023471+0000", 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023474+0000", 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023483+0000", 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6379999999999993e-06 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023490+0000", 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5930000000000002e-06 2024-09-13T07:01:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063095+0000", 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039605156000000002 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063181+0000", 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6166000000000001e-05 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063268+0000", 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7337999999999996e-05 2024-09-13T07:01:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1514 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3817472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023496+0000", 2024-09-13T07:01:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.476798704, 2024-09-13T07:01:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039930146, 2024-09-13T07:01:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1514 2024-09-13T07:01:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023496+0000", 2024-09-13T07:01:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023496+0000", 2024-09-13T07:01:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023495+0000", 2024-09-13T07:01:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023499+0000", 2024-09-13T07:01:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4379999999999999e-06 2024-09-13T07:01:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023507+0000", 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023514+0000", 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4239999999999998e-06 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063286+0000", 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039772620000000002 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063342+0000", 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5770000000000003e-05 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063426+0000", 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4011999999999995e-05 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1515 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023520+0000", 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.476775145, 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040093238000000003, 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1515 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023520+0000", 2024-09-13T07:01:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023520+0000", 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023519+0000", 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023522+0000", 2024-09-13T07:01:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9809999999999999e-06 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023533+0000", 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0852e-05 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023539+0000", 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.511e-06 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063464+0000", 2024-09-13T07:01:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039924266 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063539+0000", 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5813000000000001e-05 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063613+0000", 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3291e-05 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1516 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023548+0000", 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.476747098000001, 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040191618999999998, 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1516 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023548+0000", 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023548+0000", 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023547+0000", 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023551+0000", 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1840000000000001e-06 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023559+0000", 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0709999999999995e-06 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023567+0000", 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.306e-06 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063634+0000", 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040067047000000001 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063673+0000", 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8788000000000002e-05 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063739+0000", 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6705000000000006e-05 2024-09-13T07:01:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1517 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023573+0000", 2024-09-13T07:01:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.476721776, 2024-09-13T07:01:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040350010999999998, 2024-09-13T07:01:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1517 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023573+0000", 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023573+0000", 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023572+0000", 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023576+0000", 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8940000000000003e-06 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023584+0000", 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1529999999999997e-06 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023593+0000", 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8589999999999994e-06 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063778+0000", 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040184894999999998 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063851+0000", 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3114000000000005e-05 2024-09-13T07:01:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063923+0000", 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2084999999999998e-05 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1518 3.3 3:c427371f:::rbd_data.119438ec2c70.0000000000000001:4 [read 4145152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.023602+0000", 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.476693011, 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040449005000000003, 2024-09-13T07:01:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1518 2024-09-13T07:01:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023602+0000", 2024-09-13T07:01:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023602+0000", 2024-09-13T07:01:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023601+0000", 2024-09-13T07:01:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023604+0000", 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3739999999999998e-06 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023614+0000", 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8230000000000006e-06 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.023621+0000", 2024-09-13T07:01:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9870000000000002e-06 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063942+0000", 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040320644000000003 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.063985+0000", 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3114000000000001e-05 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.064051+0000", 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5784999999999994e-05 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1524 3.14 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.455526108000001, 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061190880000000003, 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1524 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044767+0000", 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052670+0000", 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079022450000000004 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052682+0000", 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2336e-05 2024-09-13T07:01:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052696+0000", 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052755+0000", 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9298e-05 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052947+0000", 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019153900000000001 2024-09-13T07:01:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.053694+0000", 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074663099999999999 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.102148+0000", 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048454329999999997 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105816+0000", 2024-09-13T07:01:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036675950000000001 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105921+0000", 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105874 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105959+0000", 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7991000000000003e-05 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1533 3.a e27/22 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head v 27'21, mlcod=27'19)", 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.066960+0000", 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.433334992000001, 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.038507995000000003, 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066960+0000", 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066960+0000", 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066958+0000", 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069912+0000", 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002953893 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069937+0000", 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4771000000000001e-05 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069950+0000", 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3028e-05 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070009+0000", 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9484000000000002e-05 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070042+0000", 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3025000000000003e-05 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105378+0000", 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035335706000000001 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105468+0000", 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9587999999999997e-05 2024-09-13T07:01:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.593 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.593 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.593 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.593 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.593 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:39.594 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:01:39.595 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:39.593+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:39.611 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone10 from 0x84b7f27 (overlap 0xcfaf3) to 0xbaea7 2024-09-13T07:01:39.695 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:39.689+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:39.713 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:39.713 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:39.713 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:39.713 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:39.723 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:01:39.730 INFO:tasks.ceph.mon.a.smithi118.stderr:2024-09-13T07:01:39.725+0000 7fd1ec319700 -1 mon.a@0(leader).osd e41 definitely_dead 0 2024-09-13T07:01:39.796 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:39.793+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:39.830 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:39.830 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:39.840 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:01:39.898 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:39.893+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:39.914 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-13T07:01:39.950 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:39.950 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:39.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:39.950 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:39.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.950 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:439 3.16 3:6cdf2866:::rbd_data.1194d22ad098.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:39.950 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.539100+0000", 2024-09-13T07:01:39.950 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.407684678000001, 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01241133, 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 439 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539100+0000", 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539100+0000", 2024-09-13T07:01:39.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539098+0000", 2024-09-13T07:01:39.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:39.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539113+0000", 2024-09-13T07:01:39.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5653000000000001e-05 2024-09-13T07:01:39.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539148+0000", 2024-09-13T07:01:39.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4489000000000001e-05 2024-09-13T07:01:39.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539162+0000", 2024-09-13T07:01:39.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4533e-05 2024-09-13T07:01:39.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539297+0000", 2024-09-13T07:01:39.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013447399999999999 2024-09-13T07:01:39.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539701+0000", 2024-09-13T07:01:39.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00040402400000000001 2024-09-13T07:01:39.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:39.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.542876+0000", 2024-09-13T07:01:39.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031755820000000001 2024-09-13T07:01:39.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:39.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551351+0000", 2024-09-13T07:01:39.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084747239999999995 2024-09-13T07:01:39.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:39.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551392+0000", 2024-09-13T07:01:39.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0921999999999999e-05 2024-09-13T07:01:39.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:39.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551457+0000", 2024-09-13T07:01:39.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4925999999999995e-05 2024-09-13T07:01:39.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551511+0000", 2024-09-13T07:01:39.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4446e-05 2024-09-13T07:01:39.956 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.956 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.956 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.957 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:440 3.1c 3:3b975cee:::rbd_data.1194d22ad098.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:39.957 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.539301+0000", 2024-09-13T07:01:39.957 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.407484118999999, 2024-09-13T07:01:39.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019168998999999999, 2024-09-13T07:01:39.957 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.957 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:39.957 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.957 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.957 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.957 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 440 2024-09-13T07:01:39.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.957 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539301+0000", 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539301+0000", 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539300+0000", 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539308+0000", 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-06 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539347+0000", 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9063999999999997e-05 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539353+0000", 2024-09-13T07:01:39.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3459999999999999e-06 2024-09-13T07:01:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539438+0000", 2024-09-13T07:01:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4757000000000002e-05 2024-09-13T07:01:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539706+0000", 2024-09-13T07:01:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026819000000000002 2024-09-13T07:01:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.543844+0000", 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004137338 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558385+0000", 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014541304999999999 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558397+0000", 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2269999999999999e-05 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558437+0000", 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0096999999999999e-05 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558470+0000", 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2242000000000001e-05 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1531 3.c 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:39.962 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:39.963 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.897272999, 2024-09-13T07:01:39.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063168486999999995, 2024-09-13T07:01:39.963 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.963 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:39.963 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.963 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.963 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.963 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1531 2024-09-13T07:01:39.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.963 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049511+0000", 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069234+0000", 2024-09-13T07:01:39.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019723920999999998 2024-09-13T07:01:39.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069269+0000", 2024-09-13T07:01:39.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4446999999999997e-05 2024-09-13T07:01:39.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069286+0000", 2024-09-13T07:01:39.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7405999999999999e-05 2024-09-13T07:01:39.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069370+0000", 2024-09-13T07:01:39.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-05 2024-09-13T07:01:39.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069511+0000", 2024-09-13T07:01:39.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014134299999999999 2024-09-13T07:01:39.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070463+0000", 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095167899999999998 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105076+0000", 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034613139000000001 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112562+0000", 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074861429999999998 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112631+0000", 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112680+0000", 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9607000000000001e-05 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1794 3.16 3:6a67e608:::rbd_data.11943a018f34.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:39.968 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.510292+0000", 2024-09-13T07:01:39.969 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.436493093999999, 2024-09-13T07:01:39.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015887680000000001, 2024-09-13T07:01:39.969 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.969 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:39.969 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.969 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.969 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.969 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1794 2024-09-13T07:01:39.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.969 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510292+0000", 2024-09-13T07:01:39.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510292+0000", 2024-09-13T07:01:39.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510289+0000", 2024-09-13T07:01:39.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:39.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510303+0000", 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3933e-05 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510335+0000", 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2169000000000003e-05 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510349+0000", 2024-09-13T07:01:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4205999999999999e-05 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510458+0000", 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010835100000000001 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510597+0000", 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013911699999999999 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:39.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.513457+0000", 2024-09-13T07:01:39.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028606439999999999 2024-09-13T07:01:39.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:39.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.526058+0000", 2024-09-13T07:01:39.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012600849000000001 2024-09-13T07:01:39.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:39.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.526090+0000", 2024-09-13T07:01:39.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2036999999999998e-05 2024-09-13T07:01:39.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.526139+0000", 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9042999999999998e-05 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.526179+0000", 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0077999999999998e-05 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.974 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:3917 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 13238272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653010+0000", 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2937748310000003, 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095166820000000003, 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3917 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653010+0000", 2024-09-13T07:01:39.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653010+0000", 2024-09-13T07:01:39.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653009+0000", 2024-09-13T07:01:39.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653016+0000", 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5970000000000003e-06 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653035+0000", 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8142999999999999e-05 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653050+0000", 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5449000000000001e-05 2024-09-13T07:01:39.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662404+0000", 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093544449999999994 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662452+0000", 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7975000000000001e-05 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662527+0000", 2024-09-13T07:01:39.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4140000000000005e-05 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:3918 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653062+0000", 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2937227490000005, 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095919990000000004, 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.980 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3918 2024-09-13T07:01:39.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.980 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653062+0000", 2024-09-13T07:01:39.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653062+0000", 2024-09-13T07:01:39.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653061+0000", 2024-09-13T07:01:39.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653079+0000", 2024-09-13T07:01:39.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.853e-05 2024-09-13T07:01:39.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653097+0000", 2024-09-13T07:01:39.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7156e-05 2024-09-13T07:01:39.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653115+0000", 2024-09-13T07:01:39.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8850000000000001e-05 2024-09-13T07:01:39.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662551+0000", 2024-09-13T07:01:39.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094354309999999993 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662594+0000", 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3080000000000001e-05 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662654+0000", 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0019000000000003e-05 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.984 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:3964 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:39.984 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653127+0000", 2024-09-13T07:01:39.984 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2936577039999992, 2024-09-13T07:01:39.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00965886, 2024-09-13T07:01:39.984 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.984 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.984 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3964 2024-09-13T07:01:39.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653127+0000", 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653127+0000", 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653126+0000", 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653132+0000", 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2400000000000004e-06 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653149+0000", 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6456e-05 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653165+0000", 2024-09-13T07:01:39.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6404000000000001e-05 2024-09-13T07:01:39.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662682+0000", 2024-09-13T07:01:39.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009516544 2024-09-13T07:01:39.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662728+0000", 2024-09-13T07:01:39.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6396000000000001e-05 2024-09-13T07:01:39.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662786+0000", 2024-09-13T07:01:39.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7890000000000003e-05 2024-09-13T07:01:39.988 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.988 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.988 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.988 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:3965 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:39.988 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653179+0000", 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2936061149999993, 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097343599999999992, 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3965 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653179+0000", 2024-09-13T07:01:39.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653179+0000", 2024-09-13T07:01:39.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653178+0000", 2024-09-13T07:01:39.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653184+0000", 2024-09-13T07:01:39.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6699999999999997e-06 2024-09-13T07:01:39.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653214+0000", 2024-09-13T07:01:39.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9485999999999999e-05 2024-09-13T07:01:39.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653228+0000", 2024-09-13T07:01:39.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4525999999999999e-05 2024-09-13T07:01:39.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662810+0000", 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095821139999999992 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662854+0000", 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3446999999999999e-05 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662913+0000", 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9157000000000003e-05 2024-09-13T07:01:39.992 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:3966 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653240+0000", 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2935443259999992, 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098197179999999998, 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.993 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3966 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653240+0000", 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653240+0000", 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653239+0000", 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653246+0000", 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653286+0000", 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9205999999999997e-05 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:39.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653300+0000", 2024-09-13T07:01:39.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4430000000000001e-05 2024-09-13T07:01:39.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:39.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662937+0000", 2024-09-13T07:01:39.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009637161 2024-09-13T07:01:39.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:39.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.662985+0000", 2024-09-13T07:01:39.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7339000000000002e-05 2024-09-13T07:01:39.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:39.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663060+0000", 2024-09-13T07:01:39.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5458999999999997e-05 2024-09-13T07:01:39.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.997 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:39.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:39.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.997 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:3967 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 12058624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:39.997 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653373+0000", 2024-09-13T07:01:39.997 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2934114529999992, 2024-09-13T07:01:39.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098568990000000006, 2024-09-13T07:01:39.998 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:39.998 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:39.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:39.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:39.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:39.998 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3967 2024-09-13T07:01:39.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.998 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:39.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:39.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653373+0000", 2024-09-13T07:01:39.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653373+0000", 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653372+0000", 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:39.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653379+0000", 2024-09-13T07:01:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0639999999999996e-06 2024-09-13T07:01:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653409+0000", 2024-09-13T07:01:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0151999999999999e-05 2024-09-13T07:01:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653429+0000", 2024-09-13T07:01:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0322999999999998e-05 2024-09-13T07:01:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663088+0000", 2024-09-13T07:01:40.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0096586269999999995 2024-09-13T07:01:40.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663153+0000", 2024-09-13T07:01:40.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5016000000000002e-05 2024-09-13T07:01:40.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663230+0000", 2024-09-13T07:01:40.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7066999999999999e-05 2024-09-13T07:01:40.002 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.002 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.002 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.002 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:3968 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 13369344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:40.002 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653442+0000", 2024-09-13T07:01:40.002 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2933430739999991, 2024-09-13T07:01:40.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099231100000000006, 2024-09-13T07:01:40.002 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.002 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.003 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.003 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3968 2024-09-13T07:01:40.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.003 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653442+0000", 2024-09-13T07:01:40.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653442+0000", 2024-09-13T07:01:40.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653440+0000", 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653447+0000", 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5359999999999998e-06 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653463+0000", 2024-09-13T07:01:40.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5956000000000001e-05 2024-09-13T07:01:40.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653476+0000", 2024-09-13T07:01:40.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2833999999999999e-05 2024-09-13T07:01:40.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663259+0000", 2024-09-13T07:01:40.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097832319999999993 2024-09-13T07:01:40.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663300+0000", 2024-09-13T07:01:40.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1414e-05 2024-09-13T07:01:40.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663365+0000", 2024-09-13T07:01:40.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4361000000000003e-05 2024-09-13T07:01:40.006 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.006 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.007 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:3969 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:40.007 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653487+0000", 2024-09-13T07:01:40.007 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2932976319999998, 2024-09-13T07:01:40.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010045950999999999, 2024-09-13T07:01:40.007 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.007 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.008 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3969 2024-09-13T07:01:40.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.008 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653487+0000", 2024-09-13T07:01:40.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653487+0000", 2024-09-13T07:01:40.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653486+0000", 2024-09-13T07:01:40.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:40.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653492+0000", 2024-09-13T07:01:40.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2890000000000003e-06 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653508+0000", 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.59e-05 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653521+0000", 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.272e-05 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663393+0000", 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098722719999999996 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663435+0000", 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1566999999999997e-05 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663533+0000", 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8239000000000006e-05 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.011 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.012 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4016 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:40.012 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653532+0000", 2024-09-13T07:01:40.012 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2932526029999991, 2024-09-13T07:01:40.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010191693, 2024-09-13T07:01:40.012 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.012 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.012 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4016 2024-09-13T07:01:40.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.013 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653532+0000", 2024-09-13T07:01:40.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653532+0000", 2024-09-13T07:01:40.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653531+0000", 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653537+0000", 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2559999999999999e-06 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653553+0000", 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5327000000000001e-05 2024-09-13T07:01:40.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653565+0000", 2024-09-13T07:01:40.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2563e-05 2024-09-13T07:01:40.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663578+0000", 2024-09-13T07:01:40.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010012951000000001 2024-09-13T07:01:40.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663634+0000", 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6276999999999997e-05 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663724+0000", 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9371999999999997e-05 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4017 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:40.016 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653577+0000", 2024-09-13T07:01:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2932077569999993, 2024-09-13T07:01:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010309476999999999, 2024-09-13T07:01:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4017 2024-09-13T07:01:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653577+0000", 2024-09-13T07:01:40.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653577+0000", 2024-09-13T07:01:40.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653576+0000", 2024-09-13T07:01:40.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:40.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653583+0000", 2024-09-13T07:01:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0620000000000003e-06 2024-09-13T07:01:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653598+0000", 2024-09-13T07:01:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.554e-05 2024-09-13T07:01:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653609+0000", 2024-09-13T07:01:40.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.078e-05 2024-09-13T07:01:40.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663756+0000", 2024-09-13T07:01:40.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010146887 2024-09-13T07:01:40.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663803+0000", 2024-09-13T07:01:40.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7497999999999998e-05 2024-09-13T07:01:40.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663886+0000", 2024-09-13T07:01:40.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3106e-05 2024-09-13T07:01:40.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.021 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.021 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4018 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:40.021 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653650+0000", 2024-09-13T07:01:40.021 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2931344790000008, 2024-09-13T07:01:40.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010399482, 2024-09-13T07:01:40.022 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.022 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.022 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.022 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.022 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.022 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4018 2024-09-13T07:01:40.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.022 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653650+0000", 2024-09-13T07:01:40.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653650+0000", 2024-09-13T07:01:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653649+0000", 2024-09-13T07:01:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653656+0000", 2024-09-13T07:01:40.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9739999999999996e-06 2024-09-13T07:01:40.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653690+0000", 2024-09-13T07:01:40.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4189000000000001e-05 2024-09-13T07:01:40.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653720+0000", 2024-09-13T07:01:40.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9621999999999999e-05 2024-09-13T07:01:40.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663938+0000", 2024-09-13T07:01:40.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010218287 2024-09-13T07:01:40.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663983+0000", 2024-09-13T07:01:40.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4807e-05 2024-09-13T07:01:40.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664050+0000", 2024-09-13T07:01:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7082999999999995e-05 2024-09-13T07:01:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4019 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653733+0000", 2024-09-13T07:01:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2930518939999995, 2024-09-13T07:01:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010478706000000001, 2024-09-13T07:01:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4019 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653733+0000", 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653733+0000", 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653732+0000", 2024-09-13T07:01:40.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:40.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653739+0000", 2024-09-13T07:01:40.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2699999999999999e-06 2024-09-13T07:01:40.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653770+0000", 2024-09-13T07:01:40.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1625999999999998e-05 2024-09-13T07:01:40.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653786+0000", 2024-09-13T07:01:40.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5383000000000002e-05 2024-09-13T07:01:40.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664087+0000", 2024-09-13T07:01:40.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010300741 2024-09-13T07:01:40.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664130+0000", 2024-09-13T07:01:40.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3470999999999998e-05 2024-09-13T07:01:40.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664211+0000", 2024-09-13T07:01:40.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1491999999999998e-05 2024-09-13T07:01:40.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.031 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.031 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.031 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4020 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:40.031 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653798+0000", 2024-09-13T07:01:40.031 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2929869249999992, 2024-09-13T07:01:40.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010592649000000001, 2024-09-13T07:01:40.031 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.031 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.032 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4020 2024-09-13T07:01:40.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.032 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653798+0000", 2024-09-13T07:01:40.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653798+0000", 2024-09-13T07:01:40.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653797+0000", 2024-09-13T07:01:40.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:40.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653803+0000", 2024-09-13T07:01:40.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5429999999999999e-06 2024-09-13T07:01:40.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653834+0000", 2024-09-13T07:01:40.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0825999999999999e-05 2024-09-13T07:01:40.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653863+0000", 2024-09-13T07:01:40.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8924999999999998e-05 2024-09-13T07:01:40.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664250+0000", 2024-09-13T07:01:40.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010387062000000001 2024-09-13T07:01:40.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664329+0000", 2024-09-13T07:01:40.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9209999999999995e-05 2024-09-13T07:01:40.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664390+0000", 2024-09-13T07:01:40.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1166000000000004e-05 2024-09-13T07:01:40.035 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.035 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.035 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4021 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653887+0000", 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2928976940000005, 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010633083999999999, 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4021 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.036 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653887+0000", 2024-09-13T07:01:40.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653887+0000", 2024-09-13T07:01:40.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653886+0000", 2024-09-13T07:01:40.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:40.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653899+0000", 2024-09-13T07:01:40.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2425999999999999e-05 2024-09-13T07:01:40.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653908+0000", 2024-09-13T07:01:40.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9089999999999995e-06 2024-09-13T07:01:40.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653916+0000", 2024-09-13T07:01:40.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7800000000000001e-06 2024-09-13T07:01:40.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664416+0000", 2024-09-13T07:01:40.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010499523 2024-09-13T07:01:40.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664459+0000", 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2787999999999998e-05 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664520+0000", 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1483999999999996e-05 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.041 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7628 3.1f 3:fdf2e50b:::rbd_data.1194c5936dae.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1032192~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:01:40.041 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:01:40.041 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.1843468420000001, 2024-09-13T07:01:40.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.630407088, 2024-09-13T07:01:40.041 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.041 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.041 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7628 2024-09-13T07:01:40.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.041 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762435+0000", 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:01:40.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762543+0000", 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010850500000000001 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762597+0000", 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4262000000000003e-05 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762626+0000", 2024-09-13T07:01:40.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9176e-05 2024-09-13T07:01:40.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762761+0000", 2024-09-13T07:01:40.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013405599999999999 2024-09-13T07:01:40.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:01:40.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762776+0000", 2024-09-13T07:01:40.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5679999999999999e-05 2024-09-13T07:01:40.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388492+0000", 2024-09-13T07:01:40.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.62571606300000004 2024-09-13T07:01:40.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for missing object", 2024-09-13T07:01:40.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388614+0000", 2024-09-13T07:01:40.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012126 2024-09-13T07:01:40.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390837+0000", 2024-09-13T07:01:40.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002223806 2024-09-13T07:01:40.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390963+0000", 2024-09-13T07:01:40.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012550300000000001 2024-09-13T07:01:40.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-13T07:01:40.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391127+0000", 2024-09-13T07:01:40.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163995 2024-09-13T07:01:40.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391989+0000", 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086230499999999995 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392662+0000", 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067260799999999999 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392728+0000", 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6501000000000006e-05 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392845+0000", 2024-09-13T07:01:40.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116665 2024-09-13T07:01:40.048 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.048 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.048 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.048 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11446 3.1d 3:ba689b8d:::rbd_data.1194b6604f63.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:40.048 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.895999+0000", 2024-09-13T07:01:40.048 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0507861329999999, 2024-09-13T07:01:40.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043645152999999999, 2024-09-13T07:01:40.048 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.049 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.049 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11446 2024-09-13T07:01:40.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.049 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.895999+0000", 2024-09-13T07:01:40.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.895999+0000", 2024-09-13T07:01:40.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.895997+0000", 2024-09-13T07:01:40.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:40.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.896009+0000", 2024-09-13T07:01:40.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2140999999999999e-05 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.896038+0000", 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9362000000000001e-05 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.896051+0000", 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948999999999999e-05 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.896161+0000", 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010956800000000001 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.896339+0000", 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000178495 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.905676+0000", 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009336881 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.939427+0000", 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033751100999999999 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.939456+0000", 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8904999999999999e-05 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.939559+0000", 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102532 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.939644+0000", 2024-09-13T07:01:40.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5048000000000003e-05 2024-09-13T07:01:40.054 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.054 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.054 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.054 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.054 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.054 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:40.055 INFO:teuthology.orchestra.run.smithi118.stdout:727 read 0x5826461 thru 0x5829d2d (0x38cd bytes) 2024-09-13T07:01:40.056 INFO:teuthology.orchestra.run.smithi118.stdout:728 trunc from 0x84b7f27 to 0x27c26bb 2024-09-13T07:01:40.056 INFO:teuthology.orchestra.run.smithi118.stdout:729 write 0x2217088 thru 0x2217956 (0x8cf bytes) 2024-09-13T07:01:40.056 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:01:40.057 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:40.049+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:40.081 INFO:teuthology.orchestra.run.smithi118.stdout:730 write 0x9f911e7 thru 0x9f935f0 (0x240a bytes) 2024-09-13T07:01:40.083 INFO:teuthology.orchestra.run.smithi118.stdout:731 writesame 0x9c5fd16 thru 0x9c6a5ab data_size 0x3(0xa896 bytes) 2024-09-13T07:01:40.119 INFO:teuthology.orchestra.run.smithi118.stdout:732 write 0xd7d27dc thru 0xd7dabe6 (0x840b bytes) 2024-09-13T07:01:40.126 INFO:teuthology.orchestra.run.smithi118.stdout:733 punch from 0x29e32fa to 0x29e4320, (0x1026 bytes) 2024-09-13T07:01:40.126 INFO:teuthology.orchestra.run.smithi118.stdout:734 read 0xb1a7d1e thru 0xb1b62b3 (0xe596 bytes) 2024-09-13T07:01:40.127 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:40.127 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:40.127 INFO:teuthology.orchestra.run.smithi118.stdout:738 read 0x9333a9f thru 0x93351c4 (0x1726 bytes) 2024-09-13T07:01:40.149 INFO:teuthology.orchestra.run.smithi118.stdout:739 trunc from 0xd7dabe7 to 0xb6e805 2024-09-13T07:01:40.156 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:40.153+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:40.171 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:40.172 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:40.175 INFO:teuthology.orchestra.run.smithi118.stdout:truncating to largest ever: 0xe69532a 2024-09-13T07:01:40.182 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:01:40.200 INFO:teuthology.orchestra.run.smithi118.stdout:740 trunc from 0xb6e805 to 0xe69532a 2024-09-13T07:01:40.212 INFO:teuthology.orchestra.run.smithi118.stdout:743 read 0x7a23f87 thru 0x7a28e01 (0x4e7b bytes) 2024-09-13T07:01:40.213 INFO:teuthology.orchestra.run.smithi118.stdout:744 punch from 0x27c8607 to 0x27cf4bb, (0x6eb4 bytes) 2024-09-13T07:01:40.213 INFO:teuthology.orchestra.run.smithi118.stdout:745 read 0xe17e6ec thru 0xe189bf3 (0xb508 bytes) 2024-09-13T07:01:40.214 INFO:teuthology.orchestra.run.smithi118.stdout:747 writesame 0xc86fe54 thru 0xc87d7aa data_size 0x1(0xd957 bytes) 2024-09-13T07:01:40.257 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:40.253+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:40.259 INFO:teuthology.orchestra.run.smithi118.stdout:748 write 0x7499d61 thru 0x74a8f40 (0xf1e0 bytes) 2024-09-13T07:01:40.263 INFO:teuthology.orchestra.run.smithi118.stdout:749 write 0x5e619e8 thru 0x5e65877 (0x3e90 bytes) 2024-09-13T07:01:40.265 INFO:teuthology.orchestra.run.smithi118.stdout:750 write 0xaecc094 thru 0xaed170c (0x5679 bytes) 2024-09-13T07:01:40.268 INFO:teuthology.orchestra.run.smithi118.stdout:751 write 0x8803a61 thru 0x880e836 (0xadd6 bytes) 2024-09-13T07:01:40.270 INFO:teuthology.orchestra.run.smithi118.stdout:753 writesame 0x36f9408 thru 0x3707f56 data_size 0x1(0xeb4f bytes) 2024-09-13T07:01:40.295 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:40.296 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:40.306 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:01:40.313 INFO:teuthology.orchestra.run.smithi118.stdout:756 trunc from 0xe69532a to 0x6d1c020 2024-09-13T07:01:40.352 INFO:teuthology.orchestra.run.smithi118.stdout:758 writesame 0x6e94eef thru 0x6e9ef67 data_size 0x1(0xa079 bytes) 2024-09-13T07:01:40.359 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:40.357+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:40.361 INFO:teuthology.orchestra.run.smithi118.stdout:759 trunc from 0x6e9ef68 to 0x3bb51db 2024-09-13T07:01:40.376 INFO:teuthology.orchestra.run.smithi118.stdout:761 write 0x2d2358a thru 0x2d32aaa (0xf521 bytes) 2024-09-13T07:01:40.400 INFO:teuthology.orchestra.run.smithi118.stdout:762 trunc from 0x3bb51db to 0xd1df102 2024-09-13T07:01:40.413 INFO:teuthology.orchestra.run.smithi118.stdout:763 read 0x257b8cd thru 0x25840ee (0x8822 bytes) 2024-09-13T07:01:40.414 INFO:teuthology.orchestra.run.smithi118.stdout:764 punch from 0x1a4de3b to 0x1a56484, (0x8649 bytes) 2024-09-13T07:01:40.414 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:40.414 INFO:teuthology.orchestra.run.smithi118.stdout:767 read 0x2eb5de3 thru 0x2ec2d81 (0xcf9f bytes) 2024-09-13T07:01:40.415 INFO:teuthology.orchestra.run.smithi118.stdout:769 write 0x694a428 thru 0x6954dfe (0xa9d7 bytes) 2024-09-13T07:01:40.416 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:581 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.416139+0000", 2024-09-13T07:01:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.996046960999999, 2024-09-13T07:01:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061636429999999999, 2024-09-13T07:01:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 581 2024-09-13T07:01:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416139+0000", 2024-09-13T07:01:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416139+0000", 2024-09-13T07:01:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416136+0000", 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416171+0000", 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4545000000000002e-05 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416203+0000", 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2085000000000002e-05 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416224+0000", 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1157000000000001e-05 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416339+0000", 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115455 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416349+0000", 2024-09-13T07:01:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5729999999999997e-06 2024-09-13T07:01:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.419615+0000", 2024-09-13T07:01:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032657139999999999 2024-09-13T07:01:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.419766+0000", 2024-09-13T07:01:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015095000000000001 2024-09-13T07:01:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.420094+0000", 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032800599999999999 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.421267+0000", 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011733889999999999 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.422161+0000", 2024-09-13T07:01:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089368499999999997 2024-09-13T07:01:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.422233+0000", 2024-09-13T07:01:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2619000000000004e-05 2024-09-13T07:01:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.422303+0000", 2024-09-13T07:01:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9264999999999995e-05 2024-09-13T07:01:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:949 3.a 3:52cbfadd:::rbd_data.1194d22ad098.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.495440+0000", 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.916745581000001, 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005241791, 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 949 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495440+0000", 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495440+0000", 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495439+0000", 2024-09-13T07:01:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495446+0000", 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2269999999999996e-06 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495460+0000", 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3247999999999999e-05 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495467+0000", 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5129999999999999e-06 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495521+0000", 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4020000000000001e-05 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495618+0000", 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6934000000000004e-05 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495860+0000", 2024-09-13T07:01:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024149800000000001 2024-09-13T07:01:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500444+0000", 2024-09-13T07:01:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045838909999999997 2024-09-13T07:01:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500595+0000", 2024-09-13T07:01:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015186099999999999 2024-09-13T07:01:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500648+0000", 2024-09-13T07:01:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2877999999999998e-05 2024-09-13T07:01:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500682+0000", 2024-09-13T07:01:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3818e-05 2024-09-13T07:01:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.353265208, 2024-09-13T07:01:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:01:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:01:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:01:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:01:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:01:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:01:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:01:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:01:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:01:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:01:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:01:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:01:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:01:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:01:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:01:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:01:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:01:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:01:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:01:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1534 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14365141] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.346437656999999, 2024-09-13T07:01:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040478761000000002, 2024-09-13T07:01:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1534 2024-09-13T07:01:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065747+0000", 2024-09-13T07:01:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075258+0000", 2024-09-13T07:01:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095108710000000006 2024-09-13T07:01:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075288+0000", 2024-09-13T07:01:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0085000000000001e-05 2024-09-13T07:01:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075305+0000", 2024-09-13T07:01:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7748e-05 2024-09-13T07:01:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075367+0000", 2024-09-13T07:01:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1867999999999998e-05 2024-09-13T07:01:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075520+0000", 2024-09-13T07:01:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015298800000000001 2024-09-13T07:01:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.076247+0000", 2024-09-13T07:01:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072720499999999997 2024-09-13T07:01:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105157+0000", 2024-09-13T07:01:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028909642999999999 2024-09-13T07:01:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106124+0000", 2024-09-13T07:01:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096729400000000005 2024-09-13T07:01:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106173+0000", 2024-09-13T07:01:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8196999999999999e-05 2024-09-13T07:01:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106227+0000", 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4388999999999997e-05 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1557 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.271671987, 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012339057, 2024-09-13T07:01:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1557 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140502+0000", 2024-09-13T07:01:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999881 2024-09-13T07:01:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140526+0000", 2024-09-13T07:01:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4049999999999998e-05 2024-09-13T07:01:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140563+0000", 2024-09-13T07:01:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6569000000000002e-05 2024-09-13T07:01:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140580+0000", 2024-09-13T07:01:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7725000000000001e-05 2024-09-13T07:01:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140942+0000", 2024-09-13T07:01:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036136300000000002 2024-09-13T07:01:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.141130+0000", 2024-09-13T07:01:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018814999999999999 2024-09-13T07:01:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.141433+0000", 2024-09-13T07:01:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030329899999999998 2024-09-13T07:01:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.147925+0000", 2024-09-13T07:01:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064915240000000003 2024-09-13T07:01:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152757+0000", 2024-09-13T07:01:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048328709999999999 2024-09-13T07:01:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152802+0000", 2024-09-13T07:01:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4314000000000003e-05 2024-09-13T07:01:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152853+0000", 2024-09-13T07:01:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1149000000000001e-05 2024-09-13T07:01:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1558 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.271574249, 2024-09-13T07:01:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012353627000000001, 2024-09-13T07:01:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1558 2024-09-13T07:01:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140610+0000", 2024-09-13T07:01:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140619+0000", 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8650000000000007e-06 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140634+0000", 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4515000000000001e-05 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140646+0000", 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1736000000000001e-05 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.144258+0000", 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036123499999999998 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.145034+0000", 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00077638900000000005 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.145395+0000", 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036117499999999998 2024-09-13T07:01:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.149775+0000", 2024-09-13T07:01:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043795639999999999 2024-09-13T07:01:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152906+0000", 2024-09-13T07:01:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031306929999999999 2024-09-13T07:01:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152941+0000", 2024-09-13T07:01:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4919000000000002e-05 2024-09-13T07:01:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152965+0000", 2024-09-13T07:01:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4641e-05 2024-09-13T07:01:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1648 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,zero 10485760~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.297923+0000", 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.114262767, 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053690209999999999, 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1648 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297923+0000", 2024-09-13T07:01:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297923+0000", 2024-09-13T07:01:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297920+0000", 2024-09-13T07:01:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:01:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297941+0000", 2024-09-13T07:01:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1628999999999999e-05 2024-09-13T07:01:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297986+0000", 2024-09-13T07:01:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4135999999999999e-05 2024-09-13T07:01:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298009+0000", 2024-09-13T07:01:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3206000000000001e-05 2024-09-13T07:01:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298236+0000", 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000227471 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298526+0000", 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028929800000000001 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.303292+0000", 2024-09-13T07:01:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047665570000000003 2024-09-13T07:01:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1649 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 10485760~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.298045+0000", 2024-09-13T07:01:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.114141329000001, 2024-09-13T07:01:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055560130000000003, 2024-09-13T07:01:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1649 2024-09-13T07:01:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298045+0000", 2024-09-13T07:01:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298045+0000", 2024-09-13T07:01:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298043+0000", 2024-09-13T07:01:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298052+0000", 2024-09-13T07:01:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4829999999999999e-06 2024-09-13T07:01:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298069+0000", 2024-09-13T07:01:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7073000000000001e-05 2024-09-13T07:01:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298083+0000", 2024-09-13T07:01:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3977e-05 2024-09-13T07:01:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.300163+0000", 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020800229999999999 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.300370+0000", 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000207679 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.303601+0000", 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032301550000000002 2024-09-13T07:01:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1661 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [read 551881~5223] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.317063+0000", 2024-09-13T07:01:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.095122413, 2024-09-13T07:01:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066193119999999996, 2024-09-13T07:01:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1661 2024-09-13T07:01:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317063+0000", 2024-09-13T07:01:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317063+0000", 2024-09-13T07:01:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317062+0000", 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317070+0000", 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9370000000000004e-06 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317086+0000", 2024-09-13T07:01:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5685e-05 2024-09-13T07:01:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317093+0000", 2024-09-13T07:01:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6190000000000002e-06 2024-09-13T07:01:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317144+0000", 2024-09-13T07:01:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0587999999999997e-05 2024-09-13T07:01:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317148+0000", 2024-09-13T07:01:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0949999999999998e-06 2024-09-13T07:01:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.323168+0000", 2024-09-13T07:01:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060199110000000002 2024-09-13T07:01:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.323288+0000", 2024-09-13T07:01:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120375 2024-09-13T07:01:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.323683+0000", 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00039459599999999999 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1676 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.071296045, 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067554429999999999, 2024-09-13T07:01:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1676 2024-09-13T07:01:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340889+0000", 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340896+0000", 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1420000000000004e-06 2024-09-13T07:01:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340909+0000", 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3329000000000001e-05 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340920+0000", 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1029999999999999e-05 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341653+0000", 2024-09-13T07:01:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073243400000000001 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341825+0000", 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017243099999999999 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.347645+0000", 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058200639999999998 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1761 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.454709+0000", 2024-09-13T07:01:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.957476549000001, 2024-09-13T07:01:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049941610000000004, 2024-09-13T07:01:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1761 2024-09-13T07:01:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454709+0000", 2024-09-13T07:01:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454709+0000", 2024-09-13T07:01:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454708+0000", 2024-09-13T07:01:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454714+0000", 2024-09-13T07:01:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1970000000000001e-06 2024-09-13T07:01:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454728+0000", 2024-09-13T07:01:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3212e-05 2024-09-13T07:01:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454738+0000", 2024-09-13T07:01:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0475000000000001e-05 2024-09-13T07:01:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.455883+0000", 2024-09-13T07:01:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001144438 2024-09-13T07:01:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.456054+0000", 2024-09-13T07:01:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000171402 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.459703+0000", 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036494819999999999 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1808 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.542510+0000", 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.869675843, 2024-09-13T07:01:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050197829999999999, 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1808 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542510+0000", 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542510+0000", 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542509+0000", 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542516+0000", 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2200000000000003e-06 2024-09-13T07:01:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542533+0000", 2024-09-13T07:01:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6957e-05 2024-09-13T07:01:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542549+0000", 2024-09-13T07:01:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5959000000000001e-05 2024-09-13T07:01:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.545392+0000", 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028432330000000001 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.545520+0000", 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012757400000000001 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.545722+0000", 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020242400000000001 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547178+0000", 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014557509999999999 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547448+0000", 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026981900000000003 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547496+0000", 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8680000000000001e-05 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547530+0000", 2024-09-13T07:01:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3466000000000001e-05 2024-09-13T07:01:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1854 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,zero 4194304~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.814176326, 2024-09-13T07:01:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070961979999999997, 2024-09-13T07:01:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1854 2024-09-13T07:01:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598007+0000", 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598022+0000", 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598062+0000", 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9552999999999999e-05 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598081+0000", 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9247999999999999e-05 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598256+0000", 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000174482 2024-09-13T07:01:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598365+0000", 2024-09-13T07:01:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109108 2024-09-13T07:01:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.605106+0000", 2024-09-13T07:01:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067409189999999997 2024-09-13T07:01:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1855 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.814068045999999, 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074110900000000004, 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1855 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598117+0000", 2024-09-13T07:01:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598125+0000", 2024-09-13T07:01:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3110000000000006e-06 2024-09-13T07:01:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598147+0000", 2024-09-13T07:01:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2484999999999999e-05 2024-09-13T07:01:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598159+0000", 2024-09-13T07:01:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1939999999999999e-05 2024-09-13T07:01:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.599204+0000", 2024-09-13T07:01:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010449680000000001 2024-09-13T07:01:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.599409+0000", 2024-09-13T07:01:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020456700000000001 2024-09-13T07:01:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.605529+0000", 2024-09-13T07:01:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061199790000000002 2024-09-13T07:01:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:3373 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.464963+0000", 2024-09-13T07:01:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.9472223989999993, 2024-09-13T07:01:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050049359999999998, 2024-09-13T07:01:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3373 2024-09-13T07:01:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.464963+0000", 2024-09-13T07:01:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.464963+0000", 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.464961+0000", 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.464974+0000", 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2679e-05 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.464999+0000", 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5488000000000001e-05 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.465010+0000", 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0526999999999999e-05 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.465088+0000", 2024-09-13T07:01:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7855000000000006e-05 2024-09-13T07:01:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.465094+0000", 2024-09-13T07:01:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.348e-06 2024-09-13T07:01:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.467866+0000", 2024-09-13T07:01:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002772114 2024-09-13T07:01:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.467956+0000", 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9203999999999995e-05 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.468141+0000", 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018521600000000001 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.469125+0000", 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00098440099999999998 2024-09-13T07:01:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.469884+0000", 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075881199999999998 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.469931+0000", 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6601000000000003e-05 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.469968+0000", 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7889000000000003e-05 2024-09-13T07:01:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7595 3.a 3:54f0339c:::rbd_data.1194c5936dae.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 564664~8776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.687839+0000", 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.724346776, 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013545, 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7595 2024-09-13T07:01:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.687839+0000", 2024-09-13T07:01:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.687839+0000", 2024-09-13T07:01:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.687836+0000", 2024-09-13T07:01:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690295+0000", 2024-09-13T07:01:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024590800000000002 2024-09-13T07:01:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690412+0000", 2024-09-13T07:01:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116197 2024-09-13T07:01:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690445+0000", 2024-09-13T07:01:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2947999999999997e-05 2024-09-13T07:01:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690548+0000", 2024-09-13T07:01:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103193 2024-09-13T07:01:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690733+0000", 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018520799999999999 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691055+0000", 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032229699999999998 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.695214+0000", 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041589050000000001 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701310+0000", 2024-09-13T07:01:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060959370000000001 2024-09-13T07:01:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701352+0000", 2024-09-13T07:01:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:01:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701384+0000", 2024-09-13T07:01:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1612999999999997e-05 2024-09-13T07:01:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7596 3.10 3:085a9411:::rbd_data.1194c5936dae.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 557056~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:01:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.690504+0000", 2024-09-13T07:01:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7216820039999998, 2024-09-13T07:01:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011135499, 2024-09-13T07:01:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7596 2024-09-13T07:01:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690504+0000", 2024-09-13T07:01:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690504+0000", 2024-09-13T07:01:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690503+0000", 2024-09-13T07:01:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691953+0000", 2024-09-13T07:01:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014497920000000001 2024-09-13T07:01:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691978+0000", 2024-09-13T07:01:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5384999999999998e-05 2024-09-13T07:01:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691991+0000", 2024-09-13T07:01:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.261e-05 2024-09-13T07:01:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.692271+0000", 2024-09-13T07:01:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028019399999999999 2024-09-13T07:01:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.692541+0000", 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026980400000000001 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.693006+0000", 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046576499999999998 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701109+0000", 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081030630000000006 2024-09-13T07:01:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701549+0000", 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043989499999999999 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701606+0000", 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6292999999999998e-05 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701639+0000", 2024-09-13T07:01:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3741000000000001e-05 2024-09-13T07:01:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11218 3.10 3:0ba898fc:::rbd_data.1194a7cedc35.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.629190+0000", 2024-09-13T07:01:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.7829955069999999, 2024-09-13T07:01:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070817470000000002, 2024-09-13T07:01:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11218 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.629190+0000", 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.629190+0000", 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.629188+0000", 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631056+0000", 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018678989999999999 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631082+0000", 2024-09-13T07:01:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6156000000000002e-05 2024-09-13T07:01:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631093+0000", 2024-09-13T07:01:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0696999999999999e-05 2024-09-13T07:01:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631216+0000", 2024-09-13T07:01:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012328299999999999 2024-09-13T07:01:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631435+0000", 2024-09-13T07:01:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021897299999999999 2024-09-13T07:01:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631884+0000", 2024-09-13T07:01:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044842900000000003 2024-09-13T07:01:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636066+0000", 2024-09-13T07:01:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041825059999999999 2024-09-13T07:01:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636133+0000", 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7000000000000002e-05 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636213+0000", 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9671000000000003e-05 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636272+0000", 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9126000000000003e-05 2024-09-13T07:01:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11219 3.10 3:0ba898fc:::rbd_data.1194a7cedc35.0000000000000009:head [truncate 2621440] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.631119+0000", 2024-09-13T07:01:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.7810671120000001, 2024-09-13T07:01:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054998979999999996, 2024-09-13T07:01:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11219 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631119+0000", 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631119+0000", 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631118+0000", 2024-09-13T07:01:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631123+0000", 2024-09-13T07:01:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7849999999999999e-06 2024-09-13T07:01:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631191+0000", 2024-09-13T07:01:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8246999999999999e-05 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631251+0000", 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0649000000000002e-05 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.634626+0000", 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033748960000000001 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.634700+0000", 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3393999999999996e-05 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.634878+0000", 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017833399999999999 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636330+0000", 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014522350000000001 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636524+0000", 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000194124 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636567+0000", 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2178000000000002e-05 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636619+0000", 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.206e-05 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11408 3.10 3:0f48178c:::rbd_data.1194b6604f63.000000000000000c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.833470+0000", 2024-09-13T07:01:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.578716201, 2024-09-13T07:01:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054053920000000002, 2024-09-13T07:01:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11408 2024-09-13T07:01:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.833470+0000", 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.833470+0000", 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.833468+0000", 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.833482+0000", 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3957e-05 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.833513+0000", 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1605e-05 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.833534+0000", 2024-09-13T07:01:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0528000000000001e-05 2024-09-13T07:01:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.836351+0000", 2024-09-13T07:01:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028175710000000001 2024-09-13T07:01:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.836560+0000", 2024-09-13T07:01:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000208934 2024-09-13T07:01:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.838875+0000", 2024-09-13T07:01:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002314927 2024-09-13T07:01:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:40.528 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:40.529 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:01:40.530 INFO:teuthology.orchestra.run.smithi118.stdout:770 writesame 0xc8009d9 thru 0xc8063ad data_size 0xd(0x59d5 bytes) 2024-09-13T07:01:40.530 INFO:teuthology.orchestra.run.smithi118.stdout:771 read 0x6186a80 thru 0x6189674 (0x2bf5 bytes) 2024-09-13T07:01:40.530 INFO:teuthology.orchestra.run.smithi118.stdout:772 read 0xa48d7bb thru 0xa4906f3 (0x2f39 bytes) 2024-09-13T07:01:40.530 INFO:teuthology.orchestra.run.smithi118.stdout:773 read 0xa9b4b93 thru 0xa9c3193 (0xe601 bytes) 2024-09-13T07:01:40.531 INFO:teuthology.orchestra.run.smithi118.stderr:marked out osd.2. 2024-09-13T07:01:40.531 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:01:40.532 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:40.525+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:40.532 INFO:teuthology.orchestra.run.smithi118.stdout:774 punch from 0x7ce1559 to 0x7cf0f41, (0xf9e8 bytes) 2024-09-13T07:01:40.532 INFO:teuthology.orchestra.run.smithi118.stdout:775 writesame 0xea5925c thru 0xea5cc21 data_size 0x11(0x39c6 bytes) 2024-09-13T07:01:40.532 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:40.532 INFO:teuthology.orchestra.run.smithi118.stdout:777 punch from 0x1d0a01c to 0x1d17808, (0xd7ec bytes) 2024-09-13T07:01:40.534 INFO:teuthology.orchestra.run.smithi118.stdout:779 trunc from 0xea5cc22 to 0x15af6ae 2024-09-13T07:01:40.534 INFO:teuthology.orchestra.run.smithi118.stdout:782 trunc from 0x15af6ae to 0x94140 2024-09-13T07:01:40.534 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:40.534 INFO:teuthology.orchestra.run.smithi118.stdout:785 punch from 0x97a7 to 0x132ea, (0x9b43 bytes) 2024-09-13T07:01:40.534 INFO:teuthology.orchestra.run.smithi118.stdout:786 read 0x7ef74 thru 0x83ecb (0x4f58 bytes) 2024-09-13T07:01:40.534 INFO:teuthology.orchestra.run.smithi118.stdout:787 read 0x72521 thru 0x7a1e8 (0x7cc8 bytes) 2024-09-13T07:01:40.549 INFO:teuthology.orchestra.run.smithi118.stdout:789 writesame 0x9504eca thru 0x9508202 data_size 0x2f(0x3339 bytes) 2024-09-13T07:01:40.551 INFO:teuthology.orchestra.run.smithi118.stdout:791 punch from 0x4a8bf6 to 0x4b4691, (0xba9b bytes) 2024-09-13T07:01:40.552 INFO:teuthology.orchestra.run.smithi118.stdout:792 write 0x5fa8142 thru 0x5fb6c75 (0xeb34 bytes) 2024-09-13T07:01:40.572 INFO:teuthology.orchestra.run.smithi118.stdout:793 write 0xd1c6f53 thru 0xd1d59ed (0xea9b bytes) 2024-09-13T07:01:40.581 INFO:teuthology.orchestra.run.smithi118.stdout:795 trunc from 0xd1d59ee to 0xae1072c 2024-09-13T07:01:40.607 INFO:teuthology.orchestra.run.smithi118.stdout:798 read 0x4a87736 thru 0x4a91659 (0x9f24 bytes) 2024-09-13T07:01:40.608 INFO:teuthology.orchestra.run.smithi118.stdout:800 write 0x4c87b12 thru 0x4c89bb4 (0x20a3 bytes) 2024-09-13T07:01:40.610 INFO:teuthology.orchestra.run.smithi118.stdout:802 read 0xefee0e thru 0xf0a3a2 (0xb595 bytes) 2024-09-13T07:01:40.610 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:40.611 INFO:teuthology.orchestra.run.smithi118.stdout:805 write 0x14b92c3 thru 0x14c2adc (0x981a bytes) 2024-09-13T07:01:40.615 INFO:teuthology.orchestra.run.smithi118.stdout:807 read 0x447a54a thru 0x4482033 (0x7aea bytes) 2024-09-13T07:01:40.615 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:40.616 INFO:teuthology.orchestra.run.smithi118.stdout:809 read 0x16aa007 thru 0x16aa2cd (0x2c7 bytes) 2024-09-13T07:01:40.616 INFO:teuthology.orchestra.run.smithi118.stdout:810 punch from 0x708624 to 0x7104e6, (0x7ec2 bytes) 2024-09-13T07:01:40.631 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:40.625+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:40.633 INFO:teuthology.orchestra.run.smithi118.stdout:811 write 0xcc7b35e thru 0xcc7ec32 (0x38d5 bytes) 2024-09-13T07:01:40.640 INFO:teuthology.orchestra.run.smithi118.stdout:812 write 0xbe26aea thru 0xbe2cbd0 (0x60e7 bytes) 2024-09-13T07:01:40.643 INFO:teuthology.orchestra.run.smithi118.stdout:813 read 0x2d91f24 thru 0x2da0f08 (0xefe5 bytes) 2024-09-13T07:01:40.644 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:40.644 INFO:teuthology.orchestra.run.smithi118.stdout:815 read 0x5bc184a thru 0x5bcb6cf (0x9e86 bytes) 2024-09-13T07:01:40.650 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:40.650 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:40.650 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:40.651 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:40.660 INFO:teuthology.orchestra.run.smithi118.stdout:816 trunc from 0xcc7ec33 to 0x4b75f68 2024-09-13T07:01:40.661 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:01:40.679 INFO:teuthology.orchestra.run.smithi118.stdout:817 punch from 0x2130437 to 0x213276c, (0x2335 bytes) 2024-09-13T07:01:40.680 INFO:teuthology.orchestra.run.smithi118.stdout:818 trunc from 0x4b75f68 to 0x456676d 2024-09-13T07:01:40.692 INFO:teuthology.orchestra.run.smithi118.stdout:819 write 0xb8e0c2 thru 0xb8e804 (0x743 bytes) 2024-09-13T07:01:40.700 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:40.718 INFO:teuthology.orchestra.run.smithi118.stdout:820 write 0xcd36d95 thru 0xcd4143e (0xa6aa bytes) 2024-09-13T07:01:40.726 INFO:teuthology.orchestra.run.smithi118.stdout:822 write 0x53b80dd thru 0x53c14e3 (0x9407 bytes) 2024-09-13T07:01:40.727 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:40.729 INFO:teuthology.orchestra.run.smithi118.stdout:823 read 0xabbd83d thru 0xabcbcfa (0xe4be bytes) 2024-09-13T07:01:40.730 INFO:teuthology.orchestra.run.smithi118.stdout:824 writesame 0x926f6c4 thru 0x927738c data_size 0x1(0x7cc9 bytes) 2024-09-13T07:01:40.731 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:40.729+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:40.744 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:40.754 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:40.755 INFO:teuthology.orchestra.run.smithi118.stdout:827 write 0x269d6a6 thru 0x26a196d (0x42c8 bytes) 2024-09-13T07:01:40.757 INFO:teuthology.orchestra.run.smithi118.stdout:828 read 0x2a56c45 thru 0x2a60203 (0x95bf bytes) 2024-09-13T07:01:40.764 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:40.768 INFO:teuthology.orchestra.run.smithi118.stdout:830 write 0xd88471d thru 0xd891e92 (0xd776 bytes) 2024-09-13T07:01:40.771 INFO:teuthology.orchestra.run.smithi118.stdout:831 read 0x38f71ee thru 0x38ff09e (0x7eb1 bytes) 2024-09-13T07:01:40.771 INFO:teuthology.orchestra.run.smithi118.stdout:836 read 0x1bef547 thru 0x1bfc06d (0xcb27 bytes) 2024-09-13T07:01:40.784 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:40.785 INFO:teuthology.orchestra.run.smithi118.stdout:840 trunc from 0xd891e93 to 0x594b1f6 2024-09-13T07:01:40.806 INFO:teuthology.orchestra.run.smithi118.stdout:841 read 0x1115c49 thru 0x1124328 (0xe6e0 bytes) 2024-09-13T07:01:40.807 INFO:teuthology.orchestra.run.smithi118.stdout:844 write 0x21485c7 thru 0x2150fd4 (0x8a0e bytes) 2024-09-13T07:01:40.808 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:40.816 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:40.816 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:40.816 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:40.816 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:40.816 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:40.819 INFO:teuthology.orchestra.run.smithi118.stdout:847 write 0x66f042f thru 0x66fdf33 (0xdb05 bytes) 2024-09-13T07:01:40.822 INFO:teuthology.orchestra.run.smithi118.stdout:848 write 0x1f343dc thru 0x1f3aa85 (0x66aa bytes) 2024-09-13T07:01:40.826 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:01:40.831 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:40.825+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:40.832 INFO:teuthology.orchestra.run.smithi118.stdout:850 trunc from 0x66fdf34 to 0x2f91311 2024-09-13T07:01:40.834 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:40.848 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:40.848 INFO:teuthology.orchestra.run.smithi118.stdout:852 read 0x116fae8 thru 0x1174a1a (0x4f33 bytes) 2024-09-13T07:01:40.848 INFO:teuthology.orchestra.run.smithi118.stdout:854 punch from 0x1f06b0b to 0x1f0b4b3, (0x49a8 bytes) 2024-09-13T07:01:40.848 INFO:teuthology.orchestra.run.smithi118.stdout:857 clone 12 order 23 su 4194304 sc 6 2024-09-13T07:01:40.857 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:40.860 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:40.860 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:40.860 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:40.860 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:40.860 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:40.861 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:40.861 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:40.861 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:40.931 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:40.931 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:40.933 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:40.929+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:40.941 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:01:41.034 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:41.033+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:41.059 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:41.060 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:41.070 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:01:41.135 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:41.129+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:41.180 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:41.180 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:41.190 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:01:41.236 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:41.233+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:41.294 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:41.294 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:41.303 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:01:41.337 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:41.333+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:41.438 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:41.433+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:41.439 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:01:41.471 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:41.471 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:41.483 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:01:41.538 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:41.533+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:41.626 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:41.626 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:41.636 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:01:41.639 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:41.633+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:41.739 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:41.739 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:41.739 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:41.733+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:41.748 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:01:41.840 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:41.837+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:41.849 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:41.849 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:41.858 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:01:41.941 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:41.937+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:41.964 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:41.965 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:41.975 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:01:42.042 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:42.041+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:42.078 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:42.078 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:42.087 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:01:42.143 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:42.138+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:42.239 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:42.239 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:42.243 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:42.238+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:42.248 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:01:42.344 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:42.342+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:42.351 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:42.351 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:42.361 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:01:42.441 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:01:42.444 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:42.441+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:42.470 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:42.470 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:42.479 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:01:42.546 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:42.545+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:42.598 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:42.598 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:42.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:01:42.646 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:42.645+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:42.716 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:42.716 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:42.725 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:01:42.747 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:42.745+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:42.832 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:42.832 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:42.842 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:01:42.848 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:42.846+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:42.872 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:01:42.945 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:42.945 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:42.950 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:42.949+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:42.955 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:01:43.025 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:43.046 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:43.050 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:43.049+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:43.059 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:43.059 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:43.066 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:43.069 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:01:43.087 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:43.107 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:43.134 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:43.150 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:43.149+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:43.156 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:43.179 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:43.181 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:43.182 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:43.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:43.182 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:43.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.182 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:439 3.16 3:6cdf2866:::rbd_data.1194d22ad098.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:43.182 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.539100+0000", 2024-09-13T07:01:43.182 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.638997898, 2024-09-13T07:01:43.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01241133, 2024-09-13T07:01:43.182 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.182 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.182 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.182 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.183 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.183 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 439 2024-09-13T07:01:43.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.183 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539100+0000", 2024-09-13T07:01:43.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539100+0000", 2024-09-13T07:01:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539098+0000", 2024-09-13T07:01:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539113+0000", 2024-09-13T07:01:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5653000000000001e-05 2024-09-13T07:01:43.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539148+0000", 2024-09-13T07:01:43.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4489000000000001e-05 2024-09-13T07:01:43.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539162+0000", 2024-09-13T07:01:43.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4533e-05 2024-09-13T07:01:43.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539297+0000", 2024-09-13T07:01:43.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013447399999999999 2024-09-13T07:01:43.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539701+0000", 2024-09-13T07:01:43.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00040402400000000001 2024-09-13T07:01:43.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:43.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.542876+0000", 2024-09-13T07:01:43.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031755820000000001 2024-09-13T07:01:43.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551351+0000", 2024-09-13T07:01:43.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084747239999999995 2024-09-13T07:01:43.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551392+0000", 2024-09-13T07:01:43.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0921999999999999e-05 2024-09-13T07:01:43.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551457+0000", 2024-09-13T07:01:43.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4925999999999995e-05 2024-09-13T07:01:43.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.551511+0000", 2024-09-13T07:01:43.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4446e-05 2024-09-13T07:01:43.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.188 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.189 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:440 3.1c 3:3b975cee:::rbd_data.1194d22ad098.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:43.189 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:24.539301+0000", 2024-09-13T07:01:43.189 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.638797339, 2024-09-13T07:01:43.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019168998999999999, 2024-09-13T07:01:43.189 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.189 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.189 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 440 2024-09-13T07:01:43.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.189 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539301+0000", 2024-09-13T07:01:43.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539301+0000", 2024-09-13T07:01:43.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539300+0000", 2024-09-13T07:01:43.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:43.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539308+0000", 2024-09-13T07:01:43.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-06 2024-09-13T07:01:43.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539347+0000", 2024-09-13T07:01:43.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9063999999999997e-05 2024-09-13T07:01:43.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539353+0000", 2024-09-13T07:01:43.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3459999999999999e-06 2024-09-13T07:01:43.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539438+0000", 2024-09-13T07:01:43.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4757000000000002e-05 2024-09-13T07:01:43.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.539706+0000", 2024-09-13T07:01:43.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026819000000000002 2024-09-13T07:01:43.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:43.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.543844+0000", 2024-09-13T07:01:43.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004137338 2024-09-13T07:01:43.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558385+0000", 2024-09-13T07:01:43.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014541304999999999 2024-09-13T07:01:43.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558397+0000", 2024-09-13T07:01:43.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2269999999999999e-05 2024-09-13T07:01:43.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558437+0000", 2024-09-13T07:01:43.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0096999999999999e-05 2024-09-13T07:01:43.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:24.558470+0000", 2024-09-13T07:01:43.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2242000000000001e-05 2024-09-13T07:01:43.195 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.195 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.195 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.195 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1531 3.c 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:43.196 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:43.196 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.128586218999999, 2024-09-13T07:01:43.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063168486999999995, 2024-09-13T07:01:43.196 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.196 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.196 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.196 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.196 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.196 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1531 2024-09-13T07:01:43.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.196 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:43.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:43.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049511+0000", 2024-09-13T07:01:43.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:43.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069234+0000", 2024-09-13T07:01:43.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019723920999999998 2024-09-13T07:01:43.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069269+0000", 2024-09-13T07:01:43.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4446999999999997e-05 2024-09-13T07:01:43.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069286+0000", 2024-09-13T07:01:43.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7405999999999999e-05 2024-09-13T07:01:43.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069370+0000", 2024-09-13T07:01:43.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-05 2024-09-13T07:01:43.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069511+0000", 2024-09-13T07:01:43.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014134299999999999 2024-09-13T07:01:43.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:43.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070463+0000", 2024-09-13T07:01:43.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095167899999999998 2024-09-13T07:01:43.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105076+0000", 2024-09-13T07:01:43.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034613139000000001 2024-09-13T07:01:43.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112562+0000", 2024-09-13T07:01:43.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074861429999999998 2024-09-13T07:01:43.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112631+0000", 2024-09-13T07:01:43.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:01:43.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112680+0000", 2024-09-13T07:01:43.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9607000000000001e-05 2024-09-13T07:01:43.202 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.202 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.202 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.202 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1794 3.16 3:6a67e608:::rbd_data.11943a018f34.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:43.202 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.510292+0000", 2024-09-13T07:01:43.202 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.667806314, 2024-09-13T07:01:43.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015887680000000001, 2024-09-13T07:01:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1794 2024-09-13T07:01:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510292+0000", 2024-09-13T07:01:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510292+0000", 2024-09-13T07:01:43.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510289+0000", 2024-09-13T07:01:43.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:43.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510303+0000", 2024-09-13T07:01:43.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3933e-05 2024-09-13T07:01:43.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510335+0000", 2024-09-13T07:01:43.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2169000000000003e-05 2024-09-13T07:01:43.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510349+0000", 2024-09-13T07:01:43.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4205999999999999e-05 2024-09-13T07:01:43.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510458+0000", 2024-09-13T07:01:43.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010835100000000001 2024-09-13T07:01:43.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.510597+0000", 2024-09-13T07:01:43.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013911699999999999 2024-09-13T07:01:43.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:43.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.513457+0000", 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028606439999999999 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.526058+0000", 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012600849000000001 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.526090+0000", 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2036999999999998e-05 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.526139+0000", 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9042999999999998e-05 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.526179+0000", 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0077999999999998e-05 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:3968 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 13369344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:43.208 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653442+0000", 2024-09-13T07:01:43.209 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.524656294, 2024-09-13T07:01:43.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099231100000000006, 2024-09-13T07:01:43.209 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.209 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.209 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.209 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.209 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.209 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3968 2024-09-13T07:01:43.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.209 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653442+0000", 2024-09-13T07:01:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653442+0000", 2024-09-13T07:01:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653440+0000", 2024-09-13T07:01:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653447+0000", 2024-09-13T07:01:43.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5359999999999998e-06 2024-09-13T07:01:43.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653463+0000", 2024-09-13T07:01:43.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5956000000000001e-05 2024-09-13T07:01:43.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653476+0000", 2024-09-13T07:01:43.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2833999999999999e-05 2024-09-13T07:01:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663259+0000", 2024-09-13T07:01:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097832319999999993 2024-09-13T07:01:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663300+0000", 2024-09-13T07:01:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1414e-05 2024-09-13T07:01:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663365+0000", 2024-09-13T07:01:43.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4361000000000003e-05 2024-09-13T07:01:43.213 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.213 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.213 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.213 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:3969 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:43.213 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653487+0000", 2024-09-13T07:01:43.213 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.524610852, 2024-09-13T07:01:43.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010045950999999999, 2024-09-13T07:01:43.213 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.213 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.214 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.214 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.214 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.214 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3969 2024-09-13T07:01:43.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.214 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653487+0000", 2024-09-13T07:01:43.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653487+0000", 2024-09-13T07:01:43.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653486+0000", 2024-09-13T07:01:43.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:43.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653492+0000", 2024-09-13T07:01:43.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2890000000000003e-06 2024-09-13T07:01:43.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653508+0000", 2024-09-13T07:01:43.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.59e-05 2024-09-13T07:01:43.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653521+0000", 2024-09-13T07:01:43.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.272e-05 2024-09-13T07:01:43.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663393+0000", 2024-09-13T07:01:43.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098722719999999996 2024-09-13T07:01:43.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663435+0000", 2024-09-13T07:01:43.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1566999999999997e-05 2024-09-13T07:01:43.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663533+0000", 2024-09-13T07:01:43.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8239000000000006e-05 2024-09-13T07:01:43.217 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.218 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.218 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4016 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:43.218 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653532+0000", 2024-09-13T07:01:43.218 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.524565823, 2024-09-13T07:01:43.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010191693, 2024-09-13T07:01:43.218 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.218 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.218 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.218 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.218 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.219 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4016 2024-09-13T07:01:43.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.219 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653532+0000", 2024-09-13T07:01:43.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653532+0000", 2024-09-13T07:01:43.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653531+0000", 2024-09-13T07:01:43.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:43.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653537+0000", 2024-09-13T07:01:43.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2559999999999999e-06 2024-09-13T07:01:43.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653553+0000", 2024-09-13T07:01:43.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5327000000000001e-05 2024-09-13T07:01:43.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653565+0000", 2024-09-13T07:01:43.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2563e-05 2024-09-13T07:01:43.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663578+0000", 2024-09-13T07:01:43.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010012951000000001 2024-09-13T07:01:43.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663634+0000", 2024-09-13T07:01:43.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6276999999999997e-05 2024-09-13T07:01:43.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663724+0000", 2024-09-13T07:01:43.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9371999999999997e-05 2024-09-13T07:01:43.222 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.222 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.222 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.223 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4017 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:43.223 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653577+0000", 2024-09-13T07:01:43.223 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.524520977, 2024-09-13T07:01:43.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010309476999999999, 2024-09-13T07:01:43.223 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.223 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.223 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4017 2024-09-13T07:01:43.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.223 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653577+0000", 2024-09-13T07:01:43.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653577+0000", 2024-09-13T07:01:43.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653576+0000", 2024-09-13T07:01:43.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:43.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653583+0000", 2024-09-13T07:01:43.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0620000000000003e-06 2024-09-13T07:01:43.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653598+0000", 2024-09-13T07:01:43.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.554e-05 2024-09-13T07:01:43.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653609+0000", 2024-09-13T07:01:43.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.078e-05 2024-09-13T07:01:43.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663756+0000", 2024-09-13T07:01:43.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010146887 2024-09-13T07:01:43.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663803+0000", 2024-09-13T07:01:43.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7497999999999998e-05 2024-09-13T07:01:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663886+0000", 2024-09-13T07:01:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3106e-05 2024-09-13T07:01:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4018 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653650+0000", 2024-09-13T07:01:43.227 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.524447699, 2024-09-13T07:01:43.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010399482, 2024-09-13T07:01:43.228 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.228 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.228 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.228 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.228 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.228 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4018 2024-09-13T07:01:43.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.228 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653650+0000", 2024-09-13T07:01:43.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653650+0000", 2024-09-13T07:01:43.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653649+0000", 2024-09-13T07:01:43.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:43.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653656+0000", 2024-09-13T07:01:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9739999999999996e-06 2024-09-13T07:01:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653690+0000", 2024-09-13T07:01:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4189000000000001e-05 2024-09-13T07:01:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653720+0000", 2024-09-13T07:01:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9621999999999999e-05 2024-09-13T07:01:43.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663938+0000", 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010218287 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.663983+0000", 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4807e-05 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664050+0000", 2024-09-13T07:01:43.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7082999999999995e-05 2024-09-13T07:01:43.232 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.232 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.232 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.232 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4019 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:43.232 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653733+0000", 2024-09-13T07:01:43.232 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.524365114, 2024-09-13T07:01:43.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010478706000000001, 2024-09-13T07:01:43.232 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.232 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4019 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653733+0000", 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653733+0000", 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653732+0000", 2024-09-13T07:01:43.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:43.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653739+0000", 2024-09-13T07:01:43.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2699999999999999e-06 2024-09-13T07:01:43.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653770+0000", 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1625999999999998e-05 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653786+0000", 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5383000000000002e-05 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664087+0000", 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010300741 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664130+0000", 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3470999999999998e-05 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664211+0000", 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1491999999999998e-05 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4020 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:43.236 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653798+0000", 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.524300145, 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010592649000000001, 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4020 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653798+0000", 2024-09-13T07:01:43.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653798+0000", 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653797+0000", 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653803+0000", 2024-09-13T07:01:43.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5429999999999999e-06 2024-09-13T07:01:43.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653834+0000", 2024-09-13T07:01:43.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0825999999999999e-05 2024-09-13T07:01:43.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653863+0000", 2024-09-13T07:01:43.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8924999999999998e-05 2024-09-13T07:01:43.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664250+0000", 2024-09-13T07:01:43.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010387062000000001 2024-09-13T07:01:43.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664329+0000", 2024-09-13T07:01:43.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9209999999999995e-05 2024-09-13T07:01:43.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664390+0000", 2024-09-13T07:01:43.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1166000000000004e-05 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4021 3.c 3:31f2f19c:::rbd_data.11945d91e99c.0000000000000008:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.653887+0000", 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.524210913999999, 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010633083999999999, 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.242 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4021 2024-09-13T07:01:43.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.242 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653887+0000", 2024-09-13T07:01:43.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653887+0000", 2024-09-13T07:01:43.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653886+0000", 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653899+0000", 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2425999999999999e-05 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653908+0000", 2024-09-13T07:01:43.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9089999999999995e-06 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.653916+0000", 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7800000000000001e-06 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664416+0000", 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010499523 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664459+0000", 2024-09-13T07:01:43.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2787999999999998e-05 2024-09-13T07:01:43.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.664520+0000", 2024-09-13T07:01:43.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1483999999999996e-05 2024-09-13T07:01:43.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.245 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.245 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7628 3.1f 3:fdf2e50b:::rbd_data.1194c5936dae.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1032192~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:01:43.245 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.4156600620000006, 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.630407088, 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7628 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:01:43.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762435+0000", 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762543+0000", 2024-09-13T07:01:43.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010850500000000001 2024-09-13T07:01:43.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762597+0000", 2024-09-13T07:01:43.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4262000000000003e-05 2024-09-13T07:01:43.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762626+0000", 2024-09-13T07:01:43.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9176e-05 2024-09-13T07:01:43.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762761+0000", 2024-09-13T07:01:43.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013405599999999999 2024-09-13T07:01:43.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:01:43.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762776+0000", 2024-09-13T07:01:43.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5679999999999999e-05 2024-09-13T07:01:43.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388492+0000", 2024-09-13T07:01:43.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.62571606300000004 2024-09-13T07:01:43.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for missing object", 2024-09-13T07:01:43.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388614+0000", 2024-09-13T07:01:43.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012126 2024-09-13T07:01:43.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390837+0000", 2024-09-13T07:01:43.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002223806 2024-09-13T07:01:43.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390963+0000", 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012550300000000001 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391127+0000", 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163995 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391989+0000", 2024-09-13T07:01:43.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086230499999999995 2024-09-13T07:01:43.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392662+0000", 2024-09-13T07:01:43.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067260799999999999 2024-09-13T07:01:43.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392728+0000", 2024-09-13T07:01:43.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6501000000000006e-05 2024-09-13T07:01:43.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392845+0000", 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116665 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11446 3.1d 3:ba689b8d:::rbd_data.1194b6604f63.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.895999+0000", 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2820993529999996, 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043645152999999999, 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.253 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11446 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.895999+0000", 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.895999+0000", 2024-09-13T07:01:43.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.895997+0000", 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.896009+0000", 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2140999999999999e-05 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.896038+0000", 2024-09-13T07:01:43.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9362000000000001e-05 2024-09-13T07:01:43.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.896051+0000", 2024-09-13T07:01:43.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948999999999999e-05 2024-09-13T07:01:43.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.896161+0000", 2024-09-13T07:01:43.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010956800000000001 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.896339+0000", 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000178495 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.905676+0000", 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009336881 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.939427+0000", 2024-09-13T07:01:43.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033751100999999999 2024-09-13T07:01:43.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.939456+0000", 2024-09-13T07:01:43.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8904999999999999e-05 2024-09-13T07:01:43.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.939559+0000", 2024-09-13T07:01:43.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102532 2024-09-13T07:01:43.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.939644+0000", 2024-09-13T07:01:43.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5048000000000003e-05 2024-09-13T07:01:43.259 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.259 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.259 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.259 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:12251 3.1d 3:ba689b8d:::rbd_data.1194b6604f63.0000000000000000:e [sparse-read 4063232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:43.259 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:40.019006+0000", 2024-09-13T07:01:43.259 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.1590915640000001, 2024-09-13T07:01:43.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099642329999999994, 2024-09-13T07:01:43.259 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.260 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.260 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12251 2024-09-13T07:01:43.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.260 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019006+0000", 2024-09-13T07:01:43.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019006+0000", 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019006+0000", 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019009+0000", 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6960000000000001e-06 2024-09-13T07:01:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019012+0000", 2024-09-13T07:01:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7189999999999999e-06 2024-09-13T07:01:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019015+0000", 2024-09-13T07:01:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5689999999999999e-06 2024-09-13T07:01:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.028775+0000", 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097603859999999994 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.028876+0000", 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010080499999999999 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.028971+0000", 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4456999999999994e-05 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:12252 3.1d 3:ba689b8d:::rbd_data.1194b6604f63.0000000000000000:e [sparse-read 4325376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:40.019017+0000", 2024-09-13T07:01:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.1590809499999999, 2024-09-13T07:01:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010100377000000001, 2024-09-13T07:01:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12252 2024-09-13T07:01:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019017+0000", 2024-09-13T07:01:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019017+0000", 2024-09-13T07:01:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019017+0000", 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019022+0000", 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.045e-06 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019033+0000", 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0912e-05 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019035+0000", 2024-09-13T07:01:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5629999999999999e-06 2024-09-13T07:01:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029008+0000", 2024-09-13T07:01:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099723759999999998 2024-09-13T07:01:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029053+0000", 2024-09-13T07:01:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5389000000000002e-05 2024-09-13T07:01:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029117+0000", 2024-09-13T07:01:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4307999999999995e-05 2024-09-13T07:01:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:12253 3.1d 3:ba689b8d:::rbd_data.1194b6604f63.0000000000000000:e [sparse-read 5111808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:40.019038+0000", 2024-09-13T07:01:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.1590603960000001, 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010321877, 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12253 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019038+0000", 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019038+0000", 2024-09-13T07:01:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019037+0000", 2024-09-13T07:01:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:01:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019039+0000", 2024-09-13T07:01:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.322e-06 2024-09-13T07:01:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019041+0000", 2024-09-13T07:01:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7650000000000002e-06 2024-09-13T07:01:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019044+0000", 2024-09-13T07:01:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5289999999999998e-06 2024-09-13T07:01:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029158+0000", 2024-09-13T07:01:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010113977 2024-09-13T07:01:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029260+0000", 2024-09-13T07:01:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010259000000000001 2024-09-13T07:01:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029359+0000", 2024-09-13T07:01:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8920999999999997e-05 2024-09-13T07:01:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:12254 3.1d 3:ba689b8d:::rbd_data.1194b6604f63.0000000000000000:e [sparse-read 5898240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:40.019046+0000", 2024-09-13T07:01:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.1590519939999999, 2024-09-13T07:01:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010468412999999999, 2024-09-13T07:01:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12254 2024-09-13T07:01:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019046+0000", 2024-09-13T07:01:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019046+0000", 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019046+0000", 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019047+0000", 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1170000000000001e-06 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019050+0000", 2024-09-13T07:01:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.711e-06 2024-09-13T07:01:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019054+0000", 2024-09-13T07:01:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5499999999999996e-06 2024-09-13T07:01:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029398+0000", 2024-09-13T07:01:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010344269 2024-09-13T07:01:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029445+0000", 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6492999999999997e-05 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029514+0000", 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9486999999999994e-05 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:12255 3.1d 3:ba689b8d:::rbd_data.1194b6604f63.0000000000000000:e [sparse-read 6946816~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:40.019063+0000", 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.1590346789999999, 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010714569, 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12255 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019063+0000", 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019063+0000", 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019063+0000", 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019064+0000", 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.342e-06 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019067+0000", 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.145e-06 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019074+0000", 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6429999999999998e-06 2024-09-13T07:01:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029563+0000", 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010489098000000001 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029666+0000", 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010289999999999999 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029778+0000", 2024-09-13T07:01:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011165700000000001 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:12256 3.1d 3:ba689b8d:::rbd_data.1194b6604f63.0000000000000000:e [sparse-read 7733248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:40.019083+0000", 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.159014542, 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010826661, 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12256 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019083+0000", 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019083+0000", 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019083+0000", 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019085+0000", 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4580000000000001e-06 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019088+0000", 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7680000000000002e-06 2024-09-13T07:01:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.019091+0000", 2024-09-13T07:01:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.571e-06 2024-09-13T07:01:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029800+0000", 2024-09-13T07:01:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010709262000000001 2024-09-13T07:01:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029848+0000", 2024-09-13T07:01:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8219e-05 2024-09-13T07:01:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.029910+0000", 2024-09-13T07:01:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1608999999999999e-05 2024-09-13T07:01:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.286 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:43.288 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:43.288 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:43.288 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:43.289 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:43.289 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:43.289 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:43.289 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:43.289 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:43.289 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:01:43.290 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:43.285+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:43.381 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone11 from 0x2f91311 (overlap 0x94140) to 0x67a79 2024-09-13T07:01:43.389 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:43.385+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:43.411 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:43.411 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:43.411 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:43.411 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:43.421 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:01:43.489 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:43.486+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:43.527 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:43.527 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:43.527 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:43.527 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:43.527 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:43.537 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:01:43.591 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:43.589+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:43.655 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:43.655 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:43.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:43.655 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:43.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.656 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:581 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:43.656 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.416139+0000", 2024-09-13T07:01:43.656 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.235524371, 2024-09-13T07:01:43.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061636429999999999, 2024-09-13T07:01:43.656 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.656 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.656 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 581 2024-09-13T07:01:43.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.656 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416139+0000", 2024-09-13T07:01:43.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416139+0000", 2024-09-13T07:01:43.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416136+0000", 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416171+0000", 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4545000000000002e-05 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416203+0000", 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2085000000000002e-05 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416224+0000", 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1157000000000001e-05 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416339+0000", 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115455 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.416349+0000", 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5729999999999997e-06 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.419615+0000", 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032657139999999999 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.419766+0000", 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015095000000000001 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.420094+0000", 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032800599999999999 2024-09-13T07:01:43.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.421267+0000", 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011733889999999999 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.422161+0000", 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089368499999999997 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.422233+0000", 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2619000000000004e-05 2024-09-13T07:01:43.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.422303+0000", 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9264999999999995e-05 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:949 3.a 3:52cbfadd:::rbd_data.1194d22ad098.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.495440+0000", 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.156222991, 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005241791, 2024-09-13T07:01:43.662 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 949 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495440+0000", 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495440+0000", 2024-09-13T07:01:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495439+0000", 2024-09-13T07:01:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495446+0000", 2024-09-13T07:01:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2269999999999996e-06 2024-09-13T07:01:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495460+0000", 2024-09-13T07:01:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3247999999999999e-05 2024-09-13T07:01:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495467+0000", 2024-09-13T07:01:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5129999999999999e-06 2024-09-13T07:01:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495521+0000", 2024-09-13T07:01:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4020000000000001e-05 2024-09-13T07:01:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495618+0000", 2024-09-13T07:01:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6934000000000004e-05 2024-09-13T07:01:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.495860+0000", 2024-09-13T07:01:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024149800000000001 2024-09-13T07:01:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500444+0000", 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045838909999999997 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500595+0000", 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015186099999999999 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500648+0000", 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2877999999999998e-05 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.500682+0000", 2024-09-13T07:01:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3818e-05 2024-09-13T07:01:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.592742617999999, 2024-09-13T07:01:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:01:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:01:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:01:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:01:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:01:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:01:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1534 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14365141] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.585915066999998, 2024-09-13T07:01:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040478761000000002, 2024-09-13T07:01:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1534 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065747+0000", 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075258+0000", 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095108710000000006 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075288+0000", 2024-09-13T07:01:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0085000000000001e-05 2024-09-13T07:01:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075305+0000", 2024-09-13T07:01:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7748e-05 2024-09-13T07:01:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075367+0000", 2024-09-13T07:01:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1867999999999998e-05 2024-09-13T07:01:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075520+0000", 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015298800000000001 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.076247+0000", 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072720499999999997 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105157+0000", 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028909642999999999 2024-09-13T07:01:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106124+0000", 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096729400000000005 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106173+0000", 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8196999999999999e-05 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106227+0000", 2024-09-13T07:01:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4388999999999997e-05 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1557 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.511149397000001, 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012339057, 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1557 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140502+0000", 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999881 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140526+0000", 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4049999999999998e-05 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140563+0000", 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6569000000000002e-05 2024-09-13T07:01:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140580+0000", 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7725000000000001e-05 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140942+0000", 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036136300000000002 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.141130+0000", 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018814999999999999 2024-09-13T07:01:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.141433+0000", 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030329899999999998 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.147925+0000", 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064915240000000003 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152757+0000", 2024-09-13T07:01:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048328709999999999 2024-09-13T07:01:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152802+0000", 2024-09-13T07:01:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4314000000000003e-05 2024-09-13T07:01:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152853+0000", 2024-09-13T07:01:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1149000000000001e-05 2024-09-13T07:01:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1558 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.511051659, 2024-09-13T07:01:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012353627000000001, 2024-09-13T07:01:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1558 2024-09-13T07:01:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140610+0000", 2024-09-13T07:01:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140619+0000", 2024-09-13T07:01:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8650000000000007e-06 2024-09-13T07:01:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140634+0000", 2024-09-13T07:01:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4515000000000001e-05 2024-09-13T07:01:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140646+0000", 2024-09-13T07:01:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1736000000000001e-05 2024-09-13T07:01:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.144258+0000", 2024-09-13T07:01:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036123499999999998 2024-09-13T07:01:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.145034+0000", 2024-09-13T07:01:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00077638900000000005 2024-09-13T07:01:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.145395+0000", 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036117499999999998 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.149775+0000", 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043795639999999999 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152906+0000", 2024-09-13T07:01:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031306929999999999 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152941+0000", 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4919000000000002e-05 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152965+0000", 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4641e-05 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1648 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,zero 10485760~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.297923+0000", 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.353740176999999, 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053690209999999999, 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1648 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297923+0000", 2024-09-13T07:01:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297923+0000", 2024-09-13T07:01:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297920+0000", 2024-09-13T07:01:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297941+0000", 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1628999999999999e-05 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.297986+0000", 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4135999999999999e-05 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298009+0000", 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3206000000000001e-05 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298236+0000", 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000227471 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298526+0000", 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028929800000000001 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.303292+0000", 2024-09-13T07:01:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047665570000000003 2024-09-13T07:01:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1649 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 10485760~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.298045+0000", 2024-09-13T07:01:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.353618739000002, 2024-09-13T07:01:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055560130000000003, 2024-09-13T07:01:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1649 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298045+0000", 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298045+0000", 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298043+0000", 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298052+0000", 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4829999999999999e-06 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298069+0000", 2024-09-13T07:01:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7073000000000001e-05 2024-09-13T07:01:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.298083+0000", 2024-09-13T07:01:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3977e-05 2024-09-13T07:01:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.300163+0000", 2024-09-13T07:01:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020800229999999999 2024-09-13T07:01:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.300370+0000", 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000207679 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.303601+0000", 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032301550000000002 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1661 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [read 551881~5223] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.317063+0000", 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.334599823000001, 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066193119999999996, 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1661 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317063+0000", 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317063+0000", 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317062+0000", 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317070+0000", 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9370000000000004e-06 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317086+0000", 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5685e-05 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317093+0000", 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6190000000000002e-06 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317144+0000", 2024-09-13T07:01:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0587999999999997e-05 2024-09-13T07:01:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.317148+0000", 2024-09-13T07:01:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0949999999999998e-06 2024-09-13T07:01:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.323168+0000", 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060199110000000002 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.323288+0000", 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120375 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.323683+0000", 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00039459599999999999 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1676 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.310773455, 2024-09-13T07:01:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067554429999999999, 2024-09-13T07:01:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1676 2024-09-13T07:01:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340889+0000", 2024-09-13T07:01:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340896+0000", 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1420000000000004e-06 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340909+0000", 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3329000000000001e-05 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340920+0000", 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1029999999999999e-05 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341653+0000", 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073243400000000001 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341825+0000", 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017243099999999999 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.347645+0000", 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058200639999999998 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1761 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.454709+0000", 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.196953958999998, 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049941610000000004, 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1761 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454709+0000", 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454709+0000", 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454708+0000", 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454714+0000", 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1970000000000001e-06 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454728+0000", 2024-09-13T07:01:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3212e-05 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.454738+0000", 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0475000000000001e-05 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.455883+0000", 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001144438 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.456054+0000", 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000171402 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.459703+0000", 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036494819999999999 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1808 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.542510+0000", 2024-09-13T07:01:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.109153252999999, 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050197829999999999, 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1808 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542510+0000", 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542510+0000", 2024-09-13T07:01:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542509+0000", 2024-09-13T07:01:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542516+0000", 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2200000000000003e-06 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542533+0000", 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6957e-05 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.542549+0000", 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5959000000000001e-05 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.545392+0000", 2024-09-13T07:01:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028432330000000001 2024-09-13T07:01:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.545520+0000", 2024-09-13T07:01:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012757400000000001 2024-09-13T07:01:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.545722+0000", 2024-09-13T07:01:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020242400000000001 2024-09-13T07:01:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547178+0000", 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014557509999999999 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547448+0000", 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026981900000000003 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547496+0000", 2024-09-13T07:01:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8680000000000001e-05 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.547530+0000", 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3466000000000001e-05 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1854 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,zero 4194304~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.053653736000001, 2024-09-13T07:01:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070961979999999997, 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1854 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598007+0000", 2024-09-13T07:01:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598022+0000", 2024-09-13T07:01:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-13T07:01:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598062+0000", 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9552999999999999e-05 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598081+0000", 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9247999999999999e-05 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598256+0000", 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000174482 2024-09-13T07:01:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598365+0000", 2024-09-13T07:01:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109108 2024-09-13T07:01:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.605106+0000", 2024-09-13T07:01:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067409189999999997 2024-09-13T07:01:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1855 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.053545455999998, 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074110900000000004, 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1855 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598117+0000", 2024-09-13T07:01:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598125+0000", 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3110000000000006e-06 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598147+0000", 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2484999999999999e-05 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598159+0000", 2024-09-13T07:01:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1939999999999999e-05 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.599204+0000", 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010449680000000001 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.599409+0000", 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020456700000000001 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.605529+0000", 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061199790000000002 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:3373 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.464963+0000", 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.186699809, 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050049359999999998, 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3373 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.464963+0000", 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.464963+0000", 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.464961+0000", 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.464974+0000", 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2679e-05 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.464999+0000", 2024-09-13T07:01:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5488000000000001e-05 2024-09-13T07:01:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.465010+0000", 2024-09-13T07:01:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0526999999999999e-05 2024-09-13T07:01:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.465088+0000", 2024-09-13T07:01:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7855000000000006e-05 2024-09-13T07:01:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.465094+0000", 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.348e-06 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.467866+0000", 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002772114 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.467956+0000", 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9203999999999995e-05 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.468141+0000", 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018521600000000001 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.469125+0000", 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00098440099999999998 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.469884+0000", 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075881199999999998 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.469931+0000", 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6601000000000003e-05 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.469968+0000", 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7889000000000003e-05 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7595 3.a 3:54f0339c:::rbd_data.1194c5936dae.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 564664~8776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:01:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.687839+0000", 2024-09-13T07:01:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.9638241860000001, 2024-09-13T07:01:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013545, 2024-09-13T07:01:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7595 2024-09-13T07:01:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.687839+0000", 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.687839+0000", 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.687836+0000", 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690295+0000", 2024-09-13T07:01:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024590800000000002 2024-09-13T07:01:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690412+0000", 2024-09-13T07:01:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116197 2024-09-13T07:01:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690445+0000", 2024-09-13T07:01:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2947999999999997e-05 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690548+0000", 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103193 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690733+0000", 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018520799999999999 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691055+0000", 2024-09-13T07:01:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032229699999999998 2024-09-13T07:01:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.695214+0000", 2024-09-13T07:01:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041589050000000001 2024-09-13T07:01:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701310+0000", 2024-09-13T07:01:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060959370000000001 2024-09-13T07:01:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701352+0000", 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701384+0000", 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1612999999999997e-05 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7596 3.10 3:085a9411:::rbd_data.1194c5936dae.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 557056~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.690504+0000", 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.9611594140000008, 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011135499, 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7596 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690504+0000", 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690504+0000", 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690503+0000", 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691953+0000", 2024-09-13T07:01:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014497920000000001 2024-09-13T07:01:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691978+0000", 2024-09-13T07:01:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5384999999999998e-05 2024-09-13T07:01:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691991+0000", 2024-09-13T07:01:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.261e-05 2024-09-13T07:01:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.692271+0000", 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028019399999999999 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.692541+0000", 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026980400000000001 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.693006+0000", 2024-09-13T07:01:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046576499999999998 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701109+0000", 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081030630000000006 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701549+0000", 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043989499999999999 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701606+0000", 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6292999999999998e-05 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701639+0000", 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3741000000000001e-05 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11218 3.10 3:0ba898fc:::rbd_data.1194a7cedc35.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.629190+0000", 2024-09-13T07:01:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.022472917, 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070817470000000002, 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11218 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.629190+0000", 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.629190+0000", 2024-09-13T07:01:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.629188+0000", 2024-09-13T07:01:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631056+0000", 2024-09-13T07:01:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018678989999999999 2024-09-13T07:01:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631082+0000", 2024-09-13T07:01:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6156000000000002e-05 2024-09-13T07:01:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631093+0000", 2024-09-13T07:01:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0696999999999999e-05 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631216+0000", 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012328299999999999 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631435+0000", 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021897299999999999 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631884+0000", 2024-09-13T07:01:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044842900000000003 2024-09-13T07:01:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636066+0000", 2024-09-13T07:01:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041825059999999999 2024-09-13T07:01:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636133+0000", 2024-09-13T07:01:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7000000000000002e-05 2024-09-13T07:01:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636213+0000", 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9671000000000003e-05 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636272+0000", 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9126000000000003e-05 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11219 3.10 3:0ba898fc:::rbd_data.1194a7cedc35.0000000000000009:head [truncate 2621440] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.631119+0000", 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.0205445219999998, 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054998979999999996, 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11219 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631119+0000", 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631119+0000", 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631118+0000", 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631123+0000", 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7849999999999999e-06 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631191+0000", 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8246999999999999e-05 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631251+0000", 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0649000000000002e-05 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.634626+0000", 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033748960000000001 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.634700+0000", 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3393999999999996e-05 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.634878+0000", 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017833399999999999 2024-09-13T07:01:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636330+0000", 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014522350000000001 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636524+0000", 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000194124 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636567+0000", 2024-09-13T07:01:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2178000000000002e-05 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636619+0000", 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.206e-05 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11408 3.10 3:0f48178c:::rbd_data.1194b6604f63.000000000000000c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.833470+0000", 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8181936109999999, 2024-09-13T07:01:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054053920000000002, 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11408 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.833470+0000", 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.833470+0000", 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.833468+0000", 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.833482+0000", 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3957e-05 2024-09-13T07:01:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.833513+0000", 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1605e-05 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.833534+0000", 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0528000000000001e-05 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.836351+0000", 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028175710000000001 2024-09-13T07:01:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.836560+0000", 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000208934 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.838875+0000", 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002314927 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:43.763 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:43.765 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:01:43.765 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:43.762+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:43.866 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:43.862+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:43.924 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:43.924 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:43.924 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:43.924 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:43.934 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:01:43.967 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:43.965+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:44.040 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:44.041 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:44.050 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:01:44.068 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:44.065+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:44.163 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:44.163 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:44.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:44.164 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:44.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.164 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:44.164 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:44.164 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.646204980999997, 2024-09-13T07:01:44.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018207798000000001, 2024-09-13T07:01:44.164 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.164 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:44.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:44.164 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-13T07:01:44.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.165 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:44.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:44.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513664+0000", 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513674+0000", 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8770000000000005e-06 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513695+0000", 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1319e-05 2024-09-13T07:01:44.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513714+0000", 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.914e-05 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526929+0000", 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013215146000000001 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.527080+0000", 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000150824 2024-09-13T07:01:44.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.527312+0000", 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023218100000000001 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530679+0000", 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003366529 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531659+0000", 2024-09-13T07:01:44.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0009803940000000001 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531784+0000", 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012519399999999999 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531873+0000", 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8307999999999995e-05 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.170 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:44.170 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:44.170 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.646142712, 2024-09-13T07:01:44.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018374253, 2024-09-13T07:01:44.170 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.170 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:44.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:44.170 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-13T07:01:44.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.170 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513726+0000", 2024-09-13T07:01:44.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513933+0000", 2024-09-13T07:01:44.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020722500000000001 2024-09-13T07:01:44.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513984+0000", 2024-09-13T07:01:44.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0719e-05 2024-09-13T07:01:44.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513999+0000", 2024-09-13T07:01:44.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4877e-05 2024-09-13T07:01:44.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528581+0000", 2024-09-13T07:01:44.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014581604999999999 2024-09-13T07:01:44.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528674+0000", 2024-09-13T07:01:44.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3021000000000003e-05 2024-09-13T07:01:44.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:44.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528803+0000", 2024-09-13T07:01:44.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012964199999999999 2024-09-13T07:01:44.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:44.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531891+0000", 2024-09-13T07:01:44.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00308764 2024-09-13T07:01:44.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531990+0000", 2024-09-13T07:01:44.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9407999999999994e-05 2024-09-13T07:01:44.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:44.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532064+0000", 2024-09-13T07:01:44.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3793000000000003e-05 2024-09-13T07:01:44.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532101+0000", 2024-09-13T07:01:44.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7357000000000002e-05 2024-09-13T07:01:44.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.176 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.176 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:44.176 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:44.176 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.645853453000001, 2024-09-13T07:01:44.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018326366, 2024-09-13T07:01:44.176 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.176 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.177 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.177 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:44.177 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:44.177 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-13T07:01:44.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.177 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:44.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:44.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514015+0000", 2024-09-13T07:01:44.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:44.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514026+0000", 2024-09-13T07:01:44.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1053000000000001e-05 2024-09-13T07:01:44.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514047+0000", 2024-09-13T07:01:44.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0687e-05 2024-09-13T07:01:44.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514065+0000", 2024-09-13T07:01:44.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8799000000000001e-05 2024-09-13T07:01:44.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529495+0000", 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015429877 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529642+0000", 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014682399999999999 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529871+0000", 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022936399999999999 2024-09-13T07:01:44.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532183+0000", 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023112889999999998 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532199+0000", 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6082e-05 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532269+0000", 2024-09-13T07:01:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9812000000000002e-05 2024-09-13T07:01:44.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532343+0000", 2024-09-13T07:01:44.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4141999999999996e-05 2024-09-13T07:01:44.182 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.182 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.182 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.182 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:44.182 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.645791217000003, 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018420999, 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:44.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:44.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514078+0000", 2024-09-13T07:01:44.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514088+0000", 2024-09-13T07:01:44.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0433e-05 2024-09-13T07:01:44.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514104+0000", 2024-09-13T07:01:44.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5616000000000001e-05 2024-09-13T07:01:44.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514133+0000", 2024-09-13T07:01:44.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9453000000000001e-05 2024-09-13T07:01:44.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530751+0000", 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016617996999999999 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530926+0000", 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017493599999999999 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531174+0000", 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024812100000000002 2024-09-13T07:01:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:44.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532358+0000", 2024-09-13T07:01:44.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00118396 2024-09-13T07:01:44.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532421+0000", 2024-09-13T07:01:44.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2749999999999994e-05 2024-09-13T07:01:44.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:44.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532467+0000", 2024-09-13T07:01:44.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6427000000000001e-05 2024-09-13T07:01:44.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532500+0000", 2024-09-13T07:01:44.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2329000000000002e-05 2024-09-13T07:01:44.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.188 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.188 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1174 3.13 3:cb2ed8dc:::rbd_data.11943a018f34.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 6419344~27893] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:44.189 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:01:44.189 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.276073241999999, 2024-09-13T07:01:44.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041813481999999999, 2024-09-13T07:01:44.189 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.189 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.189 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1174 2024-09-13T07:01:44.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.189 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:01:44.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:01:44.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883795+0000", 2024-09-13T07:01:44.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:44.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894234+0000", 2024-09-13T07:01:44.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010439133 2024-09-13T07:01:44.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894276+0000", 2024-09-13T07:01:44.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429000000000003e-05 2024-09-13T07:01:44.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894296+0000", 2024-09-13T07:01:44.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9279999999999998e-05 2024-09-13T07:01:44.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894477+0000", 2024-09-13T07:01:44.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018160600000000001 2024-09-13T07:01:44.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894889+0000", 2024-09-13T07:01:44.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041202399999999998 2024-09-13T07:01:44.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:44.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.898212+0000", 2024-09-13T07:01:44.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033228400000000001 2024-09-13T07:01:44.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:44.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.922541+0000", 2024-09-13T07:01:44.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024329052 2024-09-13T07:01:44.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925458+0000", 2024-09-13T07:01:44.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002916891 2024-09-13T07:01:44.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:44.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925531+0000", 2024-09-13T07:01:44.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2781999999999995e-05 2024-09-13T07:01:44.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925610+0000", 2024-09-13T07:01:44.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9355000000000002e-05 2024-09-13T07:01:44.194 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.194 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.195 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.195 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:01:44.195 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:01:44.195 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.105697441, 2024-09-13T07:01:44.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:01:44.195 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.195 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.195 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:01:44.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:01:44.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1535 3.12 e27/22 3:4d1f9422:::rbd_object_map.11943a018f34:head v 27'27, mlcod=27'25)", 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:01:44.199 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.091740609999999, 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027873574000000002, 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068128+0000", 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068140+0000", 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2279000000000001e-05 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068175+0000", 2024-09-13T07:01:44.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4872000000000001e-05 2024-09-13T07:01:44.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068193+0000", 2024-09-13T07:01:44.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7540000000000001e-05 2024-09-13T07:01:44.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068302+0000", 2024-09-13T07:01:44.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010928 2024-09-13T07:01:44.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068351+0000", 2024-09-13T07:01:44.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036000000000003e-05 2024-09-13T07:01:44.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:44.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.095950+0000", 2024-09-13T07:01:44.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027598988000000001 2024-09-13T07:01:44.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.096003+0000", 2024-09-13T07:01:44.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2722000000000001e-05 2024-09-13T07:01:44.204 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.204 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.204 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.204 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4029 3.d 3:b2910c3f:::rbd_data.11945d91e99c.000000000000000a:8 [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:44.204 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.656360+0000", 2024-09-13T07:01:44.205 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.503510151, 2024-09-13T07:01:44.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468358, 2024-09-13T07:01:44.205 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.205 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:44.205 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.205 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.205 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.205 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4029 2024-09-13T07:01:44.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.205 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656360+0000", 2024-09-13T07:01:44.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656360+0000", 2024-09-13T07:01:44.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656359+0000", 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656362+0000", 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8250000000000001e-06 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656369+0000", 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4280000000000001e-06 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656374+0000", 2024-09-13T07:01:44.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.254e-06 2024-09-13T07:01:44.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.671663+0000", 2024-09-13T07:01:44.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015289036000000001 2024-09-13T07:01:44.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.671742+0000", 2024-09-13T07:01:44.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9023999999999994e-05 2024-09-13T07:01:44.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.671828+0000", 2024-09-13T07:01:44.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6286999999999996e-05 2024-09-13T07:01:44.209 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.209 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.209 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.209 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4030 3.d 3:b2910c3f:::rbd_data.11945d91e99c.000000000000000a:8 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:44.209 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.656378+0000", 2024-09-13T07:01:44.209 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.503491540000001, 2024-09-13T07:01:44.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015571448, 2024-09-13T07:01:44.210 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.210 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:44.210 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.210 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.210 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.210 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4030 2024-09-13T07:01:44.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.210 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656378+0000", 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656378+0000", 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656378+0000", 2024-09-13T07:01:44.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656381+0000", 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6740000000000001e-06 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656391+0000", 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0049e-05 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656400+0000", 2024-09-13T07:01:44.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8430000000000005e-06 2024-09-13T07:01:44.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.671840+0000", 2024-09-13T07:01:44.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015440005 2024-09-13T07:01:44.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.671886+0000", 2024-09-13T07:01:44.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5765e-05 2024-09-13T07:01:44.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.671950+0000", 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3597999999999997e-05 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4031 3.d 3:b2910c3f:::rbd_data.11945d91e99c.000000000000000a:8 [sparse-read 11665408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.656409+0000", 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.503461167999999, 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015760274000000001, 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.214 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4031 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656409+0000", 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656409+0000", 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656408+0000", 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656414+0000", 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656426+0000", 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1694e-05 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656440+0000", 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4331999999999999e-05 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.671987+0000", 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015547204 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.672071+0000", 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3479999999999994e-05 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.672169+0000", 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8183000000000005e-05 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4032 3.d 3:b2910c3f:::rbd_data.11945d91e99c.000000000000000a:8 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.656452+0000", 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.503417833, 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015843207000000002, 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4032 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656452+0000", 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656452+0000", 2024-09-13T07:01:44.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656451+0000", 2024-09-13T07:01:44.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656457+0000", 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3589999999999999e-06 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656464+0000", 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5859999999999999e-06 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656470+0000", 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9970000000000004e-06 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.672182+0000", 2024-09-13T07:01:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015711850999999999 2024-09-13T07:01:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.672228+0000", 2024-09-13T07:01:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5936999999999997e-05 2024-09-13T07:01:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.672295+0000", 2024-09-13T07:01:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7304999999999993e-05 2024-09-13T07:01:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:4033 3.d 3:b2910c3f:::rbd_data.11945d91e99c.000000000000000a:8 [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:30.656477+0000", 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.503393323999999, 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016023463000000002, 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4033 2024-09-13T07:01:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656477+0000", 2024-09-13T07:01:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656477+0000", 2024-09-13T07:01:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656476+0000", 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656479+0000", 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8830000000000002e-06 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656485+0000", 2024-09-13T07:01:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4350000000000002e-06 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.656491+0000", 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2170000000000002e-06 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.672332+0000", 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015840989999999999 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.672411+0000", 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9839000000000006e-05 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:30.672500+0000", 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8586999999999997e-05 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11135 3.e 3:739b9dce:::rbd_data.1194897e7c0a.0000000000000004:c [sparse-read 3801088~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.606051+0000", 2024-09-13T07:01:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.5538189979999997, 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015374422, 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11135 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606051+0000", 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606051+0000", 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606049+0000", 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606057+0000", 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4340000000000003e-06 2024-09-13T07:01:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606079+0000", 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2314999999999999e-05 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606092+0000", 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3264e-05 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.620720+0000", 2024-09-13T07:01:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014627757999999999 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.620775+0000", 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5216000000000001e-05 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.621425+0000", 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00065000599999999996 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11157 3.e 3:739b9dce:::rbd_data.1194897e7c0a.0000000000000004:c [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.606105+0000", 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.5537646890000003, 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015692276000000002, 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11157 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606105+0000", 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606105+0000", 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606104+0000", 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606111+0000", 2024-09-13T07:01:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2779999999999999e-06 2024-09-13T07:01:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606139+0000", 2024-09-13T07:01:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8654e-05 2024-09-13T07:01:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606147+0000", 2024-09-13T07:01:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.621493+0000", 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015346034999999999 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.621658+0000", 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016448999999999999 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.621797+0000", 2024-09-13T07:01:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013966200000000001 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11158 3.e 3:739b9dce:::rbd_data.1194897e7c0a.0000000000000004:c [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.606156+0000", 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.5537140630000001, 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015915423000000001, 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11158 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606156+0000", 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606156+0000", 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606155+0000", 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606162+0000", 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9310000000000002e-06 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606185+0000", 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3005000000000001e-05 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606198+0000", 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3023999999999999e-05 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.621862+0000", 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015664240999999999 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.621997+0000", 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134418 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.622071+0000", 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4660999999999995e-05 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11179 3.e 3:739b9dce:::rbd_data.1194897e7c0a.0000000000000004:c [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.606210+0000", 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.5536599429999995, 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01661491, 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11179 2024-09-13T07:01:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606210+0000", 2024-09-13T07:01:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606210+0000", 2024-09-13T07:01:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606208+0000", 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606216+0000", 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3139999999999998e-06 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606237+0000", 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1742e-05 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606251+0000", 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3152999999999999e-05 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.622117+0000", 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015865898999999999 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.622199+0000", 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2262000000000006e-05 2024-09-13T07:01:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.622825+0000", 2024-09-13T07:01:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00062611399999999999 2024-09-13T07:01:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11180 3.e 3:739b9dce:::rbd_data.1194897e7c0a.0000000000000004:c [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.606262+0000", 2024-09-13T07:01:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.5536078270000004, 2024-09-13T07:01:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016844517, 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11180 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606262+0000", 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606262+0000", 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606261+0000", 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606267+0000", 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.127e-06 2024-09-13T07:01:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606284+0000", 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6623999999999999e-05 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606297+0000", 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3118e-05 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.622869+0000", 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016571931000000002 2024-09-13T07:01:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.622970+0000", 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100832 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.623107+0000", 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000136842 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11181 3.e 3:739b9dce:::rbd_data.1194897e7c0a.0000000000000004:c [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.606309+0000", 2024-09-13T07:01:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.553561191, 2024-09-13T07:01:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017040290999999999, 2024-09-13T07:01:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11181 2024-09-13T07:01:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606309+0000", 2024-09-13T07:01:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606309+0000", 2024-09-13T07:01:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606308+0000", 2024-09-13T07:01:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606314+0000", 2024-09-13T07:01:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9499999999999998e-06 2024-09-13T07:01:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606330+0000", 2024-09-13T07:01:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6291e-05 2024-09-13T07:01:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606345+0000", 2024-09-13T07:01:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4924e-05 2024-09-13T07:01:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.623135+0000", 2024-09-13T07:01:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016790623000000001 2024-09-13T07:01:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.623228+0000", 2024-09-13T07:01:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2801999999999997e-05 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.623349+0000", 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120784 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:12392 3.11 3:8b43190b:::rbd_data.11945fc173dd.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:40.270727+0000", 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.889143298, 2024-09-13T07:01:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023367789, 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12392 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270727+0000", 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270727+0000", 2024-09-13T07:01:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270724+0000", 2024-09-13T07:01:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270740+0000", 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6467999999999999e-05 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270778+0000", 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8345e-05 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270795+0000", 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6171000000000002e-05 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270852+0000", 2024-09-13T07:01:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7365999999999999e-05 2024-09-13T07:01:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270951+0000", 2024-09-13T07:01:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8908999999999998e-05 2024-09-13T07:01:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.274009+0000", 2024-09-13T07:01:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030581110000000001 2024-09-13T07:01:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.293838+0000", 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019829368 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.293936+0000", 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8066000000000006e-05 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.294022+0000", 2024-09-13T07:01:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5908000000000005e-05 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.294094+0000", 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2044000000000003e-05 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:12614 3.11 3:8b43190b:::rbd_data.11945fc173dd.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:40.730488+0000", 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4293823789999998, 2024-09-13T07:01:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023878531000000001, 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12614 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730488+0000", 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730488+0000", 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730486+0000", 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730498+0000", 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1960999999999999e-05 2024-09-13T07:01:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730524+0000", 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6301000000000001e-05 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730544+0000", 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9760999999999999e-05 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730630+0000", 2024-09-13T07:01:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6768999999999995e-05 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730723+0000", 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2071999999999995e-05 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.735004+0000", 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042816770000000002 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.754112+0000", 2024-09-13T07:01:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019107902999999999 2024-09-13T07:01:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.754210+0000", 2024-09-13T07:01:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8191999999999998e-05 2024-09-13T07:01:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.754295+0000", 2024-09-13T07:01:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4401999999999996e-05 2024-09-13T07:01:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.754366+0000", 2024-09-13T07:01:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1320000000000002e-05 2024-09-13T07:01:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:44.265 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:44.267 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:44.266+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:44.267 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:01:44.367 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:44.365+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:44.384 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:44.384 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:44.384 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:44.385 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:44.395 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:01:44.452 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:01:44.468 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:44.466+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:44.501 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:44.501 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:44.501 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:44.501 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:44.501 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:44.511 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:01:44.569 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:44.569+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:44.617 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:44.617 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:44.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:44.618 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:44.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.618 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:44.618 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:44.618 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.11093661, 2024-09-13T07:01:44.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:01:44.618 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.618 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.618 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:01:44.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.619 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:44.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:44.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:01:44.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:01:44.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:01:44.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:01:44.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:01:44.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:01:44.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:01:44.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:01:44.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:01:44.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:01:44.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:01:44.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:44.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:01:44.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:01:44.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:44.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:01:44.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:01:44.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:01:44.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:01:44.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.624 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.624 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:44.624 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:44.624 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.086108899999999, 2024-09-13T07:01:44.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:01:44.624 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.624 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.624 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.624 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.625 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:01:44.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.625 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:44.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:44.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:01:44.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:01:44.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:01:44.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:01:44.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:01:44.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:01:44.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:44.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:01:44.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:01:44.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:01:44.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:01:44.630 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.630 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.630 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.630 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:44.630 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:44.630 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.547437076000001, 2024-09-13T07:01:44.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:01:44.630 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.630 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:44.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:01:44.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:44.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:01:44.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:01:44.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:01:44.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:44.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:01:44.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:01:44.636 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.636 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.636 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.636 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:44.636 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:44.636 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.517811883, 2024-09-13T07:01:44.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:01:44.636 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.636 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:01:44.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:01:44.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:44.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:01:44.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:01:44.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:44.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.985098252, 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.642 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:44.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:44.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:01:44.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:01:44.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:01:44.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:01:44.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:01:44.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:01:44.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:01:44.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:01:44.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:01:44.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:01:44.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:44.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:01:44.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:01:44.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:01:44.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:01:44.648 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.648 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.648 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.648 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:44.648 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:44.648 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.985015795999999, 2024-09-13T07:01:44.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:01:44.648 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.649 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.649 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:01:44.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.649 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:44.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:44.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:01:44.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:01:44.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:44.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:01:44.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:01:44.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:01:44.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:01:44.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:44.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:01:44.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.984983986, 2024-09-13T07:01:44.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:01:44.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:01:44.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:01:44.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:01:44.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:01:44.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:01:44.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:01:44.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:01:44.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:01:44.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:44.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:01:44.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:01:44.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:44.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:01:44.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:01:44.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:01:44.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:01:44.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:44.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:01:44.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:01:44.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:01:44.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:01:44.660 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.660 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.660 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.984928828000001, 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.661 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:01:44.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:01:44.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:01:44.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:01:44.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:01:44.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:01:44.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:01:44.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:44.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:01:44.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:01:44.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:01:44.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:01:44.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:44.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:01:44.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.984883548999999, 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:01:44.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:01:44.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:01:44.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:01:44.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:01:44.672 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.672 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.672 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.672 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:44.672 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:44.672 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.984845015000001, 2024-09-13T07:01:44.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:44.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:01:44.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:01:44.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:01:44.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:01:44.678 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.678 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.678 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.678 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:44.678 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:44.678 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.984783396000001, 2024-09-13T07:01:44.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:01:44.679 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.679 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.679 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.679 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.679 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.679 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:01:44.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.679 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:44.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:44.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:01:44.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:44.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:01:44.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:01:44.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:01:44.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:01:44.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.684 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.970560713000001, 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:44.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:44.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:01:44.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:44.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:01:44.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:01:44.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:01:44.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:01:44.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:01:44.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:01:44.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:01:44.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:01:44.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:01:44.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:01:44.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:44.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:01:44.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:01:44.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:44.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:01:44.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:01:44.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:01:44.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:01:44.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.690 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.958682193, 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:01:44.694 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.958125131999999, 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:01:44.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.696 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:44.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:44.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:01:44.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:01:44.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:01:44.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:01:44.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:01:44.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.958102976999999, 2024-09-13T07:01:44.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:01:44.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:01:44.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:01:44.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:01:44.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:01:44.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.703 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.95808326, 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.704 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:44.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:44.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:44.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:44.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:01:44.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:01:44.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:01:44.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:01:44.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:01:44.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:01:44.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:01:44.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:01:44.708 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.708 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.708 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.708 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:44.708 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:44.708 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.95806593, 2024-09-13T07:01:44.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:01:44.709 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.709 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:44.709 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.709 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.709 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.709 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:01:44.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.709 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:44.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:44.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:01:44.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:01:44.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:01:44.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:01:44.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.714 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:44.714 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:44.714 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.957770268000001, 2024-09-13T07:01:44.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:01:44.714 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.714 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:44.714 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.714 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.714 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.714 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:01:44.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.714 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:44.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:44.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:44.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:01:44.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:01:44.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:01:44.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:01:44.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:01:44.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:01:44.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.396567449, 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.718 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:01:44.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:01:44.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:44.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:01:44.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:01:44.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:01:44.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:01:44.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:01:44.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:44.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:01:44.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:01:44.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:44.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:01:44.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:01:44.724 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.724 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.724 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.724 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:01:44.724 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:01:44.724 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.3644885460000005, 2024-09-13T07:01:44.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:01:44.724 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:44.724 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:44.724 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:44.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:44.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:01:44.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:01:44.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:44.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:01:44.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:01:44.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:44.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:44.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:44.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:01:44.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:01:44.728 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.728 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.728 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.728 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:44.728 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:44.728 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:44.730 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:01:44.731 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-13T07:01:44.731 INFO:teuthology.orchestra.run.smithi118.stdout:859 trunc from 0x2f91311 to 0x2f868d2 2024-09-13T07:01:44.731 INFO:teuthology.orchestra.run.smithi118.stdout:861 punch from 0x12236fa to 0x1223f5c, (0x862 bytes) 2024-09-13T07:01:44.731 INFO:teuthology.orchestra.run.smithi118.stdout:863 read 0x501edc thru 0x503ac0 (0x1be5 bytes) 2024-09-13T07:01:44.731 INFO:teuthology.orchestra.run.smithi118.stdout:865 write 0x5898a5c thru 0x58a5f21 (0xd4c6 bytes) 2024-09-13T07:01:44.731 INFO:teuthology.orchestra.run.smithi118.stdout:866 read 0x1f97a00 thru 0x1fa313d (0xb73e bytes) 2024-09-13T07:01:44.731 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:44.732 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:44.726+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:44.737 INFO:teuthology.orchestra.run.smithi118.stdout:869 writesame 0xb05aabf thru 0xb065059 data_size 0x5(0xa59b bytes) 2024-09-13T07:01:44.737 INFO:teuthology.orchestra.run.smithi118.stdout:871 read 0x66da935 thru 0x66dd414 (0x2ae0 bytes) 2024-09-13T07:01:44.737 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:44.737 INFO:teuthology.orchestra.run.smithi118.stdout:874 read 0x3b20045 thru 0x3b2d97f (0xd93b bytes) 2024-09-13T07:01:44.750 INFO:teuthology.orchestra.run.smithi118.stdout:876 trunc from 0xb06505a to 0x2b2a602 2024-09-13T07:01:44.794 INFO:teuthology.orchestra.run.smithi118.stdout:881 trunc from 0x2b2a602 to 0xbd07ef3 2024-09-13T07:01:44.805 INFO:teuthology.orchestra.run.smithi118.stdout:882 read 0x2a4f9a2 thru 0x2a5dc12 (0xe271 bytes) 2024-09-13T07:01:44.816 INFO:teuthology.orchestra.run.smithi118.stdout:884 trunc from 0xbd07ef3 to 0x5cb7898 2024-09-13T07:01:44.830 INFO:teuthology.orchestra.run.smithi118.stdout:885 punch from 0x5649ae6 to 0x5654280, (0xa79a bytes) 2024-09-13T07:01:44.830 INFO:teuthology.orchestra.run.smithi118.stdout:886 read 0x2cc7600 thru 0x2ccd089 (0x5a8a bytes) 2024-09-13T07:01:44.831 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:44.826+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:44.831 INFO:teuthology.orchestra.run.smithi118.stdout:887 read 0x36c78e0 thru 0x36cb1ad (0x38ce bytes) 2024-09-13T07:01:44.831 INFO:teuthology.orchestra.run.smithi118.stdout:888 writesame 0x5a1e154 thru 0x5a23d57 data_size 0x4e(0x5c04 bytes) 2024-09-13T07:01:44.836 INFO:teuthology.orchestra.run.smithi118.stdout:890 write 0x5b4e67d thru 0x5b519ec (0x3370 bytes) 2024-09-13T07:01:44.849 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:44.849 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:44.849 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:44.849 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:44.853 INFO:teuthology.orchestra.run.smithi118.stdout:891 trunc from 0x5cb7898 to 0xe5f9137 2024-09-13T07:01:44.861 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:01:44.867 INFO:teuthology.orchestra.run.smithi118.stdout:893 writesame 0x9c3280a thru 0x9c3c2e5 data_size 0x16(0x9adc bytes) 2024-09-13T07:01:44.874 INFO:teuthology.orchestra.run.smithi118.stdout:truncating to largest ever: 0xeb6f05b 2024-09-13T07:01:44.875 INFO:teuthology.orchestra.run.smithi118.stdout:894 trunc from 0xe5f9137 to 0xeb6f05b 2024-09-13T07:01:44.887 INFO:teuthology.orchestra.run.smithi118.stdout:895 writesame 0x6253320 thru 0x625655c data_size 0x3(0x323d bytes) 2024-09-13T07:01:44.895 INFO:teuthology.orchestra.run.smithi118.stdout:896 write 0xce0860e thru 0xce139a1 (0xb394 bytes) 2024-09-13T07:01:44.914 INFO:teuthology.orchestra.run.smithi118.stdout:897 trunc from 0xeb6f05b to 0x9d2959f 2024-09-13T07:01:44.932 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:44.929+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:44.955 INFO:teuthology.orchestra.run.smithi118.stdout:898 write 0xb2ac304 thru 0xb2b467f (0x837c bytes) 2024-09-13T07:01:44.961 INFO:teuthology.orchestra.run.smithi118.stdout:899 write 0x6f7de03 thru 0x6f80d90 (0x2f8e bytes) 2024-09-13T07:01:44.968 INFO:teuthology.orchestra.run.smithi118.stdout:900 read 0x3af2457 thru 0x3af5fc7 (0x3b71 bytes) 2024-09-13T07:01:44.968 INFO:teuthology.orchestra.run.smithi118.stdout:902 read 0x461749e thru 0x4625b6a (0xe6cd bytes) 2024-09-13T07:01:44.968 INFO:teuthology.orchestra.run.smithi118.stdout:905 read 0x13873c2 thru 0x1389a96 (0x26d5 bytes) 2024-09-13T07:01:44.968 INFO:teuthology.orchestra.run.smithi118.stdout:906 writesame 0x73c07e4 thru 0x73c5d88 data_size 0x5(0x55a5 bytes) 2024-09-13T07:01:44.977 INFO:teuthology.orchestra.run.smithi118.stdout:907 punch from 0x3e8f459 to 0x3e9b436, (0xbfdd bytes) 2024-09-13T07:01:44.977 INFO:teuthology.orchestra.run.smithi118.stdout:908 punch from 0x2e41904 to 0x2e4d8a2, (0xbf9e bytes) 2024-09-13T07:01:44.977 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:44.977 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:44.979 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:44.979 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:44.979 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:44.979 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:44.979 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:44.991 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:01:44.993 INFO:teuthology.orchestra.run.smithi118.stdout:912 writesame 0xd75f923 thru 0xd761c88 data_size 0x17(0x2366 bytes) 2024-09-13T07:01:44.993 INFO:teuthology.orchestra.run.smithi118.stdout:915 read 0x2d42372 thru 0x2d4ca5f (0xa6ee bytes) 2024-09-13T07:01:45.016 INFO:teuthology.orchestra.run.smithi118.stdout:917 trunc from 0xd761c89 to 0x288ffe 2024-09-13T07:01:45.032 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:45.030+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:45.045 INFO:teuthology.orchestra.run.smithi118.stdout:919 punch from 0xf7c3d to 0x104586, (0xc949 bytes) 2024-09-13T07:01:45.060 INFO:teuthology.orchestra.run.smithi118.stdout:920 write 0x21600be thru 0x2163f94 (0x3ed7 bytes) 2024-09-13T07:01:45.099 INFO:teuthology.orchestra.run.smithi118.stdout:921 writesame 0xe025d55 thru 0xe03432c data_size 0x14(0xe5d8 bytes) 2024-09-13T07:01:45.101 INFO:teuthology.orchestra.run.smithi118.stdout:922 trunc from 0xe03432d to 0xc23b0ee 2024-09-13T07:01:45.106 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:45.106 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:45.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:45.106 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:45.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.106 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1528 3.0 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:45.106 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.049314+0000", 2024-09-13T07:01:45.106 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.056538020000001, 2024-09-13T07:01:45.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026695681999999998, 2024-09-13T07:01:45.107 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.107 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:45.107 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.107 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.107 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.107 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1528 2024-09-13T07:01:45.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.107 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.049314+0000", 2024-09-13T07:01:45.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.049314+0000", 2024-09-13T07:01:45.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.049311+0000", 2024-09-13T07:01:45.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:45.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055045+0000", 2024-09-13T07:01:45.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005733106 2024-09-13T07:01:45.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055075+0000", 2024-09-13T07:01:45.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0540999999999998e-05 2024-09-13T07:01:45.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055090+0000", 2024-09-13T07:01:45.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4827999999999999e-05 2024-09-13T07:01:45.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055215+0000", 2024-09-13T07:01:45.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000124657 2024-09-13T07:01:45.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055392+0000", 2024-09-13T07:01:45.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017693900000000001 2024-09-13T07:01:45.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-13T07:01:45.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056063+0000", 2024-09-13T07:01:45.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00067108999999999995 2024-09-13T07:01:45.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:45.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075194+0000", 2024-09-13T07:01:45.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019131756999999999 2024-09-13T07:01:45.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:45.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075871+0000", 2024-09-13T07:01:45.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00067684799999999997 2024-09-13T07:01:45.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:45.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075945+0000", 2024-09-13T07:01:45.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3468999999999997e-05 2024-09-13T07:01:45.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.076010+0000", 2024-09-13T07:01:45.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4911000000000002e-05 2024-09-13T07:01:45.113 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.113 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.113 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.114 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3121 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:01:45.114 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.260057+0000", 2024-09-13T07:01:45.114 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.845794627, 2024-09-13T07:01:45.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028240511999999999, 2024-09-13T07:01:45.114 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.114 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:45.114 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.114 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.114 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.114 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3121 2024-09-13T07:01:45.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.114 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260057+0000", 2024-09-13T07:01:45.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260057+0000", 2024-09-13T07:01:45.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260056+0000", 2024-09-13T07:01:45.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:45.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260066+0000", 2024-09-13T07:01:45.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0787e-05 2024-09-13T07:01:45.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260091+0000", 2024-09-13T07:01:45.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4947e-05 2024-09-13T07:01:45.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260103+0000", 2024-09-13T07:01:45.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1435e-05 2024-09-13T07:01:45.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260188+0000", 2024-09-13T07:01:45.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5352000000000005e-05 2024-09-13T07:01:45.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260310+0000", 2024-09-13T07:01:45.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122309 2024-09-13T07:01:45.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-13T07:01:45.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.267876+0000", 2024-09-13T07:01:45.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0075653990000000004 2024-09-13T07:01:45.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:45.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.288015+0000", 2024-09-13T07:01:45.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020139221999999998 2024-09-13T07:01:45.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:45.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.288112+0000", 2024-09-13T07:01:45.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6731000000000006e-05 2024-09-13T07:01:45.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:45.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.288214+0000", 2024-09-13T07:01:45.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010206599999999999 2024-09-13T07:01:45.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.288298+0000", 2024-09-13T07:01:45.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4054000000000005e-05 2024-09-13T07:01:45.120 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.120 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.121 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.121 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3827 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.121 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.651790+0000", 2024-09-13T07:01:45.121 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.454061611, 2024-09-13T07:01:45.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017344287999999999, 2024-09-13T07:01:45.121 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.121 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.121 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.121 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.121 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.122 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3827 2024-09-13T07:01:45.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.122 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651790+0000", 2024-09-13T07:01:45.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651790+0000", 2024-09-13T07:01:45.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651790+0000", 2024-09-13T07:01:45.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651792+0000", 2024-09-13T07:01:45.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:01:45.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651798+0000", 2024-09-13T07:01:45.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8799999999999996e-06 2024-09-13T07:01:45.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651803+0000", 2024-09-13T07:01:45.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.515e-06 2024-09-13T07:01:45.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665057+0000", 2024-09-13T07:01:45.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01325455 2024-09-13T07:01:45.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665447+0000", 2024-09-13T07:01:45.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00039010200000000003 2024-09-13T07:01:45.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669135+0000", 2024-09-13T07:01:45.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036872580000000001 2024-09-13T07:01:45.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.126 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.126 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3934 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 14155776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.126 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652214+0000", 2024-09-13T07:01:45.126 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453638368, 2024-09-13T07:01:45.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017524746000000001, 2024-09-13T07:01:45.127 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.127 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.127 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.127 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.127 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.127 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3934 2024-09-13T07:01:45.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.127 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652214+0000", 2024-09-13T07:01:45.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652214+0000", 2024-09-13T07:01:45.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652213+0000", 2024-09-13T07:01:45.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652216+0000", 2024-09-13T07:01:45.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2790000000000001e-06 2024-09-13T07:01:45.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652221+0000", 2024-09-13T07:01:45.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4859999999999997e-06 2024-09-13T07:01:45.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652225+0000", 2024-09-13T07:01:45.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0090000000000001e-06 2024-09-13T07:01:45.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669598+0000", 2024-09-13T07:01:45.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017373119999999999 2024-09-13T07:01:45.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669679+0000", 2024-09-13T07:01:45.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0636000000000005e-05 2024-09-13T07:01:45.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669738+0000", 2024-09-13T07:01:45.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9534000000000003e-05 2024-09-13T07:01:45.131 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.131 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.131 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.131 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3959 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 8650752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.131 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652229+0000", 2024-09-13T07:01:45.132 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453622938000001, 2024-09-13T07:01:45.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017820598, 2024-09-13T07:01:45.132 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.132 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.132 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.132 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.132 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.132 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3959 2024-09-13T07:01:45.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.132 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652229+0000", 2024-09-13T07:01:45.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652229+0000", 2024-09-13T07:01:45.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652229+0000", 2024-09-13T07:01:45.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652231+0000", 2024-09-13T07:01:45.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2800000000000002e-06 2024-09-13T07:01:45.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652236+0000", 2024-09-13T07:01:45.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3709999999999999e-06 2024-09-13T07:01:45.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652241+0000", 2024-09-13T07:01:45.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2300000000000002e-06 2024-09-13T07:01:45.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669762+0000", 2024-09-13T07:01:45.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017521473999999999 2024-09-13T07:01:45.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669889+0000", 2024-09-13T07:01:45.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012696199999999999 2024-09-13T07:01:45.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670050+0000", 2024-09-13T07:01:45.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000160602 2024-09-13T07:01:45.137 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.137 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.137 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.137 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3960 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 9961472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.137 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652245+0000", 2024-09-13T07:01:45.137 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453607179, 2024-09-13T07:01:45.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018033285999999999, 2024-09-13T07:01:45.137 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.137 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.137 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.138 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.138 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.138 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3960 2024-09-13T07:01:45.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.138 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652245+0000", 2024-09-13T07:01:45.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652245+0000", 2024-09-13T07:01:45.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652244+0000", 2024-09-13T07:01:45.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652255+0000", 2024-09-13T07:01:45.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:01:45.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652261+0000", 2024-09-13T07:01:45.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1310000000000003e-06 2024-09-13T07:01:45.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652266+0000", 2024-09-13T07:01:45.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3859999999999998e-06 2024-09-13T07:01:45.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670080+0000", 2024-09-13T07:01:45.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017813780000000001 2024-09-13T07:01:45.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670166+0000", 2024-09-13T07:01:45.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5586000000000004e-05 2024-09-13T07:01:45.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670278+0000", 2024-09-13T07:01:45.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011231 2024-09-13T07:01:45.143 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.143 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.143 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.143 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3961 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.143 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652271+0000", 2024-09-13T07:01:45.144 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453580929999999, 2024-09-13T07:01:45.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018223255000000001, 2024-09-13T07:01:45.144 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.144 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.144 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.144 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.144 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.144 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3961 2024-09-13T07:01:45.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.144 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652271+0000", 2024-09-13T07:01:45.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652271+0000", 2024-09-13T07:01:45.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652271+0000", 2024-09-13T07:01:45.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652274+0000", 2024-09-13T07:01:45.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.016e-06 2024-09-13T07:01:45.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652279+0000", 2024-09-13T07:01:45.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4639999999999997e-06 2024-09-13T07:01:45.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652283+0000", 2024-09-13T07:01:45.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3150000000000002e-06 2024-09-13T07:01:45.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670331+0000", 2024-09-13T07:01:45.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018047210000000001 2024-09-13T07:01:45.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670404+0000", 2024-09-13T07:01:45.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3588000000000001e-05 2024-09-13T07:01:45.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670494+0000", 2024-09-13T07:01:45.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0085000000000002e-05 2024-09-13T07:01:45.148 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.148 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.148 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.149 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3962 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.149 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652288+0000", 2024-09-13T07:01:45.149 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.45356411, 2024-09-13T07:01:45.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018397475999999999, 2024-09-13T07:01:45.149 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.149 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.149 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.150 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.150 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.150 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3962 2024-09-13T07:01:45.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.150 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652288+0000", 2024-09-13T07:01:45.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652288+0000", 2024-09-13T07:01:45.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652287+0000", 2024-09-13T07:01:45.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652290+0000", 2024-09-13T07:01:45.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4849999999999999e-06 2024-09-13T07:01:45.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652295+0000", 2024-09-13T07:01:45.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4489999999999998e-06 2024-09-13T07:01:45.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652300+0000", 2024-09-13T07:01:45.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4719999999999997e-06 2024-09-13T07:01:45.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670571+0000", 2024-09-13T07:01:45.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018270967999999999 2024-09-13T07:01:45.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670627+0000", 2024-09-13T07:01:45.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6128000000000002e-05 2024-09-13T07:01:45.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.154 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.154 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670685+0000", 2024-09-13T07:01:45.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8341000000000003e-05 2024-09-13T07:01:45.154 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.154 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.154 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.154 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3963 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.154 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652304+0000", 2024-09-13T07:01:45.154 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.45354747, 2024-09-13T07:01:45.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018517675000000001, 2024-09-13T07:01:45.155 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.155 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.155 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.155 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.155 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.155 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3963 2024-09-13T07:01:45.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.155 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652304+0000", 2024-09-13T07:01:45.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652304+0000", 2024-09-13T07:01:45.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652304+0000", 2024-09-13T07:01:45.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652307+0000", 2024-09-13T07:01:45.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5790000000000002e-06 2024-09-13T07:01:45.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652312+0000", 2024-09-13T07:01:45.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7259999999999999e-06 2024-09-13T07:01:45.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652317+0000", 2024-09-13T07:01:45.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0300000000000001e-06 2024-09-13T07:01:45.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670712+0000", 2024-09-13T07:01:45.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018394534000000001 2024-09-13T07:01:45.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670770+0000", 2024-09-13T07:01:45.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8041999999999998e-05 2024-09-13T07:01:45.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670822+0000", 2024-09-13T07:01:45.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2139999999999999e-05 2024-09-13T07:01:45.160 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.160 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.160 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.160 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3982 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.160 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652322+0000", 2024-09-13T07:01:45.160 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453529646, 2024-09-13T07:01:45.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018630451999999999, 2024-09-13T07:01:45.160 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.160 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.161 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.161 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.161 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.161 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3982 2024-09-13T07:01:45.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.161 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652322+0000", 2024-09-13T07:01:45.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652322+0000", 2024-09-13T07:01:45.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652322+0000", 2024-09-13T07:01:45.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652324+0000", 2024-09-13T07:01:45.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3300000000000001e-06 2024-09-13T07:01:45.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652329+0000", 2024-09-13T07:01:45.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7940000000000002e-06 2024-09-13T07:01:45.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652333+0000", 2024-09-13T07:01:45.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3930000000000001e-06 2024-09-13T07:01:45.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670845+0000", 2024-09-13T07:01:45.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018511933000000001 2024-09-13T07:01:45.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670901+0000", 2024-09-13T07:01:45.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.541e-05 2024-09-13T07:01:45.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670953+0000", 2024-09-13T07:01:45.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1943000000000001e-05 2024-09-13T07:01:45.165 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.165 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.165 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.165 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3983 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.165 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652338+0000", 2024-09-13T07:01:45.165 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453513996, 2024-09-13T07:01:45.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018767628000000001, 2024-09-13T07:01:45.166 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.166 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.166 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.166 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.166 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.166 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3983 2024-09-13T07:01:45.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.166 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652338+0000", 2024-09-13T07:01:45.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652338+0000", 2024-09-13T07:01:45.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652338+0000", 2024-09-13T07:01:45.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652340+0000", 2024-09-13T07:01:45.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5339999999999998e-06 2024-09-13T07:01:45.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652346+0000", 2024-09-13T07:01:45.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7769999999999999e-06 2024-09-13T07:01:45.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652350+0000", 2024-09-13T07:01:45.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4920000000000004e-06 2024-09-13T07:01:45.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670978+0000", 2024-09-13T07:01:45.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018627212000000001 2024-09-13T07:01:45.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671025+0000", 2024-09-13T07:01:45.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7604e-05 2024-09-13T07:01:45.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671106+0000", 2024-09-13T07:01:45.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0365999999999996e-05 2024-09-13T07:01:45.170 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.170 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.170 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.170 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3984 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 11665408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.170 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652355+0000", 2024-09-13T07:01:45.171 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453497322, 2024-09-13T07:01:45.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020621576999999999, 2024-09-13T07:01:45.171 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.171 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.171 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.171 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.171 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.171 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3984 2024-09-13T07:01:45.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.172 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652355+0000", 2024-09-13T07:01:45.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652355+0000", 2024-09-13T07:01:45.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652354+0000", 2024-09-13T07:01:45.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652357+0000", 2024-09-13T07:01:45.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2680000000000001e-06 2024-09-13T07:01:45.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652362+0000", 2024-09-13T07:01:45.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8289999999999996e-06 2024-09-13T07:01:45.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652367+0000", 2024-09-13T07:01:45.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4139999999999996e-06 2024-09-13T07:01:45.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671142+0000", 2024-09-13T07:01:45.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018775066 2024-09-13T07:01:45.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671194+0000", 2024-09-13T07:01:45.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1983e-05 2024-09-13T07:01:45.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672976+0000", 2024-09-13T07:01:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001782393 2024-09-13T07:01:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3985 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652371+0000", 2024-09-13T07:01:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453480845, 2024-09-13T07:01:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019123712000000001, 2024-09-13T07:01:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3985 2024-09-13T07:01:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652371+0000", 2024-09-13T07:01:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652371+0000", 2024-09-13T07:01:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652371+0000", 2024-09-13T07:01:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652373+0000", 2024-09-13T07:01:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.199e-06 2024-09-13T07:01:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652378+0000", 2024-09-13T07:01:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2360000000000004e-06 2024-09-13T07:01:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652382+0000", 2024-09-13T07:01:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2359999999999998e-06 2024-09-13T07:01:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671331+0000", 2024-09-13T07:01:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018948606 2024-09-13T07:01:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671406+0000", 2024-09-13T07:01:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5009e-05 2024-09-13T07:01:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671495+0000", 2024-09-13T07:01:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8769000000000003e-05 2024-09-13T07:01:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3986 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652387+0000", 2024-09-13T07:01:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453465157, 2024-09-13T07:01:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019262962000000002, 2024-09-13T07:01:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3986 2024-09-13T07:01:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652387+0000", 2024-09-13T07:01:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652387+0000", 2024-09-13T07:01:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652386+0000", 2024-09-13T07:01:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652389+0000", 2024-09-13T07:01:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.306e-06 2024-09-13T07:01:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652394+0000", 2024-09-13T07:01:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6760000000000004e-06 2024-09-13T07:01:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652399+0000", 2024-09-13T07:01:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:01:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671515+0000", 2024-09-13T07:01:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019116024999999998 2024-09-13T07:01:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671581+0000", 2024-09-13T07:01:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6105999999999994e-05 2024-09-13T07:01:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671650+0000", 2024-09-13T07:01:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8577000000000004e-05 2024-09-13T07:01:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4011 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652403+0000", 2024-09-13T07:01:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453448665, 2024-09-13T07:01:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019398288999999999, 2024-09-13T07:01:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4011 2024-09-13T07:01:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652403+0000", 2024-09-13T07:01:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652403+0000", 2024-09-13T07:01:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652403+0000", 2024-09-13T07:01:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652405+0000", 2024-09-13T07:01:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2479999999999999e-06 2024-09-13T07:01:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652410+0000", 2024-09-13T07:01:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2240000000000003e-06 2024-09-13T07:01:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652415+0000", 2024-09-13T07:01:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3510000000000004e-06 2024-09-13T07:01:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671669+0000", 2024-09-13T07:01:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019253849999999999 2024-09-13T07:01:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671732+0000", 2024-09-13T07:01:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3474999999999998e-05 2024-09-13T07:01:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671802+0000", 2024-09-13T07:01:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9522000000000003e-05 2024-09-13T07:01:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4012 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652419+0000", 2024-09-13T07:01:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453433235, 2024-09-13T07:01:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019542509, 2024-09-13T07:01:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4012 2024-09-13T07:01:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652419+0000", 2024-09-13T07:01:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652419+0000", 2024-09-13T07:01:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652418+0000", 2024-09-13T07:01:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652421+0000", 2024-09-13T07:01:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2970000000000002e-06 2024-09-13T07:01:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652426+0000", 2024-09-13T07:01:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7379999999999999e-06 2024-09-13T07:01:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652431+0000", 2024-09-13T07:01:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5580000000000002e-06 2024-09-13T07:01:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671823+0000", 2024-09-13T07:01:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019391731999999998 2024-09-13T07:01:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671890+0000", 2024-09-13T07:01:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7563000000000003e-05 2024-09-13T07:01:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671961+0000", 2024-09-13T07:01:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0955000000000001e-05 2024-09-13T07:01:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4013 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 11403264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652435+0000", 2024-09-13T07:01:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453416983, 2024-09-13T07:01:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019849087000000001, 2024-09-13T07:01:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4013 2024-09-13T07:01:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652435+0000", 2024-09-13T07:01:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652435+0000", 2024-09-13T07:01:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652435+0000", 2024-09-13T07:01:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652437+0000", 2024-09-13T07:01:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0810000000000001e-06 2024-09-13T07:01:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652442+0000", 2024-09-13T07:01:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2889999999999997e-06 2024-09-13T07:01:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652446+0000", 2024-09-13T07:01:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4449999999999998e-06 2024-09-13T07:01:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671982+0000", 2024-09-13T07:01:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01953531 2024-09-13T07:01:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672051+0000", 2024-09-13T07:01:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9488999999999998e-05 2024-09-13T07:01:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672284+0000", 2024-09-13T07:01:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023278799999999999 2024-09-13T07:01:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4014 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652450+0000", 2024-09-13T07:01:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453401626, 2024-09-13T07:01:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020000851, 2024-09-13T07:01:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4014 2024-09-13T07:01:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652450+0000", 2024-09-13T07:01:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652450+0000", 2024-09-13T07:01:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652450+0000", 2024-09-13T07:01:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652452+0000", 2024-09-13T07:01:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0339999999999999e-06 2024-09-13T07:01:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652457+0000", 2024-09-13T07:01:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3739999999999997e-06 2024-09-13T07:01:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652462+0000", 2024-09-13T07:01:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7180000000000004e-06 2024-09-13T07:01:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672307+0000", 2024-09-13T07:01:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019844653 2024-09-13T07:01:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672380+0000", 2024-09-13T07:01:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3485000000000005e-05 2024-09-13T07:01:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672451+0000", 2024-09-13T07:01:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0901000000000004e-05 2024-09-13T07:01:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4015 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652470+0000", 2024-09-13T07:01:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.453382124999999, 2024-09-13T07:01:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020116328999999999, 2024-09-13T07:01:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4015 2024-09-13T07:01:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652470+0000", 2024-09-13T07:01:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652470+0000", 2024-09-13T07:01:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652469+0000", 2024-09-13T07:01:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652478+0000", 2024-09-13T07:01:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1990000000000006e-06 2024-09-13T07:01:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652484+0000", 2024-09-13T07:01:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9800000000000003e-06 2024-09-13T07:01:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652488+0000", 2024-09-13T07:01:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3880000000000002e-06 2024-09-13T07:01:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672465+0000", 2024-09-13T07:01:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019976694999999999 2024-09-13T07:01:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672511+0000", 2024-09-13T07:01:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6272e-05 2024-09-13T07:01:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672586+0000", 2024-09-13T07:01:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5143999999999997e-05 2024-09-13T07:01:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:01:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:01:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.6122155540000001, 2024-09-13T07:01:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:01:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:01:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:01:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:01:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:01:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:01:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:01:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:01:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:01:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:01:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:01:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:01:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:01:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:01:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:01:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:01:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:01:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:01:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:01:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:01:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:01:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:01:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:45.226 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:45.228 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:01:45.229 INFO:teuthology.orchestra.run.smithi118.stdout:923 read 0x6f691ab thru 0x6f7361f (0xa475 bytes) 2024-09-13T07:01:45.229 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:45.229 INFO:teuthology.orchestra.run.smithi118.stdout:925 read 0x813cd88 thru 0x8144f54 (0x81cd bytes) 2024-09-13T07:01:45.229 INFO:teuthology.orchestra.run.smithi118.stdout:927 trunc from 0xc23b0ee to 0x13b459b 2024-09-13T07:01:45.229 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:45.230 INFO:teuthology.orchestra.run.smithi118.stdout:929 trunc from 0x13b459b to 0x6852d70 2024-09-13T07:01:45.230 INFO:teuthology.orchestra.run.smithi118.stdout:930 punch from 0x1571558 to 0x1576d71, (0x5819 bytes) 2024-09-13T07:01:45.230 INFO:teuthology.orchestra.run.smithi118.stdout:932 read 0x5f80db2 thru 0x5f8381c (0x2a6b bytes) 2024-09-13T07:01:45.230 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:45.230 INFO:teuthology.orchestra.run.smithi118.stdout:935 write 0xda98412 thru 0xdaa3b52 (0xb741 bytes) 2024-09-13T07:01:45.230 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:45.230 INFO:teuthology.orchestra.run.smithi118.stdout:937 write 0xa513969 thru 0xa515bd3 (0x226b bytes) 2024-09-13T07:01:45.230 INFO:teuthology.orchestra.run.smithi118.stdout:939 clone 13 order 21 su 8192 sc 3 2024-09-13T07:01:45.230 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:45.222+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:45.291 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:01:45.328 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:45.325+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:45.396 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:45.397 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:45.397 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:45.397 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:45.406 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:01:45.429 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:45.426+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:45.431 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:45.457 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:45.473 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:01:45.486 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:45.505 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:45.530 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:45.530 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:45.526+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:45.534 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:01:45.534 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:01:45.534 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 3, 4, 5, 6, 7] 2024-09-13T07:01:45.534 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-13T07:01:45.543 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:45.544 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:45.554 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:01:45.557 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:45.583 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:45.606 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:45.608 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:45.608 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:45.608 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:45.608 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:45.608 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:45.608 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:45.608 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:45.608 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:45.661 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:45.661+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:45.681 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:45.681 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:45.691 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:01:45.699 INFO:teuthology.orchestra.run.smithi118.stdout:flattening image image_client.0-clone12 2024-09-13T07:01:45.731 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:45.726+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:45.808 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:45.808 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:45.819 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:01:45.832 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:45.829+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:45.855 INFO:tasks.ceph.mon.a.smithi118.stderr:2024-09-13T07:01:45.853+0000 7fd1ec319700 -1 mon.a@0(leader).osd e47 definitely_dead 0 2024-09-13T07:01:45.923 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:45.923 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:45.934 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:01:45.934 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:45.933+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:46.035 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:46.030+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:46.057 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-13T07:01:46.068 INFO:teuthology.orchestra.run.smithi118.stdout:940 trunc from 0xdaa3b53 to 0xe1f5842 2024-09-13T07:01:46.084 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:46.084 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:46.084 INFO:teuthology.orchestra.run.smithi118.stdout:943 write 0x96a8e05 thru 0x96ae7ec (0x59e8 bytes) 2024-09-13T07:01:46.093 INFO:teuthology.orchestra.run.smithi118.stdout:944 writesame 0xa6bdca0 thru 0xa6ccece data_size 0x11(0xf22f bytes) 2024-09-13T07:01:46.100 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:46.100 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:46.110 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:01:46.136 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:46.133+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:46.221 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:46.222 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:46.231 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:01:46.232 INFO:teuthology.orchestra.run.smithi118.stdout:945 trunc from 0xe1f5842 to 0xbf37593 2024-09-13T07:01:46.237 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:46.234+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:46.338 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:46.337+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:46.347 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:46.347 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:46.352 INFO:teuthology.orchestra.run.smithi118.stdout:946 trunc from 0xbf37593 to 0xc267e9e 2024-09-13T07:01:46.357 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:01:46.362 INFO:teuthology.orchestra.run.smithi118.stdout:948 read 0x1803235 thru 0x180722e (0x3ffa bytes) 2024-09-13T07:01:46.376 INFO:teuthology.orchestra.run.smithi118.stdout:949 trunc from 0xc267e9e to 0x608fe30 2024-09-13T07:01:46.403 INFO:teuthology.orchestra.run.smithi118.stdout:950 read 0x3597fb2 thru 0x35a4f78 (0xcfc7 bytes) 2024-09-13T07:01:46.425 INFO:teuthology.orchestra.run.smithi118.stdout:951 write 0xcf5684c thru 0xcf66255 (0xfa0a bytes) 2024-09-13T07:01:46.435 INFO:teuthology.orchestra.run.smithi118.stdout:952 read 0x4131d9f thru 0x4141150 (0xf3b2 bytes) 2024-09-13T07:01:46.435 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:46.435 INFO:teuthology.orchestra.run.smithi118.stdout:956 writesame 0xa8a7612 thru 0xa8ab953 data_size 0x1(0x4342 bytes) 2024-09-13T07:01:46.438 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:46.434+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:46.449 INFO:teuthology.orchestra.run.smithi118.stdout:960 read 0x75a898e thru 0x75b48a5 (0xbf18 bytes) 2024-09-13T07:01:46.450 INFO:teuthology.orchestra.run.smithi118.stdout:962 write 0xa8765c6 thru 0xa881492 (0xaecd bytes) 2024-09-13T07:01:46.454 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:46.464 INFO:teuthology.orchestra.run.smithi118.stdout:965 write 0xd6f972e thru 0xd6faad0 (0x13a3 bytes) 2024-09-13T07:01:46.467 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:46.467 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:46.478 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:01:46.485 INFO:teuthology.orchestra.run.smithi118.stdout:967 trunc from 0xd6faad1 to 0x843c0d9 2024-09-13T07:01:46.486 INFO:teuthology.orchestra.run.smithi118.stderr:marked out osd.0. 2024-09-13T07:01:46.539 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:46.534+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:46.631 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:46.632 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:46.640 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:46.641+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:46.642 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:01:46.741 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:46.737+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:46.748 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:46.748 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:46.757 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:01:46.843 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:46.838+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:46.859 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:46.859 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:46.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:01:46.943 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:46.941+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:47.021 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:47.021 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:47.031 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:01:47.043 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:47.038+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:47.141 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:47.141 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:47.144 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:47.141+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:47.151 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:01:47.245 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:47.241+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:47.298 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:47.298 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:47.308 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:01:47.346 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:47.342+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:47.414 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:47.414 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:47.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:01:47.468 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:47.466+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:47.486 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:01:47.548 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:47.546+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:47.589 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:47.590 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:47.599 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:01:47.620 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:01:47.649 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:47.646+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:47.702 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:47.702 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:47.712 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:01:47.750 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:47.746+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:47.758 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:47.778 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:47.796 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:47.813 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:47.813 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:47.813 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:47.814 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:47.814 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:47.823 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:01:47.834 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:47.851 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:47.849+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:47.857 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:47.882 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:47.908 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:47.910 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:47.911 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:47.911 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:47.911 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:47.911 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:47.911 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:47.911 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:47.911 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:47.921 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:47.921 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:47.921 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:47.930 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:01:47.951 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:47.946+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:48.053 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:48.049+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:48.086 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:48.087 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:48.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:48.087 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:48.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.087 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:48.087 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:48.087 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.580928168, 2024-09-13T07:01:48.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:01:48.087 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.087 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.088 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.088 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.088 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.088 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:01:48.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.088 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:48.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:48.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:01:48.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:48.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:01:48.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:01:48.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:01:48.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:01:48.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:01:48.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:01:48.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:01:48.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:01:48.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:01:48.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:01:48.093 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.093 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.093 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.093 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:48.093 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:48.093 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.556100458, 2024-09-13T07:01:48.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:01:48.093 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.093 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.093 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:48.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:01:48.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:48.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:01:48.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:01:48.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:01:48.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:01:48.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:48.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:01:48.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:01:48.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:01:48.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:01:48.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:01:48.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:01:48.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:01:48.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:01:48.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.017428634000002, 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:01:48.099 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.100 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.100 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.100 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.100 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.100 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:01:48.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.100 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:48.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:48.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:01:48.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:48.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:01:48.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:01:48.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:01:48.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:01:48.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:48.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:01:48.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:01:48.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:01:48.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:01:48.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:01:48.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:01:48.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:01:48.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:01:48.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:01:48.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:01:48.105 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.105 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.105 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.105 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:48.106 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:48.106 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.987803441000001, 2024-09-13T07:01:48.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:01:48.106 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.106 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.106 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.106 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.106 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.106 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:01:48.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.106 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:48.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:48.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:01:48.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:48.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:01:48.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:48.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:48.111 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:48.112 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.45508981, 2024-09-13T07:01:48.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:01:48.112 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.112 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.112 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.112 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.112 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.112 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:01:48.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.112 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:48.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:48.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:48.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:48.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:01:48.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:01:48.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:01:48.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:01:48.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:01:48.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:01:48.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:01:48.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:48.117 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.455007353999999, 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:48.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:01:48.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:48.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:01:48.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:01:48.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:01:48.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:01:48.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:01:48.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:01:48.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:01:48.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:01:48.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:01:48.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:01:48.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:01:48.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:48.123 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.454975544, 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:48.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:01:48.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:01:48.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:01:48.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:01:48.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:01:48.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:48.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:01:48.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:01:48.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:01:48.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:01:48.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:01:48.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:01:48.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:01:48.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:01:48.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:01:48.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:01:48.129 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.129 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.129 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.129 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:48.129 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:48.129 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.454920386000001, 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:01:48.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:01:48.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:01:48.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:01:48.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:01:48.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:01:48.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:01:48.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:01:48.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:01:48.135 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.135 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.135 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.135 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:48.135 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:48.135 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.454875106999999, 2024-09-13T07:01:48.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:01:48.136 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.136 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.136 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.136 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.136 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.136 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:01:48.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.136 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:48.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:01:48.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:01:48.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:01:48.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:01:48.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:01:48.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:01:48.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:01:48.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.454836573000001, 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:01:48.141 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:01:48.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:01:48.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:01:48.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:48.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:01:48.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:01:48.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:01:48.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:01:48.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:01:48.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:01:48.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:01:48.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:01:48.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.454774954000001, 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.147 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.148 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.148 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.148 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.148 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:01:48.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.148 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:48.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:01:48.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:01:48.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:01:48.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:48.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:01:48.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:01:48.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.440552271000001, 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.153 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:48.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:01:48.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:01:48.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:01:48.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:01:48.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:01:48.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:01:48.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:01:48.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:01:48.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:01:48.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:01:48.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:01:48.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:01:48.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.428673751000002, 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.159 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.160 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.160 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.160 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:01:48.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.160 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:48.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:48.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:01:48.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:01:48.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:01:48.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:01:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:01:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:01:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:01:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.164 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.164 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:48.164 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.42811669, 2024-09-13T07:01:48.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:01:48.164 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.164 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.164 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.164 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.164 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.164 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:01:48.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.164 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:01:48.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:01:48.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:01:48.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:01:48.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:01:48.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:01:48.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:01:48.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:01:48.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.428094535, 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:01:48.172 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.173 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.173 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.173 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.173 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:48.173 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.428074817999999, 2024-09-13T07:01:48.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:01:48.173 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.173 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.173 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.173 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.173 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:01:48.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:01:48.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:01:48.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:01:48.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:01:48.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:01:48.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:01:48.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:01:48.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:01:48.177 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.428057488, 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.178 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:01:48.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:01:48.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:01:48.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:01:48.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:48.182 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.427761826000001, 2024-09-13T07:01:48.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:01:48.183 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.183 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.183 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.183 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.183 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.183 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:01:48.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.183 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:48.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:48.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:48.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:01:48.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:01:48.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:01:48.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:01:48.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:01:48.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:01:48.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:01:48.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:01:48.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:01:48.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:01:48.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:01:48.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:01:48.187 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.187 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.187 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.187 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:48.187 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:01:48.187 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.8665590070000002, 2024-09-13T07:01:48.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:01:48.188 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.188 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.188 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.188 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.188 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.188 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:01:48.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.188 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:01:48.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:01:48.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:01:48.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:48.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:01:48.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:01:48.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:01:48.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:01:48.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:01:48.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:01:48.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:01:48.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:01:48.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:01:48.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:01:48.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:01:48.193 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.193 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.193 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.193 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:01:48.193 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.8344801039999998, 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:01:48.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:01:48.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:01:48.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:01:48.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:01:48.197 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.197 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.197 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.197 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.197 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.197 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:48.199 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:01:48.199 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:48.198+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:48.300 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:48.297+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:48.311 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:48.311 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:48.311 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:48.311 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:48.321 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:01:48.401 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:48.398+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:48.432 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:48.432 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:48.432 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:48.432 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:48.432 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:48.442 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:01:48.489 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:01:48.502 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:48.501+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:48.508 INFO:teuthology.orchestra.run.smithi118.stdout:968 punch from 0x3149fa2 to 0x314dde3, (0x3e41 bytes) 2024-09-13T07:01:48.508 INFO:teuthology.orchestra.run.smithi118.stdout:970 writesame 0x74c78fd thru 0x74cdc97 data_size 0x1(0x639b bytes) 2024-09-13T07:01:48.525 INFO:teuthology.orchestra.run.smithi118.stdout:971 read 0x8028c2b thru 0x802f7bb (0x6b91 bytes) 2024-09-13T07:01:48.525 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:48.525 INFO:teuthology.orchestra.run.smithi118.stdout:973 punch from 0x48995d to 0x48c625, (0x2cc8 bytes) 2024-09-13T07:01:48.525 INFO:teuthology.orchestra.run.smithi118.stdout:975 read 0x6592ccd thru 0x659e303 (0xb637 bytes) 2024-09-13T07:01:48.530 INFO:teuthology.orchestra.run.smithi118.stdout:977 write 0x211e671 thru 0x21278b9 (0x9249 bytes) 2024-09-13T07:01:48.565 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:48.565 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:48.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:48.565 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:48.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.566 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1528 3.0 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:48.566 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.049314+0000", 2024-09-13T07:01:48.566 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.515203351, 2024-09-13T07:01:48.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026695681999999998, 2024-09-13T07:01:48.566 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.566 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.566 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.566 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.566 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.566 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1528 2024-09-13T07:01:48.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.566 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.049314+0000", 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.049314+0000", 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.049311+0000", 2024-09-13T07:01:48.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:48.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055045+0000", 2024-09-13T07:01:48.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005733106 2024-09-13T07:01:48.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055075+0000", 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0540999999999998e-05 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055090+0000", 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4827999999999999e-05 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055215+0000", 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000124657 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.055392+0000", 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017693900000000001 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056063+0000", 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00067108999999999995 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075194+0000", 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019131756999999999 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075871+0000", 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00067684799999999997 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075945+0000", 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3468999999999997e-05 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.076010+0000", 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4911000000000002e-05 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.571 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3121 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.260057+0000", 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.304459957999999, 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.028240511999999999, 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3121 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.572 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260057+0000", 2024-09-13T07:01:48.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260057+0000", 2024-09-13T07:01:48.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260056+0000", 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260066+0000", 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0787e-05 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260091+0000", 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4947e-05 2024-09-13T07:01:48.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260103+0000", 2024-09-13T07:01:48.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1435e-05 2024-09-13T07:01:48.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260188+0000", 2024-09-13T07:01:48.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5352000000000005e-05 2024-09-13T07:01:48.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.260310+0000", 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000122309 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 2", 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.267876+0000", 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0075653990000000004 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.288015+0000", 2024-09-13T07:01:48.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020139221999999998 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.288112+0000", 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6731000000000006e-05 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.288214+0000", 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010206599999999999 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.288298+0000", 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4054000000000005e-05 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3827 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.651790+0000", 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912726941999999, 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017344287999999999, 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.578 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3827 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651790+0000", 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651790+0000", 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651790+0000", 2024-09-13T07:01:48.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651792+0000", 2024-09-13T07:01:48.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:01:48.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651798+0000", 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8799999999999996e-06 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.651803+0000", 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.515e-06 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665057+0000", 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01325455 2024-09-13T07:01:48.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.665447+0000", 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00039010200000000003 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669135+0000", 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0036872580000000001 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3934 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 14155776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652214+0000", 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912303698999999, 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017524746000000001, 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3934 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652214+0000", 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652214+0000", 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652213+0000", 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652216+0000", 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2790000000000001e-06 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652221+0000", 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4859999999999997e-06 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652225+0000", 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0090000000000001e-06 2024-09-13T07:01:48.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669598+0000", 2024-09-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017373119999999999 2024-09-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669679+0000", 2024-09-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0636000000000005e-05 2024-09-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669738+0000", 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9534000000000003e-05 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3959 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 8650752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652229+0000", 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912288269000001, 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017820598, 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.587 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3959 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652229+0000", 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652229+0000", 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652229+0000", 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652231+0000", 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2800000000000002e-06 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652236+0000", 2024-09-13T07:01:48.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3709999999999999e-06 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652241+0000", 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2300000000000002e-06 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669762+0000", 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017521473999999999 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669889+0000", 2024-09-13T07:01:48.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012696199999999999 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670050+0000", 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000160602 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3960 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 9961472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652245+0000", 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912272510000001, 2024-09-13T07:01:48.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018033285999999999, 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3960 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652245+0000", 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652245+0000", 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652244+0000", 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652255+0000", 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0458999999999999e-05 2024-09-13T07:01:48.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652261+0000", 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1310000000000003e-06 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652266+0000", 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3859999999999998e-06 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670080+0000", 2024-09-13T07:01:48.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017813780000000001 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670166+0000", 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5586000000000004e-05 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670278+0000", 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011231 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.596 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3961 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.596 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652271+0000", 2024-09-13T07:01:48.596 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912246261, 2024-09-13T07:01:48.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018223255000000001, 2024-09-13T07:01:48.596 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.596 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.596 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.596 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.596 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.596 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3961 2024-09-13T07:01:48.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.596 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652271+0000", 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652271+0000", 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652271+0000", 2024-09-13T07:01:48.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652274+0000", 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.016e-06 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652279+0000", 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4639999999999997e-06 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652283+0000", 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3150000000000002e-06 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670331+0000", 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018047210000000001 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670404+0000", 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3588000000000001e-05 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670494+0000", 2024-09-13T07:01:48.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0085000000000002e-05 2024-09-13T07:01:48.600 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.600 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.600 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.600 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3962 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.600 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652288+0000", 2024-09-13T07:01:48.600 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912229441000001, 2024-09-13T07:01:48.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018397475999999999, 2024-09-13T07:01:48.600 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.600 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.600 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3962 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652288+0000", 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652288+0000", 2024-09-13T07:01:48.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652287+0000", 2024-09-13T07:01:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652290+0000", 2024-09-13T07:01:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4849999999999999e-06 2024-09-13T07:01:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652295+0000", 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4489999999999998e-06 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652300+0000", 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4719999999999997e-06 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670571+0000", 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018270967999999999 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670627+0000", 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6128000000000002e-05 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670685+0000", 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8341000000000003e-05 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.604 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3963 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652304+0000", 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912212800999999, 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018517675000000001, 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3963 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652304+0000", 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652304+0000", 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652304+0000", 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652307+0000", 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5790000000000002e-06 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652312+0000", 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7259999999999999e-06 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652317+0000", 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0300000000000001e-06 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670712+0000", 2024-09-13T07:01:48.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018394534000000001 2024-09-13T07:01:48.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670770+0000", 2024-09-13T07:01:48.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8041999999999998e-05 2024-09-13T07:01:48.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670822+0000", 2024-09-13T07:01:48.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2139999999999999e-05 2024-09-13T07:01:48.608 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.608 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3982 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 9043968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652322+0000", 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912194976999999, 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018630451999999999, 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3982 2024-09-13T07:01:48.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652322+0000", 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652322+0000", 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652322+0000", 2024-09-13T07:01:48.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652324+0000", 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3300000000000001e-06 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652329+0000", 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7940000000000002e-06 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652333+0000", 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3930000000000001e-06 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670845+0000", 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018511933000000001 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670901+0000", 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.541e-05 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670953+0000", 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1943000000000001e-05 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3983 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652338+0000", 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912179327, 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018767628000000001, 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.613 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3983 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652338+0000", 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652338+0000", 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652338+0000", 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652340+0000", 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5339999999999998e-06 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652346+0000", 2024-09-13T07:01:48.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7769999999999999e-06 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652350+0000", 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4920000000000004e-06 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670978+0000", 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018627212000000001 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671025+0000", 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7604e-05 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671106+0000", 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0365999999999996e-05 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3984 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 11665408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652355+0000", 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912162652999999, 2024-09-13T07:01:48.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020621576999999999, 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3984 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652355+0000", 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652355+0000", 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652354+0000", 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652357+0000", 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2680000000000001e-06 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652362+0000", 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8289999999999996e-06 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652367+0000", 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4139999999999996e-06 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671142+0000", 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018775066 2024-09-13T07:01:48.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671194+0000", 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1983e-05 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672976+0000", 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001782393 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3985 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652371+0000", 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912146176, 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019123712000000001, 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3985 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652371+0000", 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652371+0000", 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652371+0000", 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652373+0000", 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.199e-06 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652378+0000", 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2360000000000004e-06 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652382+0000", 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2359999999999998e-06 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671331+0000", 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018948606 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671406+0000", 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5009e-05 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671495+0000", 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8769000000000003e-05 2024-09-13T07:01:48.625 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3986 3.0 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:8 [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652387+0000", 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912130487999999, 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019262962000000002, 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.626 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3986 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652387+0000", 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652387+0000", 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652386+0000", 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652389+0000", 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.306e-06 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652394+0000", 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6760000000000004e-06 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652399+0000", 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671515+0000", 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019116024999999998 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671581+0000", 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6105999999999994e-05 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671650+0000", 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8577000000000004e-05 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4011 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652403+0000", 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912113995999999, 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019398288999999999, 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4011 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652403+0000", 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652403+0000", 2024-09-13T07:01:48.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652403+0000", 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652405+0000", 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2479999999999999e-06 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652410+0000", 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2240000000000003e-06 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652415+0000", 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3510000000000004e-06 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671669+0000", 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019253849999999999 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671732+0000", 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3474999999999998e-05 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671802+0000", 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9522000000000003e-05 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4012 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652419+0000", 2024-09-13T07:01:48.634 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912098566000001, 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019542509, 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4012 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652419+0000", 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652419+0000", 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652418+0000", 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652421+0000", 2024-09-13T07:01:48.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2970000000000002e-06 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652426+0000", 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7379999999999999e-06 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652431+0000", 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5580000000000002e-06 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671823+0000", 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019391731999999998 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671890+0000", 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7563000000000003e-05 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671961+0000", 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0955000000000001e-05 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4013 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 11403264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652435+0000", 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912082313999999, 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019849087000000001, 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4013 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.639 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652435+0000", 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652435+0000", 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652435+0000", 2024-09-13T07:01:48.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652437+0000", 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0810000000000001e-06 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652442+0000", 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2889999999999997e-06 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652446+0000", 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4449999999999998e-06 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671982+0000", 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01953531 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672051+0000", 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9488999999999998e-05 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672284+0000", 2024-09-13T07:01:48.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023278799999999999 2024-09-13T07:01:48.643 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.643 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.643 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.643 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4014 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.643 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652450+0000", 2024-09-13T07:01:48.643 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912066957, 2024-09-13T07:01:48.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020000851, 2024-09-13T07:01:48.643 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.644 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.644 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.644 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.644 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.644 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4014 2024-09-13T07:01:48.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.644 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652450+0000", 2024-09-13T07:01:48.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652450+0000", 2024-09-13T07:01:48.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652450+0000", 2024-09-13T07:01:48.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652452+0000", 2024-09-13T07:01:48.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0339999999999999e-06 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652457+0000", 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3739999999999997e-06 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652462+0000", 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7180000000000004e-06 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672307+0000", 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019844653 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672380+0000", 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3485000000000005e-05 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672451+0000", 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0901000000000004e-05 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4015 3.0 3:05869443:::rbd_data.11945d91e99c.0000000000000006:8 [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.652470+0000", 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.912047456, 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020116328999999999, 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4015 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652470+0000", 2024-09-13T07:01:48.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652470+0000", 2024-09-13T07:01:48.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652469+0000", 2024-09-13T07:01:48.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:48.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652478+0000", 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1990000000000006e-06 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652484+0000", 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9800000000000003e-06 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.652488+0000", 2024-09-13T07:01:48.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3880000000000002e-06 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672465+0000", 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019976694999999999 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672511+0000", 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6272e-05 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672586+0000", 2024-09-13T07:01:48.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5143999999999997e-05 2024-09-13T07:01:48.652 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.652 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.652 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.652 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:01:48.652 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:01:48.652 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.0708808850000002, 2024-09-13T07:01:48.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:01:48.652 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:48.652 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:48.652 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:01:48.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:48.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:48.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:48.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:48.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:01:48.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:01:48.657 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.657 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.657 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.657 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:48.657 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:48.657 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:48.659 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:01:48.659 INFO:teuthology.orchestra.run.smithi118.stdout:978 write 0x2959b83 thru 0x29609d1 (0x6e4f bytes) 2024-09-13T07:01:48.659 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:48.657+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:48.674 INFO:teuthology.orchestra.run.smithi118.stdout:979 write 0xc25f490 thru 0xc26f392 (0xff03 bytes) 2024-09-13T07:01:48.683 INFO:teuthology.orchestra.run.smithi118.stdout:980 writesame 0xa650efe thru 0xa657051 data_size 0x2(0x6154 bytes) 2024-09-13T07:01:48.696 INFO:teuthology.orchestra.run.smithi118.stdout:981 read 0xbd66307 thru 0xbd725da (0xc2d4 bytes) 2024-09-13T07:01:48.696 INFO:teuthology.orchestra.run.smithi118.stdout:983 read 0xa2a805a thru 0xa2ae717 (0x66be bytes) 2024-09-13T07:01:48.697 INFO:teuthology.orchestra.run.smithi118.stdout:985 punch from 0xa634c0 to 0xa68260, (0x4da0 bytes) 2024-09-13T07:01:48.697 INFO:teuthology.orchestra.run.smithi118.stdout:988 write 0x41dbaaf thru 0x41e0ed0 (0x5422 bytes) 2024-09-13T07:01:48.707 INFO:teuthology.orchestra.run.smithi118.stdout:989 writesame 0x4abcb77 thru 0x4ac2201 data_size 0x23(0x568b bytes) 2024-09-13T07:01:48.717 INFO:teuthology.orchestra.run.smithi118.stdout:990 write 0x5724493 thru 0x5729d5e (0x58cc bytes) 2024-09-13T07:01:48.726 INFO:teuthology.orchestra.run.smithi118.stdout:991 writesame 0xaab6a25 thru 0xaac1aec data_size 0x2(0xb0c8 bytes) 2024-09-13T07:01:48.739 INFO:teuthology.orchestra.run.smithi118.stdout:992 write 0x52004a2 thru 0x520dee2 (0xda41 bytes) 2024-09-13T07:01:48.749 INFO:teuthology.orchestra.run.smithi118.stdout:993 writesame 0x407f3a3 thru 0x4080d25 data_size 0x7(0x1983 bytes) 2024-09-13T07:01:48.753 INFO:teuthology.orchestra.run.smithi118.stdout:994 read 0x1d28bfa thru 0x1d31269 (0x8670 bytes) 2024-09-13T07:01:48.754 INFO:teuthology.orchestra.run.smithi118.stdout:996 punch from 0x6711c4c to 0x671dff9, (0xc3ad bytes) 2024-09-13T07:01:48.754 INFO:teuthology.orchestra.run.smithi118.stdout:997 read 0x16b988b thru 0x16c36e8 (0x9e5e bytes) 2024-09-13T07:01:48.754 INFO:teuthology.orchestra.run.smithi118.stdout:998 punch from 0x7e1bcd2 to 0x7e274d1, (0xb7ff bytes) 2024-09-13T07:01:48.754 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:48.754 INFO:teuthology.orchestra.run.smithi118.stdout:1000 writesame 0xb97664a thru 0xb976f13 data_size 0x12(0x8ca bytes) 2024-09-13T07:01:48.759 INFO:teuthology.orchestra.run.smithi118.stdout:1001 punch from 0x60a03fb to 0x60a94db, (0x90e0 bytes) 2024-09-13T07:01:48.759 INFO:teuthology.orchestra.run.smithi118.stdout:1002 writesame 0x76becf thru 0x76e599 data_size 0x1(0x26cb bytes) 2024-09-13T07:01:48.759 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:48.757+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:48.789 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:48.790 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:48.790 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:48.790 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:48.800 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:01:48.860 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:48.857+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:48.884 INFO:teuthology.orchestra.run.smithi118.stdout:1003 write 0xda495fb thru 0xda5809f (0xeaa5 bytes) 2024-09-13T07:01:48.902 INFO:teuthology.orchestra.run.smithi118.stdout:1004 trunc from 0xda580a0 to 0x9bc3b66 2024-09-13T07:01:48.940 INFO:teuthology.orchestra.run.smithi118.stdout:1005 trunc from 0x9bc3b66 to 0x1bb8d21 2024-09-13T07:01:48.961 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:48.958+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:48.977 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:48.977 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:48.977 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:48.977 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:48.977 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:48.987 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:01:49.062 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:49.061+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:49.070 INFO:teuthology.orchestra.run.smithi118.stdout:1006 read 0x17863c2 thru 0x1790ca4 (0xa8e3 bytes) 2024-09-13T07:01:49.089 INFO:teuthology.orchestra.run.smithi118.stdout:1007 writesame 0x72f1cad thru 0x72f6c7a data_size 0x6(0x4fce bytes) 2024-09-13T07:01:49.089 INFO:teuthology.orchestra.run.smithi118.stdout:1009 writesame 0x28fd9e7 thru 0x28ffb99 data_size 0x1(0x21b3 bytes) 2024-09-13T07:01:49.103 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:49.103 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:49.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:49.103 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:49.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.103 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:434 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'4, mlcod=26'2)", 2024-09-13T07:01:49.103 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:01:49.104 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 24.566582895, 2024-09-13T07:01:49.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001931945, 2024-09-13T07:01:49.104 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.104 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.104 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:01:49.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:01:49.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535695+0000", 2024-09-13T07:01:49.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:49.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535709+0000", 2024-09-13T07:01:49.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3577999999999999e-05 2024-09-13T07:01:49.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535732+0000", 2024-09-13T07:01:49.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3085e-05 2024-09-13T07:01:49.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535742+0000", 2024-09-13T07:01:49.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.048e-05 2024-09-13T07:01:49.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535840+0000", 2024-09-13T07:01:49.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7159999999999998e-05 2024-09-13T07:01:49.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535871+0000", 2024-09-13T07:01:49.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1510999999999997e-05 2024-09-13T07:01:49.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537572+0000", 2024-09-13T07:01:49.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017008279999999999 2024-09-13T07:01:49.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537629+0000", 2024-09-13T07:01:49.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6520000000000001e-05 2024-09-13T07:01:49.108 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.108 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.108 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.108 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:435 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'5, mlcod=26'2)", 2024-09-13T07:01:49.108 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:49.108 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 24.565451189000001, 2024-09-13T07:01:49.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017126769999999999, 2024-09-13T07:01:49.108 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:49.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:49.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536835+0000", 2024-09-13T07:01:49.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4939999999999997e-06 2024-09-13T07:01:49.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536854+0000", 2024-09-13T07:01:49.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8655999999999999e-05 2024-09-13T07:01:49.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536862+0000", 2024-09-13T07:01:49.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6570000000000003e-06 2024-09-13T07:01:49.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536926+0000", 2024-09-13T07:01:49.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4037999999999999e-05 2024-09-13T07:01:49.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536958+0000", 2024-09-13T07:01:49.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1584000000000001e-05 2024-09-13T07:01:49.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538493+0000", 2024-09-13T07:01:49.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00153513 2024-09-13T07:01:49.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538541+0000", 2024-09-13T07:01:49.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7883999999999998e-05 2024-09-13T07:01:49.112 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.112 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.113 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.113 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:483 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'24, mlcod=26'21)", 2024-09-13T07:01:49.113 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:01:49.113 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 24.486972560000002, 2024-09-13T07:01:49.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018371780000000001, 2024-09-13T07:01:49.113 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.113 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.113 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615306+0000", 2024-09-13T07:01:49.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615322+0000", 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5519000000000002e-05 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615335+0000", 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3854e-05 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615345+0000", 2024-09-13T07:01:49.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4930000000000004e-06 2024-09-13T07:01:49.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615431+0000", 2024-09-13T07:01:49.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6346000000000003e-05 2024-09-13T07:01:49.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615480+0000", 2024-09-13T07:01:49.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8303999999999997e-05 2024-09-13T07:01:49.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617102+0000", 2024-09-13T07:01:49.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016225250000000001 2024-09-13T07:01:49.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617144+0000", 2024-09-13T07:01:49.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1936e-05 2024-09-13T07:01:49.117 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.117 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.117 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.118 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:518 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'38, mlcod=26'36)", 2024-09-13T07:01:49.118 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:01:49.118 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.777395821999999, 2024-09-13T07:01:49.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018201179999999999, 2024-09-13T07:01:49.118 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.118 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.118 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:01:49.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:01:49.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324883+0000", 2024-09-13T07:01:49.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324891+0000", 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1499999999999999e-06 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324924+0000", 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2601999999999997e-05 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324942+0000", 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7796999999999999e-05 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325060+0000", 2024-09-13T07:01:49.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011786900000000001 2024-09-13T07:01:49.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325123+0000", 2024-09-13T07:01:49.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3474999999999998e-05 2024-09-13T07:01:49.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326620+0000", 2024-09-13T07:01:49.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014968830000000001 2024-09-13T07:01:49.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326704+0000", 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3716999999999998e-05 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:597 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'39, mlcod=26'37)", 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.660568494, 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017959829999999999, 2024-09-13T07:01:49.122 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441709+0000", 2024-09-13T07:01:49.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:49.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441728+0000", 2024-09-13T07:01:49.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9627999999999999e-05 2024-09-13T07:01:49.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441765+0000", 2024-09-13T07:01:49.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6559e-05 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441783+0000", 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7691000000000001e-05 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441904+0000", 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121184 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441966+0000", 2024-09-13T07:01:49.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2428999999999995e-05 2024-09-13T07:01:49.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443448+0000", 2024-09-13T07:01:49.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00148197 2024-09-13T07:01:49.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443507+0000", 2024-09-13T07:01:49.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8699000000000002e-05 2024-09-13T07:01:49.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.126 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.045688200000001, 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:01:49.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1548 3.4 e27/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 27'52, mlcod=27'50)", 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.960066663999999, 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017825779999999999, 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.131 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.132 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:01:49.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:01:49.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142211+0000", 2024-09-13T07:01:49.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:49.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142232+0000", 2024-09-13T07:01:49.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0888000000000001e-05 2024-09-13T07:01:49.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142268+0000", 2024-09-13T07:01:49.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6261999999999999e-05 2024-09-13T07:01:49.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142285+0000", 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7640000000000001e-05 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142381+0000", 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5171000000000001e-05 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142450+0000", 2024-09-13T07:01:49.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9778999999999997e-05 2024-09-13T07:01:49.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143953+0000", 2024-09-13T07:01:49.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001503129 2024-09-13T07:01:49.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143995+0000", 2024-09-13T07:01:49.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1878000000000001e-05 2024-09-13T07:01:49.135 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.135 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.135 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1550 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'14, mlcod=27'12)", 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.959418953, 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002750452, 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:01:49.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142859+0000", 2024-09-13T07:01:49.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:49.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142874+0000", 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5149e-05 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142911+0000", 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6439000000000001e-05 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142926+0000", 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5079000000000001e-05 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143012+0000", 2024-09-13T07:01:49.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6230000000000006e-05 2024-09-13T07:01:49.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143063+0000", 2024-09-13T07:01:49.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1369000000000002e-05 2024-09-13T07:01:49.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145539+0000", 2024-09-13T07:01:49.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024753420000000002 2024-09-13T07:01:49.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145611+0000", 2024-09-13T07:01:49.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2392999999999996e-05 2024-09-13T07:01:49.140 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.140 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.140 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.140 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1668 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'17, mlcod=27'15)", 2024-09-13T07:01:49.140 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:49.140 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.761003491, 2024-09-13T07:01:49.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033976620000000001, 2024-09-13T07:01:49.141 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.141 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.141 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:49.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:49.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341281+0000", 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8109999999999999e-06 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341298+0000", 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6575e-05 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341308+0000", 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0369e-05 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341434+0000", 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125454 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341453+0000", 2024-09-13T07:01:49.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9159000000000001e-05 2024-09-13T07:01:49.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344638+0000", 2024-09-13T07:01:49.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031851750000000002 2024-09-13T07:01:49.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344674+0000", 2024-09-13T07:01:49.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5410999999999997e-05 2024-09-13T07:01:49.144 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.145 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.145 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.145 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1666 3.4 e28/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 28'55, mlcod=27'53)", 2024-09-13T07:01:49.145 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:01:49.145 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.760949819, 2024-09-13T07:01:49.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001805809, 2024-09-13T07:01:49.145 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.145 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.145 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341329+0000", 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341349+0000", 2024-09-13T07:01:49.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.999e-05 2024-09-13T07:01:49.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341357+0000", 2024-09-13T07:01:49.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0620000000000003e-06 2024-09-13T07:01:49.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341361+0000", 2024-09-13T07:01:49.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:01:49.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341710+0000", 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034873400000000001 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341736+0000", 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5772e-05 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343097+0000", 2024-09-13T07:01:49.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013615319999999999 2024-09-13T07:01:49.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343135+0000", 2024-09-13T07:01:49.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8111999999999997e-05 2024-09-13T07:01:49.149 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.149 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.149 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.149 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1800 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'20, mlcod=28'18)", 2024-09-13T07:01:49.149 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:01:49.149 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.558579352999999, 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001874398, 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543699+0000", 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543713+0000", 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4786e-05 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543737+0000", 2024-09-13T07:01:49.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3626e-05 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543752+0000", 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.489e-05 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543832+0000", 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0396999999999996e-05 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543994+0000", 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016129100000000001 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545471+0000", 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014771960000000001 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545575+0000", 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103732 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.153 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1882 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'23, mlcod=28'21)", 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.462946372000001, 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001767172, 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:01:49.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639332+0000", 2024-09-13T07:01:49.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:49.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639343+0000", 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1436e-05 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639361+0000", 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7915e-05 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639370+0000", 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3189999999999995e-06 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639425+0000", 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4917999999999998e-05 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639454+0000", 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9862999999999999e-05 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641059+0000", 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016048950000000001 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641100+0000", 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0883999999999997e-05 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3069 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'33, mlcod=31'31)", 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.939806656000002, 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018597010000000001, 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.158 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162471+0000", 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162520+0000", 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8466e-05 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162546+0000", 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6089999999999999e-05 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162562+0000", 2024-09-13T07:01:49.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5962000000000001e-05 2024-09-13T07:01:49.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162646+0000", 2024-09-13T07:01:49.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3985000000000003e-05 2024-09-13T07:01:49.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162691+0000", 2024-09-13T07:01:49.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5785999999999998e-05 2024-09-13T07:01:49.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164265+0000", 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015740890000000001 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164333+0000", 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7014999999999994e-05 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.163 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3121 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'36, mlcod=31'34)", 2024-09-13T07:01:49.163 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:01:49.163 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.825860269, 2024-09-13T07:01:49.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019657780000000001, 2024-09-13T07:01:49.163 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.163 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.163 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:01:49.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:01:49.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276417+0000", 2024-09-13T07:01:49.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:49.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276497+0000", 2024-09-13T07:01:49.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9729999999999997e-05 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276537+0000", 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0102999999999999e-05 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276553+0000", 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6518999999999999e-05 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276662+0000", 2024-09-13T07:01:49.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109252 2024-09-13T07:01:49.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276729+0000", 2024-09-13T07:01:49.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7056000000000003e-05 2024-09-13T07:01:49.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278270+0000", 2024-09-13T07:01:49.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015408819999999999 2024-09-13T07:01:49.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278385+0000", 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114613 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3390 3.0 e32/22 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:head v 32'45, mlcod=31'42)", 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.606771922, 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002246587, 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.167 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.168 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:01:49.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:01:49.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495507+0000", 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495530+0000", 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2592999999999999e-05 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495550+0000", 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0152999999999998e-05 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495562+0000", 2024-09-13T07:01:49.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2336e-05 2024-09-13T07:01:49.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495662+0000", 2024-09-13T07:01:49.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100317 2024-09-13T07:01:49.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495717+0000", 2024-09-13T07:01:49.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4304e-05 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497707+0000", 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019906849999999999 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497754+0000", 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6715999999999998e-05 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.172 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3399 3.4 e32/22 3:22863a52:::rbd_data.11945d91e99c.0000000000000013:head v 32'65, mlcod=31'62)", 2024-09-13T07:01:49.172 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:01:49.172 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.602835249999998, 2024-09-13T07:01:49.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028830779999999999, 2024-09-13T07:01:49.172 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.172 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.172 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:01:49.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499442+0000", 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499459+0000", 2024-09-13T07:01:49.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7042e-05 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499493+0000", 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3788000000000002e-05 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499512+0000", 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9055999999999998e-05 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499608+0000", 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6168000000000005e-05 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499658+0000", 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9469999999999999e-05 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502277+0000", 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026188639999999998 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502327+0000", 2024-09-13T07:01:49.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0578999999999998e-05 2024-09-13T07:01:49.176 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.176 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.176 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.176 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3395 3.0 e32/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 32'47, mlcod=31'43)", 2024-09-13T07:01:49.176 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:49.176 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.602737165000001, 2024-09-13T07:01:49.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027505870000000001, 2024-09-13T07:01:49.176 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.177 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.177 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:49.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:49.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499560+0000", 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8416000000000001e-05 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499576+0000", 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5602000000000001e-05 2024-09-13T07:01:49.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499587+0000", 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1112e-05 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499654+0000", 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7164999999999998e-05 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499674+0000", 2024-09-13T07:01:49.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9638000000000001e-05 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502210+0000", 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025367100000000002 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502293+0000", 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2607000000000004e-05 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.181 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:4534 3.0 e32/22 3:00392e9e:::rbd_data.11944cdc0656.000000000000000c:head v 32'51, mlcod=32'49)", 2024-09-13T07:01:49.181 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:01:49.181 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.191035978999999, 2024-09-13T07:01:49.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017597330000000001, 2024-09-13T07:01:49.181 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.181 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.181 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:01:49.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911242+0000", 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911306+0000", 2024-09-13T07:01:49.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4480999999999995e-05 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911338+0000", 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1733000000000003e-05 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911351+0000", 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3241000000000001e-05 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911455+0000", 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103723 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911495+0000", 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0683000000000003e-05 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.912933+0000", 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001437219 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.913003+0000", 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0629000000000004e-05 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11205 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'50, mlcod=38'47)", 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.477695494000001, 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031049490000000001, 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.185 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624583+0000", 2024-09-13T07:01:49.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625339+0000", 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00075563200000000001 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625358+0000", 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9916999999999999e-05 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625366+0000", 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8050000000000007e-06 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625415+0000", 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8231e-05 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625447+0000", 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2703999999999997e-05 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627652+0000", 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022050659999999999 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627689+0000", 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6612000000000001e-05 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11206 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'52, mlcod=38'47)", 2024-09-13T07:01:49.189 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.476424828000001, 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022238470000000002, 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:01:49.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625854+0000", 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625863+0000", 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5380000000000008e-06 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625883+0000", 2024-09-13T07:01:49.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9247000000000001e-05 2024-09-13T07:01:49.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625893+0000", 2024-09-13T07:01:49.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0176e-05 2024-09-13T07:01:49.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626411+0000", 2024-09-13T07:01:49.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051847600000000003 2024-09-13T07:01:49.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626431+0000", 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0199000000000001e-05 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628028+0000", 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015968110000000001 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628078+0000", 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0170000000000002e-05 2024-09-13T07:01:49.193 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.194 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.194 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.194 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.194 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.194 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:49.195 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:01:49.196 INFO:teuthology.orchestra.run.smithi118.stdout:1012 write 0xabf22c9 thru 0xabf3aeb (0x1823 bytes) 2024-09-13T07:01:49.196 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:49.190+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:49.296 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:49.294+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:49.353 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:49.354 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:49.354 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:49.354 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:49.364 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:01:49.398 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:49.394+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:49.469 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:49.470 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:49.470 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:49.470 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:49.470 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:49.479 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:01:49.498 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:49.497+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:49.591 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:49.591 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:49.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:49.592 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:49.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.592 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:441 3.b 3:d465e8df:::rbd_data.1194d22ad098.0000000000000036:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:49.592 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:49.592 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.048152199, 2024-09-13T07:01:49.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012043093, 2024-09-13T07:01:49.593 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.593 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.593 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.593 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.593 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.593 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 441 2024-09-13T07:01:49.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.594 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:49.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:49.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541122+0000", 2024-09-13T07:01:49.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:49.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541136+0000", 2024-09-13T07:01:49.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3899e-05 2024-09-13T07:01:49.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541163+0000", 2024-09-13T07:01:49.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.694e-05 2024-09-13T07:01:49.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541176+0000", 2024-09-13T07:01:49.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2802e-05 2024-09-13T07:01:49.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541257+0000", 2024-09-13T07:01:49.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1018000000000003e-05 2024-09-13T07:01:49.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541414+0000", 2024-09-13T07:01:49.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015745000000000001 2024-09-13T07:01:49.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:49.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.544450+0000", 2024-09-13T07:01:49.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030357209999999999 2024-09-13T07:01:49.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:49.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553002+0000", 2024-09-13T07:01:49.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085521020000000007 2024-09-13T07:01:49.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:49.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553022+0000", 2024-09-13T07:01:49.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9694000000000002e-05 2024-09-13T07:01:49.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553103+0000", 2024-09-13T07:01:49.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1008999999999996e-05 2024-09-13T07:01:49.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553168+0000", 2024-09-13T07:01:49.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5010000000000003e-05 2024-09-13T07:01:49.603 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.603 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.603 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.604 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1155 3.7 3:e095a6bd:::rbd_data.11943a018f34.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:49.604 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:01:49.604 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.756512936, 2024-09-13T07:01:49.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022798902999999999, 2024-09-13T07:01:49.605 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.605 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.605 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1155 2024-09-13T07:01:49.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832762+0000", 2024-09-13T07:01:49.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:49.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832776+0000", 2024-09-13T07:01:49.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3828e-05 2024-09-13T07:01:49.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832806+0000", 2024-09-13T07:01:49.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0159e-05 2024-09-13T07:01:49.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832820+0000", 2024-09-13T07:01:49.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.384e-05 2024-09-13T07:01:49.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832904+0000", 2024-09-13T07:01:49.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4506999999999996e-05 2024-09-13T07:01:49.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.833121+0000", 2024-09-13T07:01:49.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000217223 2024-09-13T07:01:49.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:49.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.839072+0000", 2024-09-13T07:01:49.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0059509849999999998 2024-09-13T07:01:49.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855489+0000", 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016416907000000001 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855503+0000", 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3658999999999999e-05 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855535+0000", 2024-09-13T07:01:49.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1696000000000003e-05 2024-09-13T07:01:49.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855563+0000", 2024-09-13T07:01:49.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8578999999999999e-05 2024-09-13T07:01:49.611 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.611 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.611 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.611 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1200 3.7 3:e095a6bd:::rbd_data.11943a018f34.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:49.611 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:01:49.612 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.610173986, 2024-09-13T07:01:49.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018736263, 2024-09-13T07:01:49.612 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.612 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.612 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.612 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.612 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.612 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1200 2024-09-13T07:01:49.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.612 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:01:49.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:01:49.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979100+0000", 2024-09-13T07:01:49.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:49.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979115+0000", 2024-09-13T07:01:49.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4823e-05 2024-09-13T07:01:49.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979149+0000", 2024-09-13T07:01:49.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3772999999999997e-05 2024-09-13T07:01:49.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979163+0000", 2024-09-13T07:01:49.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3845999999999999e-05 2024-09-13T07:01:49.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979333+0000", 2024-09-13T07:01:49.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000169622 2024-09-13T07:01:49.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979603+0000", 2024-09-13T07:01:49.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00027078999999999997 2024-09-13T07:01:49.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:49.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.986017+0000", 2024-09-13T07:01:49.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064139590000000003 2024-09-13T07:01:49.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:49.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997476+0000", 2024-09-13T07:01:49.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011458582 2024-09-13T07:01:49.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:49.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997588+0000", 2024-09-13T07:01:49.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011161 2024-09-13T07:01:49.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997732+0000", 2024-09-13T07:01:49.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000144 2024-09-13T07:01:49.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997839+0000", 2024-09-13T07:01:49.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107892 2024-09-13T07:01:49.619 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.619 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.619 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.620 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1531 3.c e27/22 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head v 27'9, mlcod=27'7)", 2024-09-13T07:01:49.620 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:01:49.620 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.514745828999999, 2024-09-13T07:01:49.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039127038000000003, 2024-09-13T07:01:49.620 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.620 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.620 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:01:49.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074530+0000", 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099434+0000", 2024-09-13T07:01:49.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.024903926999999999 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099459+0000", 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5384999999999998e-05 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099469+0000", 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.024e-05 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099566+0000", 2024-09-13T07:01:49.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6533999999999994e-05 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099600+0000", 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4007000000000002e-05 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113477+0000", 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01387741 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113658+0000", 2024-09-13T07:01:49.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000180953 2024-09-13T07:01:49.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.624 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.624 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3187 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:01:49.624 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.410269+0000", 2024-09-13T07:01:49.624 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 20.179008404000001, 2024-09-13T07:01:49.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012395160000000001, 2024-09-13T07:01:49.624 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.625 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.625 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3187 2024-09-13T07:01:49.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.625 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410269+0000", 2024-09-13T07:01:49.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410269+0000", 2024-09-13T07:01:49.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410266+0000", 2024-09-13T07:01:49.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:49.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410283+0000", 2024-09-13T07:01:49.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7527999999999999e-05 2024-09-13T07:01:49.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410317+0000", 2024-09-13T07:01:49.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3726000000000002e-05 2024-09-13T07:01:49.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410340+0000", 2024-09-13T07:01:49.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3252e-05 2024-09-13T07:01:49.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410425+0000", 2024-09-13T07:01:49.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4338999999999993e-05 2024-09-13T07:01:49.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410528+0000", 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103058 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.412544+0000", 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020164639999999999 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422499+0000", 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099549960000000007 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:49.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422521+0000", 2024-09-13T07:01:49.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1433e-05 2024-09-13T07:01:49.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422604+0000", 2024-09-13T07:01:49.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2820999999999999e-05 2024-09-13T07:01:49.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422664+0000", 2024-09-13T07:01:49.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0362999999999998e-05 2024-09-13T07:01:49.631 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.631 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.631 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.631 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3975 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:49.631 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:49.631 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.934881880999999, 2024-09-13T07:01:49.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011993248, 2024-09-13T07:01:49.632 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.632 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.632 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.632 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.632 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.633 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3975 2024-09-13T07:01:49.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.633 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:49.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:49.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:49.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654398+0000", 2024-09-13T07:01:49.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8619999999999996e-06 2024-09-13T07:01:49.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654408+0000", 2024-09-13T07:01:49.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8209999999999997e-06 2024-09-13T07:01:49.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654416+0000", 2024-09-13T07:01:49.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0180000000000001e-06 2024-09-13T07:01:49.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666002+0000", 2024-09-13T07:01:49.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011586008 2024-09-13T07:01:49.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666074+0000", 2024-09-13T07:01:49.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1500000000000003e-05 2024-09-13T07:01:49.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666388+0000", 2024-09-13T07:01:49.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00031467300000000001 2024-09-13T07:01:49.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.638 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.638 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4022 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:49.638 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:01:49.638 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.934853385, 2024-09-13T07:01:49.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014992557, 2024-09-13T07:01:49.638 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4022 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654423+0000", 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654427+0000", 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0960000000000003e-06 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654437+0000", 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7070000000000004e-06 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654445+0000", 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9049999999999997e-06 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666423+0000", 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011978223 2024-09-13T07:01:49.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666521+0000", 2024-09-13T07:01:49.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7832999999999997e-05 2024-09-13T07:01:49.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669416+0000", 2024-09-13T07:01:49.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002895393 2024-09-13T07:01:49.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.642 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4023 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.934824499000001, 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015452399, 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4023 2024-09-13T07:01:49.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.644 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:01:49.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:01:49.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654452+0000", 2024-09-13T07:01:49.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654456+0000", 2024-09-13T07:01:49.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7570000000000002e-06 2024-09-13T07:01:49.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654465+0000", 2024-09-13T07:01:49.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7140000000000005e-06 2024-09-13T07:01:49.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654486+0000", 2024-09-13T07:01:49.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0290000000000001e-05 2024-09-13T07:01:49.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669455+0000", 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014969651 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669722+0000", 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026677899999999997 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669905+0000", 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018285999999999999 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.648 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4024 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:49.648 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:49.648 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.934783699, 2024-09-13T07:01:49.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015682979, 2024-09-13T07:01:49.648 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.648 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.649 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4024 2024-09-13T07:01:49.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.649 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:49.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:49.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:49.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654497+0000", 2024-09-13T07:01:49.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4800000000000003e-06 2024-09-13T07:01:49.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654507+0000", 2024-09-13T07:01:49.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8749999999999995e-06 2024-09-13T07:01:49.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654515+0000", 2024-09-13T07:01:49.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0350000000000001e-06 2024-09-13T07:01:49.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669936+0000", 2024-09-13T07:01:49.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015420916 2024-09-13T07:01:49.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670030+0000", 2024-09-13T07:01:49.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3712e-05 2024-09-13T07:01:49.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670176+0000", 2024-09-13T07:01:49.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000146613 2024-09-13T07:01:49.657 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.657 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.657 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.657 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4025 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:49.658 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:49.658 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.934754690999998, 2024-09-13T07:01:49.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016683559000000001, 2024-09-13T07:01:49.658 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.659 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.659 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.659 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.659 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.659 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4025 2024-09-13T07:01:49.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.660 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:49.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:49.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:49.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654526+0000", 2024-09-13T07:01:49.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-13T07:01:49.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654535+0000", 2024-09-13T07:01:49.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4199999999999996e-06 2024-09-13T07:01:49.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654543+0000", 2024-09-13T07:01:49.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:01:49.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670307+0000", 2024-09-13T07:01:49.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015763482999999998 2024-09-13T07:01:49.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670399+0000", 2024-09-13T07:01:49.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2068e-05 2024-09-13T07:01:49.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671206+0000", 2024-09-13T07:01:49.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000807352 2024-09-13T07:01:49.666 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.666 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.666 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.667 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4026 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 13238272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:49.667 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:49.667 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.934726961999999, 2024-09-13T07:01:49.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016876344000000001, 2024-09-13T07:01:49.667 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.667 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.667 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.668 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.668 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.668 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4026 2024-09-13T07:01:49.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.668 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:49.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:49.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:49.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654554+0000", 2024-09-13T07:01:49.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1119999999999998e-06 2024-09-13T07:01:49.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654564+0000", 2024-09-13T07:01:49.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0327e-05 2024-09-13T07:01:49.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654571+0000", 2024-09-13T07:01:49.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3440000000000003e-06 2024-09-13T07:01:49.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671239+0000", 2024-09-13T07:01:49.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016668039999999999 2024-09-13T07:01:49.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671327+0000", 2024-09-13T07:01:49.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7818999999999994e-05 2024-09-13T07:01:49.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671427+0000", 2024-09-13T07:01:49.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9324e-05 2024-09-13T07:01:49.674 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.674 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.674 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.675 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4027 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:49.675 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:01:49.675 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.9346982, 2024-09-13T07:01:49.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017835597000000002, 2024-09-13T07:01:49.675 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.675 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.675 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.676 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.676 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.676 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4027 2024-09-13T07:01:49.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.676 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:01:49.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:01:49.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654578+0000", 2024-09-13T07:01:49.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654590+0000", 2024-09-13T07:01:49.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1595e-05 2024-09-13T07:01:49.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654601+0000", 2024-09-13T07:01:49.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0827e-05 2024-09-13T07:01:49.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654609+0000", 2024-09-13T07:01:49.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4919999999999993e-06 2024-09-13T07:01:49.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672157+0000", 2024-09-13T07:01:49.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017547770000000001 2024-09-13T07:01:49.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672234+0000", 2024-09-13T07:01:49.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7509999999999995e-05 2024-09-13T07:01:49.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672415+0000", 2024-09-13T07:01:49.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018005500000000001 2024-09-13T07:01:49.680 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.680 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.680 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.681 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11064 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:49.681 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.607882+0000", 2024-09-13T07:01:49.681 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.981395602999999, 2024-09-13T07:01:49.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012098661, 2024-09-13T07:01:49.681 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.681 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.681 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.682 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.682 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.682 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11064 2024-09-13T07:01:49.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.682 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607882+0000", 2024-09-13T07:01:49.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607882+0000", 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607881+0000", 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607885+0000", 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5530000000000003e-06 2024-09-13T07:01:49.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607896+0000", 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1073999999999999e-05 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607905+0000", 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8310000000000007e-06 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.617863+0000", 2024-09-13T07:01:49.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099584400000000007 2024-09-13T07:01:49.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.617953+0000", 2024-09-13T07:01:49.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0050999999999996e-05 2024-09-13T07:01:49.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.619980+0000", 2024-09-13T07:01:49.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002026721 2024-09-13T07:01:49.685 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11151 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608582+0000", 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.980695048999999, 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012261311, 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.686 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11151 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608582+0000", 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608582+0000", 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608581+0000", 2024-09-13T07:01:49.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608586+0000", 2024-09-13T07:01:49.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1289999999999998e-06 2024-09-13T07:01:49.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608596+0000", 2024-09-13T07:01:49.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0787e-05 2024-09-13T07:01:49.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608618+0000", 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1971e-05 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620453+0000", 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011834829999999999 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620542+0000", 2024-09-13T07:01:49.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8947000000000001e-05 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620843+0000", 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00030128199999999999 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11152 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608627+0000", 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.980650208, 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012436806, 2024-09-13T07:01:49.690 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.691 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.691 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.691 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.691 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.691 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11152 2024-09-13T07:01:49.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.691 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608627+0000", 2024-09-13T07:01:49.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608627+0000", 2024-09-13T07:01:49.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608626+0000", 2024-09-13T07:01:49.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:49.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608631+0000", 2024-09-13T07:01:49.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4109999999999998e-06 2024-09-13T07:01:49.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608642+0000", 2024-09-13T07:01:49.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1158000000000001e-05 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608650+0000", 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5110000000000003e-06 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620871+0000", 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012221019 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620956+0000", 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5223999999999996e-05 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621064+0000", 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010727599999999999 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11153 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608658+0000", 2024-09-13T07:01:49.695 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.980619093, 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012691595, 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11153 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608658+0000", 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608658+0000", 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608657+0000", 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608661+0000", 2024-09-13T07:01:49.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7629999999999998e-06 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608672+0000", 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0732e-05 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608680+0000", 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8429999999999993e-06 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621106+0000", 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01242594 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621190+0000", 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4005000000000006e-05 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621350+0000", 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000699999999999 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.699 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11173 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608785+0000", 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.980492547000001, 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012946599, 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11173 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.700 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608785+0000", 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608785+0000", 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608784+0000", 2024-09-13T07:01:49.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608788+0000", 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5070000000000002e-06 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608799+0000", 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0431000000000001e-05 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608807+0000", 2024-09-13T07:01:49.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5220000000000007e-06 2024-09-13T07:01:49.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621397+0000", 2024-09-13T07:01:49.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01258975 2024-09-13T07:01:49.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621478+0000", 2024-09-13T07:01:49.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0408000000000007e-05 2024-09-13T07:01:49.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621731+0000", 2024-09-13T07:01:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025365899999999999 2024-09-13T07:01:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11174 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:01:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.980461629000001, 2024-09-13T07:01:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01318603, 2024-09-13T07:01:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.704 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11174 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608819+0000", 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9740000000000004e-06 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608829+0000", 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0383999999999999e-05 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608838+0000", 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6719999999999993e-06 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621751+0000", 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012913017000000001 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621809+0000", 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8505000000000003e-05 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622002+0000", 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192141 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11175 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:49.708 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.980431768999999, 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013984661000000001, 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11175 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:01:49.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608849+0000", 2024-09-13T07:01:49.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8449999999999996e-06 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608859+0000", 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0335e-05 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608868+0000", 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6070000000000008e-06 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622108+0000", 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013240676 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622223+0000", 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115277 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622830+0000", 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00060653900000000001 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.712 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12393 3.b 3:d3c3e30e:::rbd_data.11945fc173dd.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.3168128960000001, 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027522214, 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12393 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:49.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272462+0000", 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272471+0000", 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8510000000000005e-06 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272486+0000", 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5409000000000001e-05 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272494+0000", 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0020000000000006e-06 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272575+0000", 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0467000000000001e-05 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272710+0000", 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135554 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.279539+0000", 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068285439999999998 2024-09-13T07:01:49.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299872+0000", 2024-09-13T07:01:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020332507 2024-09-13T07:01:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299920+0000", 2024-09-13T07:01:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8009000000000001e-05 2024-09-13T07:01:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:49.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299956+0000", 2024-09-13T07:01:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6554999999999998e-05 2024-09-13T07:01:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299986+0000", 2024-09-13T07:01:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0331000000000001e-05 2024-09-13T07:01:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:49.718 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:49.718 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:49.720 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:01:49.720 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:49.718+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:49.821 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:49.818+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:49.836 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:49.836 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:49.836 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:49.836 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:49.846 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:01:49.921 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:49.921+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:49.923 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:01:49.948 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:49.950 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:49.958 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:01:50.022 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:50.018+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:50.068 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1524 3.14 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:01:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.022416129, 2024-09-13T07:01:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061190880000000003, 2024-09-13T07:01:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:50.069 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1524 2024-09-13T07:01:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:01:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:01:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044767+0000", 2024-09-13T07:01:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052670+0000", 2024-09-13T07:01:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079022450000000004 2024-09-13T07:01:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052682+0000", 2024-09-13T07:01:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2336e-05 2024-09-13T07:01:50.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052696+0000", 2024-09-13T07:01:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-13T07:01:50.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052755+0000", 2024-09-13T07:01:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9298e-05 2024-09-13T07:01:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052947+0000", 2024-09-13T07:01:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019153900000000001 2024-09-13T07:01:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.053694+0000", 2024-09-13T07:01:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074663099999999999 2024-09-13T07:01:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.102148+0000", 2024-09-13T07:01:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048454329999999997 2024-09-13T07:01:50.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105816+0000", 2024-09-13T07:01:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036675950000000001 2024-09-13T07:01:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105921+0000", 2024-09-13T07:01:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105874 2024-09-13T07:01:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105959+0000", 2024-09-13T07:01:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7991000000000003e-05 2024-09-13T07:01:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16541 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 5758976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.984769+0000", 2024-09-13T07:01:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0824158800000001, 2024-09-13T07:01:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047534260000000002, 2024-09-13T07:01:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.078 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16541 2024-09-13T07:01:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984769+0000", 2024-09-13T07:01:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984769+0000", 2024-09-13T07:01:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984768+0000", 2024-09-13T07:01:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984773+0000", 2024-09-13T07:01:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.357e-06 2024-09-13T07:01:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984786+0000", 2024-09-13T07:01:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3943e-05 2024-09-13T07:01:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984799+0000", 2024-09-13T07:01:50.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2472e-05 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032038+0000", 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047239194999999998 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032173+0000", 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013521 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032303+0000", 2024-09-13T07:01:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012971799999999999 2024-09-13T07:01:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16542 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 5783552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.984806+0000", 2024-09-13T07:01:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.082378689, 2024-09-13T07:01:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047678218000000001, 2024-09-13T07:01:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.083 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16542 2024-09-13T07:01:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984806+0000", 2024-09-13T07:01:50.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984806+0000", 2024-09-13T07:01:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984805+0000", 2024-09-13T07:01:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:50.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984811+0000", 2024-09-13T07:01:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:01:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984827+0000", 2024-09-13T07:01:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6667000000000002e-05 2024-09-13T07:01:50.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984835+0000", 2024-09-13T07:01:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.165e-06 2024-09-13T07:01:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032327+0000", 2024-09-13T07:01:50.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047492151000000003 2024-09-13T07:01:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032407+0000", 2024-09-13T07:01:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9782000000000003e-05 2024-09-13T07:01:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032484+0000", 2024-09-13T07:01:50.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.771e-05 2024-09-13T07:01:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16543 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.984842+0000", 2024-09-13T07:01:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0823431610000001, 2024-09-13T07:01:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047953233999999997, 2024-09-13T07:01:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16543 2024-09-13T07:01:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984842+0000", 2024-09-13T07:01:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984842+0000", 2024-09-13T07:01:50.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984841+0000", 2024-09-13T07:01:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984845+0000", 2024-09-13T07:01:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7689999999999998e-06 2024-09-13T07:01:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984855+0000", 2024-09-13T07:01:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0484e-05 2024-09-13T07:01:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984869+0000", 2024-09-13T07:01:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4004e-05 2024-09-13T07:01:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032542+0000", 2024-09-13T07:01:50.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047672288 2024-09-13T07:01:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032667+0000", 2024-09-13T07:01:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012577000000000001 2024-09-13T07:01:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032795+0000", 2024-09-13T07:01:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000127458 2024-09-13T07:01:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.093 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16556 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6127616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988454+0000", 2024-09-13T07:01:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.078730628, 2024-09-13T07:01:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047410069999999999, 2024-09-13T07:01:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16556 2024-09-13T07:01:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988454+0000", 2024-09-13T07:01:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988454+0000", 2024-09-13T07:01:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988454+0000", 2024-09-13T07:01:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988455+0000", 2024-09-13T07:01:50.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.491e-06 2024-09-13T07:01:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988458+0000", 2024-09-13T07:01:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0419999999999998e-06 2024-09-13T07:01:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988461+0000", 2024-09-13T07:01:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0070000000000002e-06 2024-09-13T07:01:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.035709+0000", 2024-09-13T07:01:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047248050999999999 2024-09-13T07:01:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.035786+0000", 2024-09-13T07:01:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6477999999999995e-05 2024-09-13T07:01:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.035864+0000", 2024-09-13T07:01:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8246999999999998e-05 2024-09-13T07:01:50.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16557 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6152192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988466+0000", 2024-09-13T07:01:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.078718788, 2024-09-13T07:01:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047718719999999999, 2024-09-13T07:01:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16557 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988466+0000", 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988466+0000", 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988466+0000", 2024-09-13T07:01:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988467+0000", 2024-09-13T07:01:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.48e-06 2024-09-13T07:01:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988476+0000", 2024-09-13T07:01:50.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4549999999999995e-06 2024-09-13T07:01:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988484+0000", 2024-09-13T07:01:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6649999999999992e-06 2024-09-13T07:01:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.035920+0000", 2024-09-13T07:01:50.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047435828999999999 2024-09-13T07:01:50.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036042+0000", 2024-09-13T07:01:50.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012243000000000001 2024-09-13T07:01:50.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036185+0000", 2024-09-13T07:01:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014214100000000001 2024-09-13T07:01:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16558 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6176768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988487+0000", 2024-09-13T07:01:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.078697539, 2024-09-13T07:01:50.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047874958000000002, 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16558 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988487+0000", 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988487+0000", 2024-09-13T07:01:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988487+0000", 2024-09-13T07:01:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988488+0000", 2024-09-13T07:01:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.525e-06 2024-09-13T07:01:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988493+0000", 2024-09-13T07:01:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5719999999999996e-06 2024-09-13T07:01:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988498+0000", 2024-09-13T07:01:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:01:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036208+0000", 2024-09-13T07:01:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047710474000000003 2024-09-13T07:01:50.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036285+0000", 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6798000000000006e-05 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036362+0000", 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7107999999999994e-05 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16559 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988549+0000", 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.078635727, 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048124801000000002, 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16559 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988549+0000", 2024-09-13T07:01:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988549+0000", 2024-09-13T07:01:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988548+0000", 2024-09-13T07:01:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988552+0000", 2024-09-13T07:01:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5980000000000001e-06 2024-09-13T07:01:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988564+0000", 2024-09-13T07:01:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.181e-05 2024-09-13T07:01:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988573+0000", 2024-09-13T07:01:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4069999999999999e-06 2024-09-13T07:01:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036419+0000", 2024-09-13T07:01:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047845461999999998 2024-09-13T07:01:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036543+0000", 2024-09-13T07:01:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012405000000000001 2024-09-13T07:01:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036674+0000", 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013107599999999999 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16560 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6225920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988579+0000", 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.078605979, 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048274951000000003, 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16560 2024-09-13T07:01:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988579+0000", 2024-09-13T07:01:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988579+0000", 2024-09-13T07:01:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988578+0000", 2024-09-13T07:01:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988581+0000", 2024-09-13T07:01:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0589999999999998e-06 2024-09-13T07:01:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988593+0000", 2024-09-13T07:01:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2058e-05 2024-09-13T07:01:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988605+0000", 2024-09-13T07:01:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1206000000000001e-05 2024-09-13T07:01:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036697+0000", 2024-09-13T07:01:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048092477000000002 2024-09-13T07:01:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036778+0000", 2024-09-13T07:01:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0766000000000006e-05 2024-09-13T07:01:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036854+0000", 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5852999999999994e-05 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16561 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6250496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988610+0000", 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.078574806, 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048548085999999997, 2024-09-13T07:01:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16561 2024-09-13T07:01:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988610+0000", 2024-09-13T07:01:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988610+0000", 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988610+0000", 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988614+0000", 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0949999999999998e-06 2024-09-13T07:01:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988625+0000", 2024-09-13T07:01:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1347e-05 2024-09-13T07:01:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988631+0000", 2024-09-13T07:01:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0639999999999999e-06 2024-09-13T07:01:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036910+0000", 2024-09-13T07:01:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048278910000000001 2024-09-13T07:01:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.037030+0000", 2024-09-13T07:01:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119912 2024-09-13T07:01:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.037158+0000", 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128121 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16562 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6275072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988636+0000", 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0785490360000001, 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048699601000000002, 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16562 2024-09-13T07:01:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988636+0000", 2024-09-13T07:01:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988636+0000", 2024-09-13T07:01:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988635+0000", 2024-09-13T07:01:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988638+0000", 2024-09-13T07:01:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9440000000000001e-06 2024-09-13T07:01:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988652+0000", 2024-09-13T07:01:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3638000000000001e-05 2024-09-13T07:01:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988662+0000", 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0447e-05 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.037180+0000", 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048517376000000001 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.037261+0000", 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1143000000000006e-05 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.037335+0000", 2024-09-13T07:01:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4466000000000002e-05 2024-09-13T07:01:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16576 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992577+0000", 2024-09-13T07:01:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0746077940000001, 2024-09-13T07:01:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047360526, 2024-09-13T07:01:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16576 2024-09-13T07:01:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992577+0000", 2024-09-13T07:01:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992577+0000", 2024-09-13T07:01:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992576+0000", 2024-09-13T07:01:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992585+0000", 2024-09-13T07:01:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3999999999999992e-06 2024-09-13T07:01:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992591+0000", 2024-09-13T07:01:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.409e-06 2024-09-13T07:01:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992601+0000", 2024-09-13T07:01:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8649999999999996e-06 2024-09-13T07:01:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.039790+0000", 2024-09-13T07:01:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047188859 2024-09-13T07:01:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.039865+0000", 2024-09-13T07:01:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5117999999999995e-05 2024-09-13T07:01:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.039937+0000", 2024-09-13T07:01:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2298000000000005e-05 2024-09-13T07:01:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16577 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6643712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992607+0000", 2024-09-13T07:01:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.074578069, 2024-09-13T07:01:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047460953, 2024-09-13T07:01:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16577 2024-09-13T07:01:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992607+0000", 2024-09-13T07:01:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992607+0000", 2024-09-13T07:01:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992606+0000", 2024-09-13T07:01:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992609+0000", 2024-09-13T07:01:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0249999999999998e-06 2024-09-13T07:01:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992621+0000", 2024-09-13T07:01:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1379999999999999e-05 2024-09-13T07:01:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992626+0000", 2024-09-13T07:01:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2240000000000003e-06 2024-09-13T07:01:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.039968+0000", 2024-09-13T07:01:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047342238000000002 2024-09-13T07:01:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040019+0000", 2024-09-13T07:01:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0948000000000001e-05 2024-09-13T07:01:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040068+0000", 2024-09-13T07:01:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8628000000000003e-05 2024-09-13T07:01:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16578 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992631+0000", 2024-09-13T07:01:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0745540950000001, 2024-09-13T07:01:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047631675999999998, 2024-09-13T07:01:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16578 2024-09-13T07:01:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992631+0000", 2024-09-13T07:01:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992631+0000", 2024-09-13T07:01:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992630+0000", 2024-09-13T07:01:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992633+0000", 2024-09-13T07:01:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:01:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992645+0000", 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2169e-05 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992654+0000", 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8370000000000003e-06 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040107+0000", 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047452544999999999 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040187+0000", 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0382000000000004e-05 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040262+0000", 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5244e-05 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16579 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6692864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992659+0000", 2024-09-13T07:01:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.074525255, 2024-09-13T07:01:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047731126999999998, 2024-09-13T07:01:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16579 2024-09-13T07:01:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992659+0000", 2024-09-13T07:01:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992659+0000", 2024-09-13T07:01:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992659+0000", 2024-09-13T07:01:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992662+0000", 2024-09-13T07:01:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0800000000000002e-06 2024-09-13T07:01:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992673+0000", 2024-09-13T07:01:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1338999999999999e-05 2024-09-13T07:01:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992682+0000", 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5609999999999999e-06 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040294+0000", 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047611852000000003 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040345+0000", 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1001999999999997e-05 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040391+0000", 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5719000000000001e-05 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16580 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992728+0000", 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.074457065, 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047850404999999999, 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16580 2024-09-13T07:01:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992728+0000", 2024-09-13T07:01:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992728+0000", 2024-09-13T07:01:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992727+0000", 2024-09-13T07:01:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992731+0000", 2024-09-13T07:01:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3469999999999999e-06 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992740+0000", 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3279999999999995e-06 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992752+0000", 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2272e-05 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040430+0000", 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047678164000000002 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040506+0000", 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5539999999999998e-05 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040578+0000", 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2083999999999996e-05 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16581 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6742016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992756+0000", 2024-09-13T07:01:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0744285419999999, 2024-09-13T07:01:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047949586000000002, 2024-09-13T07:01:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16581 2024-09-13T07:01:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992756+0000", 2024-09-13T07:01:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992756+0000", 2024-09-13T07:01:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992756+0000", 2024-09-13T07:01:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992758+0000", 2024-09-13T07:01:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.057e-06 2024-09-13T07:01:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992770+0000", 2024-09-13T07:01:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1936000000000001e-05 2024-09-13T07:01:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992777+0000", 2024-09-13T07:01:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6479999999999997e-06 2024-09-13T07:01:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040607+0000", 2024-09-13T07:01:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047830841999999998 2024-09-13T07:01:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040657+0000", 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9181000000000002e-05 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040706+0000", 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9190000000000002e-05 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16582 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6766592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992780+0000", 2024-09-13T07:01:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.074404718, 2024-09-13T07:01:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048118462000000001, 2024-09-13T07:01:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16582 2024-09-13T07:01:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992780+0000", 2024-09-13T07:01:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992780+0000", 2024-09-13T07:01:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992780+0000", 2024-09-13T07:01:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992782+0000", 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1059999999999998e-06 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992785+0000", 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2990000000000001e-06 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992793+0000", 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3450000000000006e-06 2024-09-13T07:01:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040746+0000", 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047952616000000003 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040825+0000", 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9340999999999998e-05 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040898+0000", 2024-09-13T07:01:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3195000000000006e-05 2024-09-13T07:01:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16583 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6791168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992796+0000", 2024-09-13T07:01:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0743884859999999, 2024-09-13T07:01:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048227689999999997, 2024-09-13T07:01:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16583 2024-09-13T07:01:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992796+0000", 2024-09-13T07:01:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992796+0000", 2024-09-13T07:01:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992796+0000", 2024-09-13T07:01:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992798+0000", 2024-09-13T07:01:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.782e-06 2024-09-13T07:01:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992801+0000", 2024-09-13T07:01:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1030000000000002e-06 2024-09-13T07:01:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992804+0000", 2024-09-13T07:01:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9469999999999999e-06 2024-09-13T07:01:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040929+0000", 2024-09-13T07:01:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048125384 2024-09-13T07:01:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040979+0000", 2024-09-13T07:01:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9625e-05 2024-09-13T07:01:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.041024+0000", 2024-09-13T07:01:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5105999999999998e-05 2024-09-13T07:01:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16584 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992806+0000", 2024-09-13T07:01:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0743785100000001, 2024-09-13T07:01:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048406193, 2024-09-13T07:01:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16584 2024-09-13T07:01:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992806+0000", 2024-09-13T07:01:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992806+0000", 2024-09-13T07:01:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992806+0000", 2024-09-13T07:01:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:01:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992807+0000", 2024-09-13T07:01:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.514e-06 2024-09-13T07:01:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992811+0000", 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0359999999999998e-06 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992813+0000", 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.914e-06 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.041061+0000", 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048247440000000003 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.041139+0000", 2024-09-13T07:01:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7723000000000001e-05 2024-09-13T07:01:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.041212+0000", 2024-09-13T07:01:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3792000000000001e-05 2024-09-13T07:01:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:50.183 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:50.184 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:50.186 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:01:50.187 INFO:teuthology.orchestra.run.smithi118.stdout:1013 writesame 0x7665250 thru 0x76653cb data_size 0xa(0x17c bytes) 2024-09-13T07:01:50.187 INFO:teuthology.orchestra.run.smithi118.stdout:1014 write 0x3ab416b thru 0x3ab4d71 (0xc07 bytes) 2024-09-13T07:01:50.187 INFO:teuthology.orchestra.run.smithi118.stdout:1015 read 0x4147def thru 0x41498ba (0x1acc bytes) 2024-09-13T07:01:50.187 INFO:teuthology.orchestra.run.smithi118.stdout:1016 read 0xaa1a1c4 thru 0xaa1e378 (0x41b5 bytes) 2024-09-13T07:01:50.188 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:50.188 INFO:teuthology.orchestra.run.smithi118.stdout:1020 write 0x862d8f9 thru 0x863d473 (0xfb7b bytes) 2024-09-13T07:01:50.188 INFO:teuthology.orchestra.run.smithi118.stdout:1021 read 0x889ed2d thru 0x88adfa2 (0xf276 bytes) 2024-09-13T07:01:50.188 INFO:teuthology.orchestra.run.smithi118.stdout:1022 punch from 0x803f7f1 to 0x8042601, (0x2e10 bytes) 2024-09-13T07:01:50.188 INFO:teuthology.orchestra.run.smithi118.stdout:1023 write 0x9dc0176 thru 0x9dc3768 (0x35f3 bytes) 2024-09-13T07:01:50.188 INFO:teuthology.orchestra.run.smithi118.stdout:1024 read 0x569c8c thru 0x56ff9f (0x6314 bytes) 2024-09-13T07:01:50.188 INFO:teuthology.orchestra.run.smithi118.stdout:1025 read 0x68e8eec thru 0x68e9a87 (0xb9c bytes) 2024-09-13T07:01:50.188 INFO:teuthology.orchestra.run.smithi118.stdout:1026 read 0x228c297 thru 0x228d949 (0x16b3 bytes) 2024-09-13T07:01:50.189 INFO:teuthology.orchestra.run.smithi118.stdout:1028 writesame 0x5b85983 thru 0x5b87797 data_size 0x11(0x1e15 bytes) 2024-09-13T07:01:50.189 INFO:teuthology.orchestra.run.smithi118.stdout:1029 punch from 0x9513d to 0x9dae2, (0x89a5 bytes) 2024-09-13T07:01:50.189 INFO:teuthology.orchestra.run.smithi118.stdout:1031 write 0xbde83f thru 0xbed6c5 (0xee87 bytes) 2024-09-13T07:01:50.189 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:50.189 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:50.189 INFO:teuthology.orchestra.run.smithi118.stdout:1034 write 0x1b753f5 thru 0x1b78afb (0x3707 bytes) 2024-09-13T07:01:50.189 INFO:teuthology.orchestra.run.smithi118.stdout:1035 trunc from 0xabf3aec to 0xcf3fff9 2024-09-13T07:01:50.189 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:1036 write 0xb964e67 thru 0xb965a58 (0xbf2 bytes) 2024-09-13T07:01:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:1037 punch from 0x2f88fcc to 0x2f93a84, (0xaab8 bytes) 2024-09-13T07:01:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:1039 punch from 0x8e07e3c to 0x8e104a3, (0x8667 bytes) 2024-09-13T07:01:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:1040 read 0x4e31360 thru 0x4e3974a (0x83eb bytes) 2024-09-13T07:01:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:1042 read 0x99a3538 thru 0x99aa56f (0x7038 bytes) 2024-09-13T07:01:50.190 INFO:teuthology.orchestra.run.smithi118.stdout:1043 writesame 0xcbaa15 thru 0xcca1e7 data_size 0x1(0xf7d3 bytes) 2024-09-13T07:01:50.191 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:50.191 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:50.191 INFO:teuthology.orchestra.run.smithi118.stdout:1045 writesame 0xe3491f3 thru 0xe34aee9 data_size 0x1(0x1cf7 bytes) 2024-09-13T07:01:50.192 INFO:teuthology.orchestra.run.smithi118.stdout:1046 trunc from 0xe34aeea to 0xe2142f3 2024-09-13T07:01:50.192 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:50.181+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:50.192 INFO:teuthology.orchestra.run.smithi118.stdout:1048 write 0xce86c2b thru 0xce94f3d (0xe313 bytes) 2024-09-13T07:01:50.193 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:50.193 INFO:teuthology.orchestra.run.smithi118.stdout:1050 punch from 0x83e5c41 to 0x83e7ed7, (0x2296 bytes) 2024-09-13T07:01:50.194 INFO:teuthology.orchestra.run.smithi118.stdout:1051 punch from 0xe09b5b1 to 0xe09eecc, (0x391b bytes) 2024-09-13T07:01:50.194 INFO:teuthology.orchestra.run.smithi118.stdout:1052 writesame 0x8baf929 thru 0x8bbb3e3 data_size 0x1(0xbabb bytes) 2024-09-13T07:01:50.217 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:50.220 INFO:teuthology.orchestra.run.smithi118.stdout:1053 read 0xbca648b thru 0xbcb0b28 (0xa69e bytes) 2024-09-13T07:01:50.220 INFO:teuthology.orchestra.run.smithi118.stdout:1054 read 0xac86474 thru 0xac91227 (0xadb4 bytes) 2024-09-13T07:01:50.221 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:50.221 INFO:teuthology.orchestra.run.smithi118.stdout:1057 punch from 0x1bf0c05 to 0x1bfd418, (0xc813 bytes) 2024-09-13T07:01:50.221 INFO:teuthology.orchestra.run.smithi118.stdout:1059 punch from 0x878729a to 0x8790df5, (0x9b5b bytes) 2024-09-13T07:01:50.221 INFO:teuthology.orchestra.run.smithi118.stdout:1060 read 0x4e07 thru 0x8dd5 (0x3fcf bytes) 2024-09-13T07:01:50.222 INFO:teuthology.orchestra.run.smithi118.stdout:1061 read 0x24ec0b4 thru 0x24f6a5c (0xa9a9 bytes) 2024-09-13T07:01:50.223 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:50.223 INFO:teuthology.orchestra.run.smithi118.stdout:1063 write 0xb7b45ee thru 0xb7c0650 (0xc063 bytes) 2024-09-13T07:01:50.230 INFO:teuthology.orchestra.run.smithi118.stdout:1066 punch from 0x7c8e4ef to 0x7c95069, (0x6b7a bytes) 2024-09-13T07:01:50.231 INFO:teuthology.orchestra.run.smithi118.stdout:1067 write 0x31abf7c thru 0x31b8ad2 (0xcb57 bytes) 2024-09-13T07:01:50.239 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:50.239 INFO:teuthology.orchestra.run.smithi118.stdout:1069 write 0x48493ce thru 0x4857880 (0xe4b3 bytes) 2024-09-13T07:01:50.239 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:50.247 INFO:teuthology.orchestra.run.smithi118.stdout:1071 punch from 0xd803699 to 0xd809972, (0x62d9 bytes) 2024-09-13T07:01:50.248 INFO:teuthology.orchestra.run.smithi118.stdout:1072 punch from 0x6c74d1a to 0x6c7fa0e, (0xacf4 bytes) 2024-09-13T07:01:50.248 INFO:teuthology.orchestra.run.smithi118.stdout:1074 read 0x836ac1e thru 0x8371440 (0x6823 bytes) 2024-09-13T07:01:50.248 INFO:teuthology.orchestra.run.smithi118.stdout:1075 read 0xbc21245 thru 0xbc24f7b (0x3d37 bytes) 2024-09-13T07:01:50.248 INFO:teuthology.orchestra.run.smithi118.stdout:1076 write 0x9c54837 thru 0x9c61a64 (0xd22e bytes) 2024-09-13T07:01:50.251 INFO:teuthology.orchestra.run.smithi118.stdout:1077 punch from 0x70213f7 to 0x70283e7, (0x6ff0 bytes) 2024-09-13T07:01:50.251 INFO:teuthology.orchestra.run.smithi118.stdout:1078 punch from 0x1654552 to 0x16568a0, (0x234e bytes) 2024-09-13T07:01:50.251 INFO:teuthology.orchestra.run.smithi118.stdout:1079 punch from 0xa2174d6 to 0xa2183bf, (0xee9 bytes) 2024-09-13T07:01:50.255 INFO:teuthology.orchestra.run.smithi118.stdout:1080 trunc from 0xe2142f3 to 0xb0bca75 2024-09-13T07:01:50.263 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:50.265 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:50.265 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:50.265 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:50.265 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:50.265 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:50.265 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:50.265 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:50.265 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:50.283 INFO:teuthology.orchestra.run.smithi118.stdout:1081 trunc from 0xb0bca75 to 0x53c98f8 2024-09-13T07:01:50.285 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:50.285+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:50.311 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:50.311 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:50.312 INFO:teuthology.orchestra.run.smithi118.stdout:1082 trunc from 0x53c98f8 to 0xc174858 2024-09-13T07:01:50.321 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:01:50.322 INFO:teuthology.orchestra.run.smithi118.stdout:1083 read 0xc0913b7 thru 0xc093725 (0x236f bytes) 2024-09-13T07:01:50.332 INFO:teuthology.orchestra.run.smithi118.stdout:1084 writesame 0xd472905 thru 0xd4761fe data_size 0x42(0x38fa bytes) 2024-09-13T07:01:50.332 INFO:teuthology.orchestra.run.smithi118.stdout:1085 read 0x1093e6 thru 0x1170f7 (0xdd12 bytes) 2024-09-13T07:01:50.333 INFO:teuthology.orchestra.run.smithi118.stdout:1086 write 0xcf66be2 thru 0xcf74efd (0xe31c bytes) 2024-09-13T07:01:50.342 INFO:teuthology.orchestra.run.smithi118.stdout:1088 read 0x6fcec5 thru 0x6ff8b1 (0x29ed bytes) 2024-09-13T07:01:50.343 INFO:teuthology.orchestra.run.smithi118.stdout:1089 write 0xba5c9fc thru 0xba5ecdf (0x22e4 bytes) 2024-09-13T07:01:50.349 INFO:teuthology.orchestra.run.smithi118.stdout:1090 write 0x9d931ab thru 0x9d954c4 (0x231a bytes) 2024-09-13T07:01:50.354 INFO:teuthology.orchestra.run.smithi118.stdout:1092 write 0x5ea0244 thru 0x5ea30c4 (0x2e81 bytes) 2024-09-13T07:01:50.360 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:50.360 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:50.360 INFO:teuthology.orchestra.run.smithi118.stdout:1096 read 0x1e8394 thru 0x1f0e46 (0x8ab3 bytes) 2024-09-13T07:01:50.361 INFO:teuthology.orchestra.run.smithi118.stdout:1097 write 0x9f7bee7 thru 0x9f7e24e (0x2368 bytes) 2024-09-13T07:01:50.376 INFO:teuthology.orchestra.run.smithi118.stdout:1098 write 0xe48181c thru 0xe48e8e4 (0xd0c9 bytes) 2024-09-13T07:01:50.386 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:50.382+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:50.389 INFO:teuthology.orchestra.run.smithi118.stdout:1101 trunc from 0xe48e8e5 to 0xb4a7891 2024-09-13T07:01:50.424 INFO:teuthology.orchestra.run.smithi118.stdout:1105 write 0xcd23156 thru 0xcd2acdd (0x7b88 bytes) 2024-09-13T07:01:50.433 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:50.433 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:50.433 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:50.434 INFO:teuthology.orchestra.run.smithi118.stdout:1107 write 0xc1f5860 thru 0xc20320b (0xd9ac bytes) 2024-09-13T07:01:50.441 INFO:teuthology.orchestra.run.smithi118.stdout:1109 read 0xc70fac8 thru 0xc71e427 (0xe960 bytes) 2024-09-13T07:01:50.441 INFO:teuthology.orchestra.run.smithi118.stdout:1110 write 0x39d2984 thru 0x39dc924 (0x9fa1 bytes) 2024-09-13T07:01:50.444 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:01:50.446 INFO:teuthology.orchestra.run.smithi118.stdout:1111 writesame 0x5795bda thru 0x57a56f4 data_size 0x1(0xfb1b bytes) 2024-09-13T07:01:50.470 INFO:teuthology.orchestra.run.smithi118.stdout:1113 read 0xc1d6d05 thru 0xc1e3670 (0xc96c bytes) 2024-09-13T07:01:50.471 INFO:teuthology.orchestra.run.smithi118.stdout:1114 read 0x11080d7 thru 0x110d91c (0x5846 bytes) 2024-09-13T07:01:50.472 INFO:teuthology.orchestra.run.smithi118.stdout:1116 writesame 0x7b5bc2c thru 0x7b66a5d data_size 0xb(0xae32 bytes) 2024-09-13T07:01:50.486 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:50.482+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:50.496 INFO:teuthology.orchestra.run.smithi118.stdout:1117 writesame 0xec7da35 thru 0xec8744b data_size 0x1b(0x9a17 bytes) 2024-09-13T07:01:50.496 INFO:teuthology.orchestra.run.smithi118.stdout:1118 write 0xb7fd478 thru 0xb80005d (0x2be6 bytes) 2024-09-13T07:01:50.502 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:01:50.503 INFO:teuthology.orchestra.run.smithi118.stdout:1120 clone 14 order 23 su 524288 sc 15 2024-09-13T07:01:50.545 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:50.546 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:50.555 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:01:50.587 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:50.585+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:50.688 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:50.685+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:50.707 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:50.708 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:50.717 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:01:50.789 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:50.786+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:50.822 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:50.822 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:50.832 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:01:50.890 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:50.889+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:50.938 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:50.938 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:50.949 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:01:50.990 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:50.989+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:51.052 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:51.052 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:51.062 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:01:51.092 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:51.090+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:51.099 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:01:51.174 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:51.174 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:51.184 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:01:51.220 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone13 from 0xec8744c (overlap 0x1bb8d21) to 0x18f3237 2024-09-13T07:01:51.228 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:51.226+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:51.294 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:51.293+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:51.332 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:51.332 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:51.342 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:01:51.349 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-13T07:01:51.394 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:51.390+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:51.461 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:51.462 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:51.471 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:01:51.495 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:51.490+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:51.497 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:01:51.498 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:01:51.498 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:01:51.596 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:51.594+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:51.626 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:51.626 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:51.637 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:01:51.697 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:51.694+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:51.735 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:51.735 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:51.744 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:01:51.757 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:51.758 INFO:teuthology.orchestra.run.smithi118.stdout:1122 trunc from 0xec8744c to 0xdee5a26 2024-09-13T07:01:51.797 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:51.797+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:51.807 INFO:teuthology.orchestra.run.smithi118.stdout:1123 read 0x311a343 thru 0x311d877 (0x3535 bytes) 2024-09-13T07:01:51.808 INFO:teuthology.orchestra.run.smithi118.stdout:1124 writesame 0x55ef920 thru 0x55f64ab data_size 0x2(0x6b8c bytes) 2024-09-13T07:01:51.839 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:51.839 INFO:teuthology.orchestra.run.smithi118.stdout:1125 read 0x636b91a thru 0x6374199 (0x8880 bytes) 2024-09-13T07:01:51.842 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:01:51.851 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:51.851 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:51.858 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-13T07:01:51.091503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":516424929,"num_objects":419,"num_object_clones":93,"num_object_copies":838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":419,"num_whiteouts":152,"num_read":27588,"num_read_kb":94191,"num_write":3419,"num_write_kb":71445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12139577,"num_keys_recovered":59,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2866,"ondisk_log_size":2866,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":237792,"kb_used_data":46964,"kb_used_omap":9,"kb_used_meta":190710,"kb_avail":565993248,"statfs":{"total":579820584960,"available":579577085952,"internally_reserved":0,"allocated":48091136,"data_stored":50297704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9556,"internal_metadata":195287724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42771936,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":4350,"num_read_kb":5465,"num_write":551,"num_write_kb":2680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":9797872,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.613507"},"pg_stats":[{"pgid":"3.1f","version":"52'61","reported_seq":608,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.399930+0000","last_change":"2024-09-13T07:01:41.516000+0000","last_active":"2024-09-13T07:01:50.399930+0000","last_peered":"2024-09-13T07:01:50.399930+0000","last_clean":"2024-09-13T07:01:50.399930+0000","last_became_active":"2024-09-13T07:01:41.437584+0000","last_became_peered":"2024-09-13T07:01:41.437584+0000","last_unstale":"2024-09-13T07:01:50.399930+0000","last_undegraded":"2024-09-13T07:01:50.399930+0000","last_fullsized":"2024-09-13T07:01:50.399930+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:47.432372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013321120000000001,"stat_sum":{"num_bytes":32473406,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":436,"num_read_kb":2363,"num_write":54,"num_write_kb":834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":834141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"52'43","reported_seq":367,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.291245+0000","last_change":"2024-09-13T07:01:21.291845+0000","last_active":"2024-09-13T07:01:50.291245+0000","last_peered":"2024-09-13T07:01:50.291245+0000","last_clean":"2024-09-13T07:01:50.291245+0000","last_became_active":"2024-09-13T07:01:19.281907+0000","last_became_peered":"2024-09-13T07:01:19.281907+0000","last_unstale":"2024-09-13T07:01:50.291245+0000","last_undegraded":"2024-09-13T07:01:50.291245+0000","last_fullsized":"2024-09-13T07:01:50.291245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:43.514663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001950586,"stat_sum":{"num_bytes":17152381,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":6,"num_read":311,"num_read_kb":557,"num_write":39,"num_write_kb":738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"52'26","reported_seq":176,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.343306+0000","last_change":"2024-09-13T07:01:21.293288+0000","last_active":"2024-09-13T07:01:50.343306+0000","last_peered":"2024-09-13T07:01:50.343306+0000","last_clean":"2024-09-13T07:01:50.343306+0000","last_became_active":"2024-09-13T07:01:19.276445+0000","last_became_peered":"2024-09-13T07:01:19.276445+0000","last_unstale":"2024-09-13T07:01:50.343306+0000","last_undegraded":"2024-09-13T07:01:50.343306+0000","last_fullsized":"2024-09-13T07:01:50.343306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:16:37.731881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076373900000000004,"stat_sum":{"num_bytes":11691762,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":123,"num_read_kb":117,"num_write":22,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"52'58","reported_seq":644,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.471762+0000","last_change":"2024-09-13T07:01:21.292234+0000","last_active":"2024-09-13T07:01:50.471762+0000","last_peered":"2024-09-13T07:01:50.471762+0000","last_clean":"2024-09-13T07:01:50.471762+0000","last_became_active":"2024-09-13T07:01:19.276459+0000","last_became_peered":"2024-09-13T07:01:19.276459+0000","last_unstale":"2024-09-13T07:01:50.471762+0000","last_undegraded":"2024-09-13T07:01:50.471762+0000","last_fullsized":"2024-09-13T07:01:50.471762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:50:50.173262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050866599999999996,"stat_sum":{"num_bytes":10164823,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":1185,"num_read_kb":2355,"num_write":52,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"47'40","reported_seq":377,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.083537+0000","last_change":"2024-09-13T07:01:34.715025+0000","last_active":"2024-09-13T07:01:49.083537+0000","last_peered":"2024-09-13T07:01:49.083537+0000","last_clean":"2024-09-13T07:01:49.083537+0000","last_became_active":"2024-09-13T07:01:34.676996+0000","last_became_peered":"2024-09-13T07:01:34.676996+0000","last_unstale":"2024-09-13T07:01:49.083537+0000","last_undegraded":"2024-09-13T07:01:49.083537+0000","last_fullsized":"2024-09-13T07:01:49.083537+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:26:51.849086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083111000000000003,"stat_sum":{"num_bytes":13357036,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":303,"num_read_kb":956,"num_write":33,"num_write_kb":263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":67840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"52'93","reported_seq":636,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.423558+0000","last_change":"2024-09-13T07:01:21.297776+0000","last_active":"2024-09-13T07:01:50.423558+0000","last_peered":"2024-09-13T07:01:50.423558+0000","last_clean":"2024-09-13T07:01:50.423558+0000","last_became_active":"2024-09-13T07:01:19.275470+0000","last_became_peered":"2024-09-13T07:01:19.275470+0000","last_unstale":"2024-09-13T07:01:50.423558+0000","last_undegraded":"2024-09-13T07:01:50.423558+0000","last_fullsized":"2024-09-13T07:01:50.423558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:15.860822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001799911,"stat_sum":{"num_bytes":11910298,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":800,"num_read_kb":799,"num_write":122,"num_write_kb":430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"52'121","reported_seq":309,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.473959+0000","last_change":"2024-09-13T07:01:21.300421+0000","last_active":"2024-09-13T07:01:50.473959+0000","last_peered":"2024-09-13T07:01:50.473959+0000","last_clean":"2024-09-13T07:01:50.473959+0000","last_became_active":"2024-09-13T07:01:19.280734+0000","last_became_peered":"2024-09-13T07:01:19.280734+0000","last_unstale":"2024-09-13T07:01:50.473959+0000","last_undegraded":"2024-09-13T07:01:50.473959+0000","last_fullsized":"2024-09-13T07:01:50.473959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:02.332240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055727200000000004,"stat_sum":{"num_bytes":15860753,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":515,"num_read_kb":1350,"num_write":164,"num_write_kb":1198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"50'86","reported_seq":880,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.072856+0000","last_change":"2024-09-13T07:01:35.443178+0000","last_active":"2024-09-13T07:01:49.072856+0000","last_peered":"2024-09-13T07:01:49.072856+0000","last_clean":"2024-09-13T07:01:49.072856+0000","last_became_active":"2024-09-13T07:01:35.423891+0000","last_became_peered":"2024-09-13T07:01:35.423891+0000","last_unstale":"2024-09-13T07:01:49.072856+0000","last_undegraded":"2024-09-13T07:01:49.072856+0000","last_fullsized":"2024-09-13T07:01:49.072856+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:20:08.704987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082396300000000004,"stat_sum":{"num_bytes":407719,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":6,"num_read":2325,"num_read_kb":1995,"num_write":73,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":14503,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"52'262","reported_seq":1102,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.292225+0000","last_change":"2024-09-13T07:01:50.079211+0000","last_active":"2024-09-13T07:01:50.292225+0000","last_peered":"2024-09-13T07:01:50.292225+0000","last_clean":"2024-09-13T07:01:50.292225+0000","last_became_active":"2024-09-13T07:01:50.078679+0000","last_became_peered":"2024-09-13T07:01:50.078679+0000","last_unstale":"2024-09-13T07:01:50.292225+0000","last_undegraded":"2024-09-13T07:01:50.292225+0000","last_fullsized":"2024-09-13T07:01:50.292225+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:30:15.075968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001182647,"stat_sum":{"num_bytes":24082180,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":2098,"num_read_kb":11064,"num_write":400,"num_write_kb":4015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2756020,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"52'100","reported_seq":2233,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.383599+0000","last_change":"2024-09-13T07:01:21.293012+0000","last_active":"2024-09-13T07:01:50.383599+0000","last_peered":"2024-09-13T07:01:50.383599+0000","last_clean":"2024-09-13T07:01:50.383599+0000","last_became_active":"2024-09-13T07:01:19.277380+0000","last_became_peered":"2024-09-13T07:01:19.277380+0000","last_unstale":"2024-09-13T07:01:50.383599+0000","last_undegraded":"2024-09-13T07:01:50.383599+0000","last_fullsized":"2024-09-13T07:01:50.383599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:05:57.870409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010463009999999999,"stat_sum":{"num_bytes":8068075,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":2386,"num_read_kb":5518,"num_write":129,"num_write_kb":421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"52'54","reported_seq":282,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.364209+0000","last_change":"2024-09-13T07:01:21.314353+0000","last_active":"2024-09-13T07:01:50.364209+0000","last_peered":"2024-09-13T07:01:50.364209+0000","last_clean":"2024-09-13T07:01:50.364209+0000","last_became_active":"2024-09-13T07:01:19.280203+0000","last_became_peered":"2024-09-13T07:01:19.280203+0000","last_unstale":"2024-09-13T07:01:50.364209+0000","last_undegraded":"2024-09-13T07:01:50.364209+0000","last_fullsized":"2024-09-13T07:01:50.364209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:38.598517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024699230000000002,"stat_sum":{"num_bytes":3430133,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":552,"num_read_kb":466,"num_write":53,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"52'136","reported_seq":487,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.443948+0000","last_change":"2024-09-13T07:01:21.298190+0000","last_active":"2024-09-13T07:01:50.443948+0000","last_peered":"2024-09-13T07:01:50.443948+0000","last_clean":"2024-09-13T07:01:50.443948+0000","last_became_active":"2024-09-13T07:01:19.276431+0000","last_became_peered":"2024-09-13T07:01:19.276431+0000","last_unstale":"2024-09-13T07:01:50.443948+0000","last_undegraded":"2024-09-13T07:01:50.443948+0000","last_fullsized":"2024-09-13T07:01:50.443948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:50:28.618968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001937825,"stat_sum":{"num_bytes":28715903,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":757,"num_read_kb":5754,"num_write":178,"num_write_kb":7689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.078655+0000","last_change":"2024-09-13T07:01:11.221939+0000","last_active":"2024-09-13T07:01:49.078655+0000","last_peered":"2024-09-13T07:01:49.078655+0000","last_clean":"2024-09-13T07:01:49.078655+0000","last_became_active":"2024-09-13T07:01:09.212302+0000","last_became_peered":"2024-09-13T07:01:09.212302+0000","last_unstale":"2024-09-13T07:01:49.078655+0000","last_undegraded":"2024-09-13T07:01:49.078655+0000","last_fullsized":"2024-09-13T07:01:49.078655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:49.220240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"36'28","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.088035+0000","last_change":"2024-09-13T07:01:47.512765+0000","last_active":"2024-09-13T07:01:50.088035+0000","last_peered":"2024-09-13T07:01:50.088035+0000","last_clean":"2024-09-13T07:01:50.088035+0000","last_became_active":"2024-09-13T07:01:47.491472+0000","last_became_peered":"2024-09-13T07:01:47.491472+0000","last_unstale":"2024-09-13T07:01:50.088035+0000","last_undegraded":"2024-09-13T07:01:50.088035+0000","last_fullsized":"2024-09-13T07:01:50.088035+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:06:58.026055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038364399999999999,"stat_sum":{"num_bytes":1548478,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":6,"num_read":11,"num_read_kb":0,"num_write":22,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":20355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.083128+0000","last_change":"2024-09-13T07:01:11.232122+0000","last_active":"2024-09-13T07:01:49.083128+0000","last_peered":"2024-09-13T07:01:49.083128+0000","last_clean":"2024-09-13T07:01:49.083128+0000","last_became_active":"2024-09-13T07:01:09.213058+0000","last_became_peered":"2024-09-13T07:01:09.213058+0000","last_unstale":"2024-09-13T07:01:49.083128+0000","last_undegraded":"2024-09-13T07:01:49.083128+0000","last_fullsized":"2024-09-13T07:01:49.083128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:10.839167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067353900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"47'86","reported_seq":352,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.073362+0000","last_change":"2024-09-13T07:01:21.291928+0000","last_active":"2024-09-13T07:01:49.073362+0000","last_peered":"2024-09-13T07:01:49.073362+0000","last_clean":"2024-09-13T07:01:49.073362+0000","last_became_active":"2024-09-13T07:01:19.286391+0000","last_became_peered":"2024-09-13T07:01:19.286391+0000","last_unstale":"2024-09-13T07:01:49.073362+0000","last_undegraded":"2024-09-13T07:01:49.073362+0000","last_fullsized":"2024-09-13T07:01:49.073362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:58:51.082865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016908909999999999,"stat_sum":{"num_bytes":25352197,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":458,"num_read_kb":2111,"num_write":95,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.074613+0000","last_change":"2024-09-13T07:01:47.489709+0000","last_active":"2024-09-13T07:01:49.074613+0000","last_peered":"2024-09-13T07:01:49.074613+0000","last_clean":"2024-09-13T07:01:49.074613+0000","last_became_active":"2024-09-13T07:01:47.489419+0000","last_became_peered":"2024-09-13T07:01:47.489419+0000","last_unstale":"2024-09-13T07:01:49.074613+0000","last_undegraded":"2024-09-13T07:01:49.074613+0000","last_fullsized":"2024-09-13T07:01:49.074613+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:41.389847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"52'94","reported_seq":231,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.442035+0000","last_change":"2024-09-13T07:01:41.478005+0000","last_active":"2024-09-13T07:01:50.442035+0000","last_peered":"2024-09-13T07:01:50.442035+0000","last_clean":"2024-09-13T07:01:50.442035+0000","last_became_active":"2024-09-13T07:01:41.438741+0000","last_became_peered":"2024-09-13T07:01:41.438741+0000","last_unstale":"2024-09-13T07:01:50.442035+0000","last_undegraded":"2024-09-13T07:01:50.442035+0000","last_fullsized":"2024-09-13T07:01:50.442035+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:03.836314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085579199999999997,"stat_sum":{"num_bytes":20887922,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":308,"num_read_kb":205,"num_write":118,"num_write_kb":304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":3786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":56,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.083087+0000","last_change":"2024-09-13T07:01:47.486387+0000","last_active":"2024-09-13T07:01:49.083087+0000","last_peered":"2024-09-13T07:01:49.083087+0000","last_clean":"2024-09-13T07:01:49.083087+0000","last_became_active":"2024-09-13T07:01:47.485603+0000","last_became_peered":"2024-09-13T07:01:47.485603+0000","last_unstale":"2024-09-13T07:01:49.083087+0000","last_undegraded":"2024-09-13T07:01:49.083087+0000","last_fullsized":"2024-09-13T07:01:49.083087+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:57.144719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"52'44","reported_seq":497,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.289451+0000","last_change":"2024-09-13T07:01:21.293697+0000","last_active":"2024-09-13T07:01:50.289451+0000","last_peered":"2024-09-13T07:01:50.289451+0000","last_clean":"2024-09-13T07:01:50.289451+0000","last_became_active":"2024-09-13T07:01:19.279719+0000","last_became_peered":"2024-09-13T07:01:19.279719+0000","last_unstale":"2024-09-13T07:01:50.289451+0000","last_undegraded":"2024-09-13T07:01:50.289451+0000","last_fullsized":"2024-09-13T07:01:50.289451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:05:06.593984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085976499999999997,"stat_sum":{"num_bytes":18957152,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":510,"num_read_kb":3729,"num_write":44,"num_write_kb":332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"52'35","reported_seq":1143,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.088257+0000","last_change":"2024-09-13T07:01:47.540796+0000","last_active":"2024-09-13T07:01:50.088257+0000","last_peered":"2024-09-13T07:01:50.088257+0000","last_clean":"2024-09-13T07:01:50.088257+0000","last_became_active":"2024-09-13T07:01:47.491983+0000","last_became_peered":"2024-09-13T07:01:47.491983+0000","last_unstale":"2024-09-13T07:01:50.088257+0000","last_undegraded":"2024-09-13T07:01:50.088257+0000","last_fullsized":"2024-09-13T07:01:50.088257+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:18.382486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017265959999999999,"stat_sum":{"num_bytes":17305735,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":1041,"num_read_kb":13318,"num_write":45,"num_write_kb":4109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":3083106,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":50,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.071396+0000","last_change":"2024-09-13T07:01:11.228726+0000","last_active":"2024-09-13T07:01:49.071396+0000","last_peered":"2024-09-13T07:01:49.071396+0000","last_clean":"2024-09-13T07:01:49.071396+0000","last_became_active":"2024-09-13T07:01:09.211518+0000","last_became_peered":"2024-09-13T07:01:09.211518+0000","last_unstale":"2024-09-13T07:01:49.071396+0000","last_undegraded":"2024-09-13T07:01:49.071396+0000","last_fullsized":"2024-09-13T07:01:49.071396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:21.453172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"52'121","reported_seq":1147,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.396272+0000","last_change":"2024-09-13T07:01:44.465048+0000","last_active":"2024-09-13T07:01:50.396272+0000","last_peered":"2024-09-13T07:01:50.396272+0000","last_clean":"2024-09-13T07:01:50.396272+0000","last_became_active":"2024-09-13T07:01:44.463948+0000","last_became_peered":"2024-09-13T07:01:44.463948+0000","last_unstale":"2024-09-13T07:01:50.396272+0000","last_undegraded":"2024-09-13T07:01:50.396272+0000","last_fullsized":"2024-09-13T07:01:50.396272+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:34.829699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065804299999999997,"stat_sum":{"num_bytes":31667428,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":1621,"num_read_kb":4513,"num_write":141,"num_write_kb":8883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1395197,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":28,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.073205+0000","last_change":"2024-09-13T07:01:47.491502+0000","last_active":"2024-09-13T07:01:49.073205+0000","last_peered":"2024-09-13T07:01:49.073205+0000","last_clean":"2024-09-13T07:01:49.073205+0000","last_became_active":"2024-09-13T07:01:47.491145+0000","last_became_peered":"2024-09-13T07:01:47.491145+0000","last_unstale":"2024-09-13T07:01:49.073205+0000","last_undegraded":"2024-09-13T07:01:49.073205+0000","last_fullsized":"2024-09-13T07:01:49.073205+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:36:07.497138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"52'73","reported_seq":927,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.504568+0000","last_change":"2024-09-13T07:01:47.571765+0000","last_active":"2024-09-13T07:01:50.504568+0000","last_peered":"2024-09-13T07:01:50.504568+0000","last_clean":"2024-09-13T07:01:50.504568+0000","last_became_active":"2024-09-13T07:01:47.493229+0000","last_became_peered":"2024-09-13T07:01:47.493229+0000","last_unstale":"2024-09-13T07:01:50.504568+0000","last_undegraded":"2024-09-13T07:01:50.504568+0000","last_fullsized":"2024-09-13T07:01:50.504568+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:50:50.993168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044811299999999998,"stat_sum":{"num_bytes":21389127,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":1068,"num_read_kb":600,"num_write":60,"num_write_kb":598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":106600,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.074030+0000","last_change":"2024-09-13T07:01:11.859918+0000","last_active":"2024-09-13T07:01:49.074030+0000","last_peered":"2024-09-13T07:01:49.074030+0000","last_clean":"2024-09-13T07:01:49.074030+0000","last_became_active":"2024-09-13T07:01:09.210971+0000","last_became_peered":"2024-09-13T07:01:09.210971+0000","last_unstale":"2024-09-13T07:01:49.074030+0000","last_undegraded":"2024-09-13T07:01:49.074030+0000","last_fullsized":"2024-09-13T07:01:49.074030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:30.775691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"52'111","reported_seq":909,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.344677+0000","last_change":"2024-09-13T07:01:21.290419+0000","last_active":"2024-09-13T07:01:50.344677+0000","last_peered":"2024-09-13T07:01:50.344677+0000","last_clean":"2024-09-13T07:01:50.344677+0000","last_became_active":"2024-09-13T07:01:19.282935+0000","last_became_peered":"2024-09-13T07:01:19.282935+0000","last_unstale":"2024-09-13T07:01:50.344677+0000","last_undegraded":"2024-09-13T07:01:50.344677+0000","last_fullsized":"2024-09-13T07:01:50.344677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:36:53.773908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064294199999999997,"stat_sum":{"num_bytes":12168132,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":2326,"num_read_kb":3671,"num_write":117,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":61,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.071070+0000","last_change":"2024-09-13T07:01:41.443501+0000","last_active":"2024-09-13T07:01:49.071070+0000","last_peered":"2024-09-13T07:01:49.071070+0000","last_clean":"2024-09-13T07:01:49.071070+0000","last_became_active":"2024-09-13T07:01:41.436739+0000","last_became_peered":"2024-09-13T07:01:41.436739+0000","last_unstale":"2024-09-13T07:01:49.071070+0000","last_undegraded":"2024-09-13T07:01:49.071070+0000","last_fullsized":"2024-09-13T07:01:49.071070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:33:04.362119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":217,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.080564+0000","last_change":"2024-09-13T07:01:50.080564+0000","last_active":"2024-09-13T07:01:50.080564+0000","last_peered":"2024-09-13T07:01:50.080564+0000","last_clean":"2024-09-13T07:01:50.080564+0000","last_became_active":"2024-09-13T07:01:50.080072+0000","last_became_peered":"2024-09-13T07:01:50.080072+0000","last_unstale":"2024-09-13T07:01:50.080564+0000","last_undegraded":"2024-09-13T07:01:50.080564+0000","last_fullsized":"2024-09-13T07:01:50.080564+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:14:58.662626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"52'203","reported_seq":257,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.501767+0000","last_change":"2024-09-13T07:01:21.291091+0000","last_active":"2024-09-13T07:01:50.501767+0000","last_peered":"2024-09-13T07:01:50.501767+0000","last_clean":"2024-09-13T07:01:50.501767+0000","last_became_active":"2024-09-13T07:01:19.280796+0000","last_became_peered":"2024-09-13T07:01:19.280796+0000","last_unstale":"2024-09-13T07:01:50.501767+0000","last_undegraded":"2024-09-13T07:01:50.501767+0000","last_fullsized":"2024-09-13T07:01:50.501767+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:20:10.332964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001264996,"stat_sum":{"num_bytes":2247342,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":5,"num_read":914,"num_read_kb":743,"num_write":331,"num_write_kb":619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"41'20","reported_seq":161,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.083568+0000","last_change":"2024-09-13T07:01:41.467037+0000","last_active":"2024-09-13T07:01:49.083568+0000","last_peered":"2024-09-13T07:01:49.083568+0000","last_clean":"2024-09-13T07:01:49.083568+0000","last_became_active":"2024-09-13T07:01:41.438743+0000","last_became_peered":"2024-09-13T07:01:41.438743+0000","last_unstale":"2024-09-13T07:01:49.083568+0000","last_undegraded":"2024-09-13T07:01:49.083568+0000","last_fullsized":"2024-09-13T07:01:49.083568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:11:01.568164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022509729999999999,"stat_sum":{"num_bytes":1072125,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":3,"num_read":64,"num_read_kb":1036,"num_write":17,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":26238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"52'103","reported_seq":286,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.354962+0000","last_change":"2024-09-13T07:01:22.025365+0000","last_active":"2024-09-13T07:01:50.354962+0000","last_peered":"2024-09-13T07:01:50.354962+0000","last_clean":"2024-09-13T07:01:50.354962+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:01:50.354962+0000","last_undegraded":"2024-09-13T07:01:50.354962+0000","last_fullsized":"2024-09-13T07:01:50.354962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:11.537644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064478199999999999,"stat_sum":{"num_bytes":12279629,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":211,"num_read_kb":2425,"num_write":97,"num_write_kb":12112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"52'94","reported_seq":608,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.473334+0000","last_change":"2024-09-13T07:01:47.564025+0000","last_active":"2024-09-13T07:01:50.473334+0000","last_peered":"2024-09-13T07:01:50.473334+0000","last_clean":"2024-09-13T07:01:50.473334+0000","last_became_active":"2024-09-13T07:01:47.492956+0000","last_became_peered":"2024-09-13T07:01:47.492956+0000","last_unstale":"2024-09-13T07:01:50.473334+0000","last_undegraded":"2024-09-13T07:01:50.473334+0000","last_fullsized":"2024-09-13T07:01:50.473334+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:23:37.432022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001533932,"stat_sum":{"num_bytes":32735412,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":871,"num_read_kb":4478,"num_write":93,"num_write_kb":2277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1474279,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"47'154","reported_seq":1469,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.071295+0000","last_change":"2024-09-13T07:01:21.292633+0000","last_active":"2024-09-13T07:01:49.071295+0000","last_peered":"2024-09-13T07:01:49.071295+0000","last_clean":"2024-09-13T07:01:49.071295+0000","last_became_active":"2024-09-13T07:01:19.279813+0000","last_became_peered":"2024-09-13T07:01:19.279813+0000","last_unstale":"2024-09-13T07:01:49.071295+0000","last_undegraded":"2024-09-13T07:01:49.071295+0000","last_fullsized":"2024-09-13T07:01:49.071295+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:39.697691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072600099999999999,"stat_sum":{"num_bytes":27952104,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":3013,"num_read_kb":3570,"num_write":179,"num_write_kb":7193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"52'88","reported_seq":829,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.348829+0000","last_change":"2024-09-13T07:01:21.313310+0000","last_active":"2024-09-13T07:01:50.348829+0000","last_peered":"2024-09-13T07:01:50.348829+0000","last_clean":"2024-09-13T07:01:50.348829+0000","last_became_active":"2024-09-13T07:01:19.282520+0000","last_became_peered":"2024-09-13T07:01:19.282520+0000","last_unstale":"2024-09-13T07:01:50.348829+0000","last_undegraded":"2024-09-13T07:01:50.348829+0000","last_fullsized":"2024-09-13T07:01:50.348829+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:41:44.801487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018248170000000001,"stat_sum":{"num_bytes":24616915,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":959,"num_read_kb":4815,"num_write":76,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"52'51","reported_seq":432,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.400091+0000","last_change":"2024-09-13T07:01:21.313674+0000","last_active":"2024-09-13T07:01:50.400091+0000","last_peered":"2024-09-13T07:01:50.400091+0000","last_clean":"2024-09-13T07:01:50.400091+0000","last_became_active":"2024-09-13T07:01:19.281215+0000","last_became_peered":"2024-09-13T07:01:19.281215+0000","last_unstale":"2024-09-13T07:01:50.400091+0000","last_undegraded":"2024-09-13T07:01:50.400091+0000","last_fullsized":"2024-09-13T07:01:50.400091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:22.161596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019878309999999998,"stat_sum":{"num_bytes":15049741,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":367,"num_read_kb":4414,"num_write":56,"num_write_kb":2087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"50'31","reported_seq":155,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.074861+0000","last_change":"2024-09-13T07:01:21.314048+0000","last_active":"2024-09-13T07:01:49.074861+0000","last_peered":"2024-09-13T07:01:49.074861+0000","last_clean":"2024-09-13T07:01:49.074861+0000","last_became_active":"2024-09-13T07:01:19.281169+0000","last_became_peered":"2024-09-13T07:01:19.281169+0000","last_unstale":"2024-09-13T07:01:49.074861+0000","last_undegraded":"2024-09-13T07:01:49.074861+0000","last_fullsized":"2024-09-13T07:01:49.074861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:49:46.525500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023438180000000001,"stat_sum":{"num_bytes":9371279,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":108,"num_read_kb":51,"num_write":25,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"50'36","reported_seq":312,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T07:01:49.079109+0000","last_change":"2024-09-13T07:01:47.526764+0000","last_active":"2024-09-13T07:01:49.079109+0000","last_peered":"2024-09-13T07:01:49.079109+0000","last_clean":"2024-09-13T07:01:49.079109+0000","last_became_active":"2024-09-13T07:01:47.488355+0000","last_became_peered":"2024-09-13T07:01:47.488355+0000","last_unstale":"2024-09-13T07:01:49.079109+0000","last_undegraded":"2024-09-13T07:01:49.079109+0000","last_fullsized":"2024-09-13T07:01:49.079109+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:21:34.227554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061928799999999996,"stat_sum":{"num_bytes":5019187,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":206,"num_read_kb":3586,"num_write":38,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1706762,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"52'72","reported_seq":464,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.106351+0000","last_change":"2024-09-13T07:01:21.313825+0000","last_active":"2024-09-13T07:01:50.106351+0000","last_peered":"2024-09-13T07:01:50.106351+0000","last_clean":"2024-09-13T07:01:50.106351+0000","last_became_active":"2024-09-13T07:01:19.279714+0000","last_became_peered":"2024-09-13T07:01:19.279714+0000","last_unstale":"2024-09-13T07:01:50.106351+0000","last_undegraded":"2024-09-13T07:01:50.106351+0000","last_fullsized":"2024-09-13T07:01:50.106351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:06:09.717433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002154132,"stat_sum":{"num_bytes":23299166,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":520,"num_read_kb":3862,"num_write":86,"num_write_kb":383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"52'104","reported_seq":430,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.472420+0000","last_change":"2024-09-13T07:01:50.078398+0000","last_active":"2024-09-13T07:01:50.472420+0000","last_peered":"2024-09-13T07:01:50.472420+0000","last_clean":"2024-09-13T07:01:50.472420+0000","last_became_active":"2024-09-13T07:01:50.078015+0000","last_became_peered":"2024-09-13T07:01:50.078015+0000","last_unstale":"2024-09-13T07:01:50.472420+0000","last_undegraded":"2024-09-13T07:01:50.472420+0000","last_fullsized":"2024-09-13T07:01:50.472420+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:52.203434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060210699999999995,"stat_sum":{"num_bytes":20091763,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":560,"num_read_kb":2344,"num_write":132,"num_write_kb":382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":60382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"52'118","reported_seq":394,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T07:01:50.397460+0000","last_change":"2024-09-13T07:01:21.313226+0000","last_active":"2024-09-13T07:01:50.397460+0000","last_peered":"2024-09-13T07:01:50.397460+0000","last_clean":"2024-09-13T07:01:50.397460+0000","last_became_active":"2024-09-13T07:01:19.279419+0000","last_became_peered":"2024-09-13T07:01:19.279419+0000","last_unstale":"2024-09-13T07:01:50.397460+0000","last_undegraded":"2024-09-13T07:01:50.397460+0000","last_fullsized":"2024-09-13T07:01:50.397460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:46:29.817757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001803165,"stat_sum":{"num_bytes":15509239,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":545,"num_read_kb":1326,"num_write":167,"num_write_kb":7399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":515834542,"num_objects":415,"num_object_clones":93,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":152,"num_read":27472,"num_read_kb":94091,"num_write":3258,"num_write_kb":69027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":11549209,"num_keys_recovered":59,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":55951360,"data_stored":61492980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2746,"ondisk_log_size":2746,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37744,"kb_used_data":6620,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94334096,"statfs":{"total":96636764160,"available":96598114304,"internally_reserved":0,"allocated":6778880,"data_stored":6296109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36796,"kb_used_data":9756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335044,"statfs":{"total":96636764160,"available":96599085056,"internally_reserved":0,"allocated":9990144,"data_stored":11110600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39988,"kb_used_data":8872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94331852,"statfs":{"total":96636764160,"available":96595816448,"internally_reserved":0,"allocated":9084928,"data_stored":9846191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41668,"kb_used_data":10544,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94330172,"statfs":{"total":96636764160,"available":96594096128,"internally_reserved":0,"allocated":10797056,"data_stored":10979616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43504,"kb_used_data":8292,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94328336,"statfs":{"total":96636764160,"available":96592216064,"internally_reserved":0,"allocated":8491008,"data_stored":8965085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":2,"up_from":0,"seq":55834574861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38092,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333748,"statfs":{"total":96636764160,"available":96597757952,"internally_reserved":0,"allocated":2949120,"data_stored":3100103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]}]},{"osd":0,"up_from":0,"seq":55834574861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9293824,"data_stored":10365051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2555904,"data_stored":3026764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2158592,"data_stored":2536533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8183808,"data_stored":8948819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10973473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8695808,"data_stored":9777208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8863744,"data_stored":10322835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5713920,"data_stored":5542297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:01:51.860 INFO:teuthology.orchestra.run.smithi118.stdout:1126 trunc from 0xdee5a26 to 0xc7259b 2024-09-13T07:01:51.864 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:01:51.898 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:51.894+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:52.000 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:51.997+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:52.008 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:52.008 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:52.017 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:01:52.028 INFO:teuthology.orchestra.run.smithi118.stdout:1128 punch from 0x8312a4 to 0x83effc, (0xdd58 bytes) 2024-09-13T07:01:52.054 INFO:teuthology.orchestra.run.smithi118.stdout:1129 writesame 0xb350aa8 thru 0xb358988 data_size 0x1b(0x7ee1 bytes) 2024-09-13T07:01:52.054 INFO:teuthology.orchestra.run.smithi118.stdout:1130 punch from 0x35e921e to 0x35f2222, (0x9004 bytes) 2024-09-13T07:01:52.054 INFO:teuthology.orchestra.run.smithi118.stdout:1131 read 0x219fc21 thru 0x21a5839 (0x5c19 bytes) 2024-09-13T07:01:52.054 INFO:teuthology.orchestra.run.smithi118.stdout:1133 read 0x9cc2113 thru 0x9ccb113 (0x9001 bytes) 2024-09-13T07:01:52.055 INFO:teuthology.orchestra.run.smithi118.stdout:1134 write 0x5ad6129 thru 0x5adeb43 (0x8a1b bytes) 2024-09-13T07:01:52.058 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:52.058 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:52.058 INFO:teuthology.orchestra.run.smithi118.stdout:1138 punch from 0x219a049 to 0x21a3693, (0x964a bytes) 2024-09-13T07:01:52.058 INFO:teuthology.orchestra.run.smithi118.stdout:1139 read 0x562dbec thru 0x5635581 (0x7996 bytes) 2024-09-13T07:01:52.059 INFO:teuthology.orchestra.run.smithi118.stdout:1140 read 0x8d958fd thru 0x8d95e11 (0x515 bytes) 2024-09-13T07:01:52.059 INFO:teuthology.orchestra.run.smithi118.stdout:1142 write 0xb5ea78 thru 0xb6374a (0x4cd3 bytes) 2024-09-13T07:01:52.062 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:52.062 INFO:teuthology.orchestra.run.smithi118.stdout:1144 read 0xaf3369b thru 0xaf354e7 (0x1e4d bytes) 2024-09-13T07:01:52.073 INFO:teuthology.orchestra.run.smithi118.stdout:1145 trunc from 0xb358989 to 0x42d9cdb 2024-09-13T07:01:52.100 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:52.097+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:52.107 INFO:teuthology.orchestra.run.smithi118.stdout:1146 write 0x9e630d9 thru 0x9e72abf (0xf9e7 bytes) 2024-09-13T07:01:52.126 INFO:teuthology.orchestra.run.smithi118.stdout:1147 write 0xd036ab9 thru 0xd03af87 (0x44cf bytes) 2024-09-13T07:01:52.131 INFO:teuthology.orchestra.run.smithi118.stdout:1150 write 0xcc76c1c thru 0xcc7b81b (0x4c00 bytes) 2024-09-13T07:01:52.134 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:52.134 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:52.137 INFO:teuthology.orchestra.run.smithi118.stdout:1152 read 0x765fac8 thru 0x766e68b (0xebc4 bytes) 2024-09-13T07:01:52.138 INFO:teuthology.orchestra.run.smithi118.stdout:1153 punch from 0xce381db to 0xce3b175, (0x2f9a bytes) 2024-09-13T07:01:52.144 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:01:52.151 INFO:teuthology.orchestra.run.smithi118.stdout:1154 write 0xea0c9ec thru 0xea19775 (0xcd8a bytes) 2024-09-13T07:01:52.154 INFO:teuthology.orchestra.run.smithi118.stdout:1155 read 0x51c7a31 thru 0x51d16d9 (0x9ca9 bytes) 2024-09-13T07:01:52.168 INFO:teuthology.orchestra.run.smithi118.stdout:1156 trunc from 0xea19776 to 0x66a2f60 2024-09-13T07:01:52.195 INFO:teuthology.orchestra.run.smithi118.stdout:1157 punch from 0x3d592a4 to 0x3d5ba80, (0x27dc bytes) 2024-09-13T07:01:52.198 INFO:teuthology.orchestra.run.smithi118.stdout:1159 trunc from 0x66a2f60 to 0x541724b 2024-09-13T07:01:52.201 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:52.198+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:52.214 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:52.235 INFO:teuthology.orchestra.run.smithi118.stdout:1165 writesame 0xdc3c228 thru 0xdc44b3d data_size 0x3(0x8916 bytes) 2024-09-13T07:01:52.235 INFO:teuthology.orchestra.run.smithi118.stdout:1166 read 0x78b30c1 thru 0x78be6cd (0xb60d bytes) 2024-09-13T07:01:52.235 INFO:teuthology.orchestra.run.smithi118.stdout:1168 punch from 0x2f12418 to 0x2f14c7f, (0x2867 bytes) 2024-09-13T07:01:52.235 INFO:teuthology.orchestra.run.smithi118.stdout:1170 write 0x2b21297 thru 0x2b213b7 (0x121 bytes) 2024-09-13T07:01:52.238 INFO:teuthology.orchestra.run.smithi118.stdout:1171 write 0x10cbb61 thru 0x10cec2f (0x30cf bytes) 2024-09-13T07:01:52.241 INFO:teuthology.orchestra.run.smithi118.stdout:1173 writesame 0xc45cc65 thru 0xc45ea90 data_size 0x2(0x1e2c bytes) 2024-09-13T07:01:52.249 INFO:teuthology.orchestra.run.smithi118.stdout:1174 punch from 0x8217791 to 0x821c3f5, (0x4c64 bytes) 2024-09-13T07:01:52.249 INFO:teuthology.orchestra.run.smithi118.stdout:1175 write 0x5db9c56 thru 0x5dc37f5 (0x9ba0 bytes) 2024-09-13T07:01:52.252 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:52.252 INFO:teuthology.orchestra.run.smithi118.stdout:1178 read 0xaa6d286 thru 0xaa6efd6 (0x1d51 bytes) 2024-09-13T07:01:52.252 INFO:teuthology.orchestra.run.smithi118.stdout:1179 writesame 0xa551ae9 thru 0xa560267 data_size 0x1(0xe77f bytes) 2024-09-13T07:01:52.256 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:52.256 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:52.266 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:01:52.278 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:01:52.297 INFO:teuthology.orchestra.run.smithi118.stdout:1180 write 0x37000fa thru 0x37033a6 (0x32ad bytes) 2024-09-13T07:01:52.301 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:52.298+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:52.311 INFO:teuthology.orchestra.run.smithi118.stdout:1181 write 0xe3c9f28 thru 0xe3caa88 (0xb61 bytes) 2024-09-13T07:01:52.325 INFO:teuthology.orchestra.run.smithi118.stdout:1182 write 0xe46334e thru 0xe46d832 (0xa4e5 bytes) 2024-09-13T07:01:52.331 INFO:teuthology.orchestra.run.smithi118.stdout:1183 write 0x2863fe6 thru 0x286ece4 (0xacff bytes) 2024-09-13T07:01:52.334 INFO:teuthology.orchestra.run.smithi118.stdout:1184 punch from 0x3c7d37e to 0x3c8162f, (0x42b1 bytes) 2024-09-13T07:01:52.334 INFO:teuthology.orchestra.run.smithi118.stdout:1185 writesame 0x31783f2 thru 0x3180dea data_size 0x13(0x89f9 bytes) 2024-09-13T07:01:52.338 INFO:teuthology.orchestra.run.smithi118.stdout:1187 read 0x71603d8 thru 0x71610d8 (0xd01 bytes) 2024-09-13T07:01:52.339 INFO:teuthology.orchestra.run.smithi118.stdout:1188 writesame 0x5f2f6c3 thru 0x5f3afa4 data_size 0x5(0xb8e2 bytes) 2024-09-13T07:01:52.348 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:52.348 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:52.348 INFO:teuthology.orchestra.run.smithi118.stdout:1192 writesame 0x2ac2799 thru 0x2ac4081 data_size 0x1(0x18e9 bytes) 2024-09-13T07:01:52.353 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:52.353 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:52.353 INFO:teuthology.orchestra.run.smithi118.stdout:1196 write 0x183861b thru 0x183c24a (0x3c30 bytes) 2024-09-13T07:01:52.365 INFO:teuthology.orchestra.run.smithi118.stdout:1198 trunc from 0xe46d833 to 0x8b85c8b 2024-09-13T07:01:52.380 INFO:teuthology.orchestra.run.smithi118.stdout:1199 write 0x85d9b62 thru 0x85e9238 (0xf6d7 bytes) 2024-09-13T07:01:52.386 INFO:teuthology.orchestra.run.smithi118.stdout:1201 writesame 0x60a4a86 thru 0x60acdd7 data_size 0x1a(0x8352 bytes) 2024-09-13T07:01:52.389 INFO:teuthology.orchestra.run.smithi118.stdout:1202 read 0x4f809cf thru 0x4f88ef1 (0x8523 bytes) 2024-09-13T07:01:52.390 INFO:teuthology.orchestra.run.smithi118.stdout:1203 read 0x52e96cd thru 0x52eb7d3 (0x2107 bytes) 2024-09-13T07:01:52.397 INFO:teuthology.orchestra.run.smithi118.stdout:1204 trunc from 0x8b85c8b to 0x4ce6123 2024-09-13T07:01:52.403 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:52.398+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:52.409 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:52.409 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:52.422 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:52.422 INFO:teuthology.orchestra.run.smithi118.stdout:1207 punch from 0x13736d1 to 0x137ee71, (0xb7a0 bytes) 2024-09-13T07:01:52.425 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:01:52.437 INFO:teuthology.orchestra.run.smithi118.stdout:1208 write 0x9f0b7ed thru 0x9f1a5cb (0xeddf bytes) 2024-09-13T07:01:52.439 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:52.455 INFO:teuthology.orchestra.run.smithi118.stdout:1210 write 0xcbcbef4 thru 0xcbd2567 (0x6674 bytes) 2024-09-13T07:01:52.458 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:52.461 INFO:teuthology.orchestra.run.smithi118.stdout:1211 clone 15 order 23 su 65536 sc 8 2024-09-13T07:01:52.476 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:52.493 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:52.503 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:52.501+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:52.514 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:52.538 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:52.541 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:52.541 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:434 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'4, mlcod=26'2)", 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.006328451000002, 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001931945, 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:01:52.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.543 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:01:52.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.543 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535695+0000", 2024-09-13T07:01:52.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:52.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.544 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535709+0000", 2024-09-13T07:01:52.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3577999999999999e-05 2024-09-13T07:01:52.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.544 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535732+0000", 2024-09-13T07:01:52.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3085e-05 2024-09-13T07:01:52.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.544 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535742+0000", 2024-09-13T07:01:52.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.048e-05 2024-09-13T07:01:52.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535840+0000", 2024-09-13T07:01:52.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7159999999999998e-05 2024-09-13T07:01:52.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535871+0000", 2024-09-13T07:01:52.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1510999999999997e-05 2024-09-13T07:01:52.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.546 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537572+0000", 2024-09-13T07:01:52.546 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017008279999999999 2024-09-13T07:01:52.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.546 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.546 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537629+0000", 2024-09-13T07:01:52.546 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6520000000000001e-05 2024-09-13T07:01:52.546 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.546 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.546 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.547 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:435 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'5, mlcod=26'2)", 2024-09-13T07:01:52.547 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:52.547 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.005196744999999, 2024-09-13T07:01:52.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017126769999999999, 2024-09-13T07:01:52.547 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.547 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.547 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:52.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:52.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:01:52.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:52.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536835+0000", 2024-09-13T07:01:52.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4939999999999997e-06 2024-09-13T07:01:52.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536854+0000", 2024-09-13T07:01:52.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8655999999999999e-05 2024-09-13T07:01:52.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.550 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536862+0000", 2024-09-13T07:01:52.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6570000000000003e-06 2024-09-13T07:01:52.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.550 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536926+0000", 2024-09-13T07:01:52.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4037999999999999e-05 2024-09-13T07:01:52.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536958+0000", 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1584000000000001e-05 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538493+0000", 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00153513 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538541+0000", 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7883999999999998e-05 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.551 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.552 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.552 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:483 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'24, mlcod=26'21)", 2024-09-13T07:01:52.552 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:01:52.552 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.926718116, 2024-09-13T07:01:52.552 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018371780000000001, 2024-09-13T07:01:52.552 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.552 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.552 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.552 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:01:52.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.553 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:01:52.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.553 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615306+0000", 2024-09-13T07:01:52.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:52.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615322+0000", 2024-09-13T07:01:52.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5519000000000002e-05 2024-09-13T07:01:52.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615335+0000", 2024-09-13T07:01:52.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3854e-05 2024-09-13T07:01:52.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.555 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.555 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615345+0000", 2024-09-13T07:01:52.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4930000000000004e-06 2024-09-13T07:01:52.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.555 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.555 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615431+0000", 2024-09-13T07:01:52.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6346000000000003e-05 2024-09-13T07:01:52.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.555 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615480+0000", 2024-09-13T07:01:52.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8303999999999997e-05 2024-09-13T07:01:52.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617102+0000", 2024-09-13T07:01:52.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016225250000000001 2024-09-13T07:01:52.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617144+0000", 2024-09-13T07:01:52.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1936e-05 2024-09-13T07:01:52.556 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.557 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.557 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.557 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:518 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'38, mlcod=26'36)", 2024-09-13T07:01:52.557 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:01:52.557 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.217141378000001, 2024-09-13T07:01:52.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018201179999999999, 2024-09-13T07:01:52.557 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.557 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.557 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324883+0000", 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324891+0000", 2024-09-13T07:01:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1499999999999999e-06 2024-09-13T07:01:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324924+0000", 2024-09-13T07:01:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2601999999999997e-05 2024-09-13T07:01:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324942+0000", 2024-09-13T07:01:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7796999999999999e-05 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325060+0000", 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011786900000000001 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325123+0000", 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3474999999999998e-05 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326620+0000", 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014968830000000001 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326704+0000", 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3716999999999998e-05 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:597 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'39, mlcod=26'37)", 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:01:52.561 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.100314050000001, 2024-09-13T07:01:52.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017959829999999999, 2024-09-13T07:01:52.562 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.562 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.562 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:01:52.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:01:52.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441709+0000", 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441728+0000", 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9627999999999999e-05 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441765+0000", 2024-09-13T07:01:52.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6559e-05 2024-09-13T07:01:52.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.564 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441783+0000", 2024-09-13T07:01:52.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7691000000000001e-05 2024-09-13T07:01:52.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.564 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441904+0000", 2024-09-13T07:01:52.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121184 2024-09-13T07:01:52.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441966+0000", 2024-09-13T07:01:52.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2428999999999995e-05 2024-09-13T07:01:52.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443448+0000", 2024-09-13T07:01:52.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00148197 2024-09-13T07:01:52.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443507+0000", 2024-09-13T07:01:52.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8699000000000002e-05 2024-09-13T07:01:52.566 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.566 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.566 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.566 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:01:52.566 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:01:52.566 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.485433755999999, 2024-09-13T07:01:52.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:01:52.566 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.566 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.566 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:01:52.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:01:52.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:01:52.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:52.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:01:52.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:01:52.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:01:52.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:01:52.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:01:52.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:01:52.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:01:52.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:01:52.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:01:52.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:01:52.570 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.570 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.570 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1548 3.4 e27/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 27'52, mlcod=27'50)", 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.399812220000001, 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017825779999999999, 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142211+0000", 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142232+0000", 2024-09-13T07:01:52.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0888000000000001e-05 2024-09-13T07:01:52.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142268+0000", 2024-09-13T07:01:52.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6261999999999999e-05 2024-09-13T07:01:52.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142285+0000", 2024-09-13T07:01:52.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7640000000000001e-05 2024-09-13T07:01:52.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142381+0000", 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5171000000000001e-05 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142450+0000", 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9778999999999997e-05 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143953+0000", 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001503129 2024-09-13T07:01:52.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143995+0000", 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1878000000000001e-05 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1550 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'14, mlcod=27'12)", 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.399164508999998, 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002750452, 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.575 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142859+0000", 2024-09-13T07:01:52.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142874+0000", 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5149e-05 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142911+0000", 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6439000000000001e-05 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142926+0000", 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5079000000000001e-05 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143012+0000", 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6230000000000006e-05 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143063+0000", 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1369000000000002e-05 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145539+0000", 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024753420000000002 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145611+0000", 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2392999999999996e-05 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1668 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'17, mlcod=27'15)", 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:52.579 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.200749046999999, 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033976620000000001, 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:01:52.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:52.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341281+0000", 2024-09-13T07:01:52.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8109999999999999e-06 2024-09-13T07:01:52.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341298+0000", 2024-09-13T07:01:52.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6575e-05 2024-09-13T07:01:52.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341308+0000", 2024-09-13T07:01:52.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0369e-05 2024-09-13T07:01:52.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341434+0000", 2024-09-13T07:01:52.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125454 2024-09-13T07:01:52.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341453+0000", 2024-09-13T07:01:52.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9159000000000001e-05 2024-09-13T07:01:52.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344638+0000", 2024-09-13T07:01:52.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031851750000000002 2024-09-13T07:01:52.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344674+0000", 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5410999999999997e-05 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1666 3.4 e28/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 28'55, mlcod=27'53)", 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.200695374999999, 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001805809, 2024-09-13T07:01:52.584 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341329+0000", 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341349+0000", 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.999e-05 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341357+0000", 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0620000000000003e-06 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341361+0000", 2024-09-13T07:01:52.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:01:52.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341710+0000", 2024-09-13T07:01:52.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034873400000000001 2024-09-13T07:01:52.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341736+0000", 2024-09-13T07:01:52.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5772e-05 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343097+0000", 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013615319999999999 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343135+0000", 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8111999999999997e-05 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.589 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1800 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'20, mlcod=28'18)", 2024-09-13T07:01:52.589 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:01:52.589 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.998324909000001, 2024-09-13T07:01:52.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001874398, 2024-09-13T07:01:52.589 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.589 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.589 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:01:52.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:01:52.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543699+0000", 2024-09-13T07:01:52.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:52.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543713+0000", 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4786e-05 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543737+0000", 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3626e-05 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543752+0000", 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.489e-05 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543832+0000", 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0396999999999996e-05 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543994+0000", 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016129100000000001 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545471+0000", 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014771960000000001 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545575+0000", 2024-09-13T07:01:52.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103732 2024-09-13T07:01:52.593 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.593 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.593 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.593 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1882 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'23, mlcod=28'21)", 2024-09-13T07:01:52.593 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:01:52.593 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.902691927999999, 2024-09-13T07:01:52.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001767172, 2024-09-13T07:01:52.594 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.594 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.594 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:01:52.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:01:52.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639332+0000", 2024-09-13T07:01:52.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:52.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639343+0000", 2024-09-13T07:01:52.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1436e-05 2024-09-13T07:01:52.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639361+0000", 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7915e-05 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639370+0000", 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3189999999999995e-06 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639425+0000", 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4917999999999998e-05 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639454+0000", 2024-09-13T07:01:52.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9862999999999999e-05 2024-09-13T07:01:52.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641059+0000", 2024-09-13T07:01:52.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016048950000000001 2024-09-13T07:01:52.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641100+0000", 2024-09-13T07:01:52.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0883999999999997e-05 2024-09-13T07:01:52.597 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3069 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'33, mlcod=31'31)", 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.379552212, 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018597010000000001, 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:01:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:01:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162471+0000", 2024-09-13T07:01:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162520+0000", 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8466e-05 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162546+0000", 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6089999999999999e-05 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162562+0000", 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5962000000000001e-05 2024-09-13T07:01:52.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162646+0000", 2024-09-13T07:01:52.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3985000000000003e-05 2024-09-13T07:01:52.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162691+0000", 2024-09-13T07:01:52.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5785999999999998e-05 2024-09-13T07:01:52.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164265+0000", 2024-09-13T07:01:52.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015740890000000001 2024-09-13T07:01:52.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164333+0000", 2024-09-13T07:01:52.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7014999999999994e-05 2024-09-13T07:01:52.602 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.602 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3121 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'36, mlcod=31'34)", 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.265605825000002, 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019657780000000001, 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:01:52.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276417+0000", 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276497+0000", 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9729999999999997e-05 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276537+0000", 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0102999999999999e-05 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276553+0000", 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6518999999999999e-05 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276662+0000", 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109252 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276729+0000", 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7056000000000003e-05 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278270+0000", 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015408819999999999 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278385+0000", 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114613 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3390 3.0 e32/22 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:head v 32'45, mlcod=31'42)", 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.046517477999998, 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002246587, 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495507+0000", 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:52.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495530+0000", 2024-09-13T07:01:52.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2592999999999999e-05 2024-09-13T07:01:52.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495550+0000", 2024-09-13T07:01:52.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0152999999999998e-05 2024-09-13T07:01:52.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495562+0000", 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2336e-05 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495662+0000", 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100317 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495717+0000", 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4304e-05 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497707+0000", 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019906849999999999 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497754+0000", 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6715999999999998e-05 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3399 3.4 e32/22 3:22863a52:::rbd_data.11945d91e99c.0000000000000013:head v 32'65, mlcod=31'62)", 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:01:52.611 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.042580806, 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028830779999999999, 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499442+0000", 2024-09-13T07:01:52.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:52.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499459+0000", 2024-09-13T07:01:52.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7042e-05 2024-09-13T07:01:52.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499493+0000", 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3788000000000002e-05 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499512+0000", 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9055999999999998e-05 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499608+0000", 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6168000000000005e-05 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499658+0000", 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9469999999999999e-05 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502277+0000", 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026188639999999998 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502327+0000", 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0578999999999998e-05 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.615 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3395 3.0 e32/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 32'47, mlcod=31'43)", 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.042482720999999, 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027505870000000001, 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499560+0000", 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8416000000000001e-05 2024-09-13T07:01:52.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499576+0000", 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5602000000000001e-05 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499587+0000", 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1112e-05 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499654+0000", 2024-09-13T07:01:52.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7164999999999998e-05 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499674+0000", 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9638000000000001e-05 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502210+0000", 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025367100000000002 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502293+0000", 2024-09-13T07:01:52.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2607000000000004e-05 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:4534 3.0 e32/22 3:00392e9e:::rbd_data.11944cdc0656.000000000000000c:head v 32'51, mlcod=32'49)", 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.630781535000001, 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017597330000000001, 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:01:52.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:01:52.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911242+0000", 2024-09-13T07:01:52.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:52.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911306+0000", 2024-09-13T07:01:52.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4480999999999995e-05 2024-09-13T07:01:52.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911338+0000", 2024-09-13T07:01:52.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1733000000000003e-05 2024-09-13T07:01:52.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911351+0000", 2024-09-13T07:01:52.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3241000000000001e-05 2024-09-13T07:01:52.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911455+0000", 2024-09-13T07:01:52.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103723 2024-09-13T07:01:52.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911495+0000", 2024-09-13T07:01:52.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0683000000000003e-05 2024-09-13T07:01:52.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.912933+0000", 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001437219 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.913003+0000", 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0629000000000004e-05 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11205 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'50, mlcod=38'47)", 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:01:52.624 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.917441050000001, 2024-09-13T07:01:52.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031049490000000001, 2024-09-13T07:01:52.625 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.625 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.625 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:01:52.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:01:52.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624583+0000", 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625339+0000", 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00075563200000000001 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625358+0000", 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9916999999999999e-05 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625366+0000", 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8050000000000007e-06 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625415+0000", 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8231e-05 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625447+0000", 2024-09-13T07:01:52.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2703999999999997e-05 2024-09-13T07:01:52.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627652+0000", 2024-09-13T07:01:52.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022050659999999999 2024-09-13T07:01:52.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627689+0000", 2024-09-13T07:01:52.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6612000000000001e-05 2024-09-13T07:01:52.628 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.628 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.628 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.629 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11206 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'52, mlcod=38'47)", 2024-09-13T07:01:52.629 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:01:52.629 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.916170384000001, 2024-09-13T07:01:52.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022238470000000002, 2024-09-13T07:01:52.629 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:52.629 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:52.629 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:52.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:52.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625854+0000", 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625863+0000", 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5380000000000008e-06 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625883+0000", 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9247000000000001e-05 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625893+0000", 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0176e-05 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626411+0000", 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051847600000000003 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626431+0000", 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0199000000000001e-05 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628028+0000", 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015968110000000001 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:52.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:52.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:52.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628078+0000", 2024-09-13T07:01:52.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0170000000000002e-05 2024-09-13T07:01:52.633 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.633 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.633 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.633 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:52.633 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:52.633 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:52.637 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:01:52.637 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:52.638 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:52.638 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:52.638 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:52.639 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:52.639 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:52.639 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:52.639 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:52.639 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:52.639 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:52.639 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:52.633+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:52.737 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:52.734+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:52.793 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:52.793 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:52.793 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:52.793 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:52.803 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:01:52.838 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:52.834+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:52.906 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:52.906 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:52.906 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:52.906 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:52.906 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:52.915 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:01:52.939 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:52.937+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:53.026 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:53.026 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:53.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:53.026 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:53.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.026 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:441 3.b 3:d465e8df:::rbd_data.1194d22ad098.0000000000000036:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T07:01:53.026 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:53.026 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.483382147, 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012043093, 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 441 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541125+0000", 2024-09-13T07:01:53.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541122+0000", 2024-09-13T07:01:53.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:53.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541136+0000", 2024-09-13T07:01:53.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3899e-05 2024-09-13T07:01:53.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541163+0000", 2024-09-13T07:01:53.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.694e-05 2024-09-13T07:01:53.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541176+0000", 2024-09-13T07:01:53.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2802e-05 2024-09-13T07:01:53.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541257+0000", 2024-09-13T07:01:53.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1018000000000003e-05 2024-09-13T07:01:53.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.541414+0000", 2024-09-13T07:01:53.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015745000000000001 2024-09-13T07:01:53.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:53.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.544450+0000", 2024-09-13T07:01:53.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0030357209999999999 2024-09-13T07:01:53.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:53.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553002+0000", 2024-09-13T07:01:53.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085521020000000007 2024-09-13T07:01:53.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:53.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553022+0000", 2024-09-13T07:01:53.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9694000000000002e-05 2024-09-13T07:01:53.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:53.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553103+0000", 2024-09-13T07:01:53.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1008999999999996e-05 2024-09-13T07:01:53.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.553168+0000", 2024-09-13T07:01:53.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5010000000000003e-05 2024-09-13T07:01:53.032 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.032 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.032 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.033 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1155 3.7 3:e095a6bd:::rbd_data.11943a018f34.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:53.033 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:01:53.033 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.191742884, 2024-09-13T07:01:53.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022798902999999999, 2024-09-13T07:01:53.033 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.033 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:53.033 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.033 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.033 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.033 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1155 2024-09-13T07:01:53.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.033 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832762+0000", 2024-09-13T07:01:53.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832776+0000", 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3828e-05 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832806+0000", 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0159e-05 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832820+0000", 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.384e-05 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832904+0000", 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4506999999999996e-05 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.833121+0000", 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000217223 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:53.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.839072+0000", 2024-09-13T07:01:53.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0059509849999999998 2024-09-13T07:01:53.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:53.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855489+0000", 2024-09-13T07:01:53.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016416907000000001 2024-09-13T07:01:53.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:53.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855503+0000", 2024-09-13T07:01:53.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3658999999999999e-05 2024-09-13T07:01:53.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855535+0000", 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1696000000000003e-05 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855563+0000", 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8578999999999999e-05 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1200 3.7 3:e095a6bd:::rbd_data.11943a018f34.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.045403933999999, 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018736263, 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1200 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:01:53.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:01:53.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979100+0000", 2024-09-13T07:01:53.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979115+0000", 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4823e-05 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979149+0000", 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3772999999999997e-05 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979163+0000", 2024-09-13T07:01:53.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3845999999999999e-05 2024-09-13T07:01:53.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979333+0000", 2024-09-13T07:01:53.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000169622 2024-09-13T07:01:53.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979603+0000", 2024-09-13T07:01:53.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00027078999999999997 2024-09-13T07:01:53.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:53.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.986017+0000", 2024-09-13T07:01:53.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064139590000000003 2024-09-13T07:01:53.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:53.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997476+0000", 2024-09-13T07:01:53.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011458582 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997588+0000", 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011161 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997732+0000", 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000144 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997839+0000", 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107892 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1531 3.c e27/22 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head v 27'9, mlcod=27'7)", 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.949975776999999, 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039127038000000003, 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074530+0000", 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099434+0000", 2024-09-13T07:01:53.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.024903926999999999 2024-09-13T07:01:53.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099459+0000", 2024-09-13T07:01:53.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5384999999999998e-05 2024-09-13T07:01:53.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099469+0000", 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.024e-05 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099566+0000", 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6533999999999994e-05 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099600+0000", 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4007000000000002e-05 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:53.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113477+0000", 2024-09-13T07:01:53.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01387741 2024-09-13T07:01:53.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113658+0000", 2024-09-13T07:01:53.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000180953 2024-09-13T07:01:53.050 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.050 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.050 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.050 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3187 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T07:01:53.051 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.410269+0000", 2024-09-13T07:01:53.051 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.614238352000001, 2024-09-13T07:01:53.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012395160000000001, 2024-09-13T07:01:53.051 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.051 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:53.051 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.051 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.051 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.051 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3187 2024-09-13T07:01:53.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.051 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410269+0000", 2024-09-13T07:01:53.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410269+0000", 2024-09-13T07:01:53.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410266+0000", 2024-09-13T07:01:53.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:53.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410283+0000", 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7527999999999999e-05 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410317+0000", 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3726000000000002e-05 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410340+0000", 2024-09-13T07:01:53.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3252e-05 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410425+0000", 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4338999999999993e-05 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.410528+0000", 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103058 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:01:53.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.412544+0000", 2024-09-13T07:01:53.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020164639999999999 2024-09-13T07:01:53.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:53.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422499+0000", 2024-09-13T07:01:53.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099549960000000007 2024-09-13T07:01:53.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:53.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422521+0000", 2024-09-13T07:01:53.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1433e-05 2024-09-13T07:01:53.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:53.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422604+0000", 2024-09-13T07:01:53.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2820999999999999e-05 2024-09-13T07:01:53.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.422664+0000", 2024-09-13T07:01:53.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0362999999999998e-05 2024-09-13T07:01:53.056 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.056 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.056 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.057 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3975 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:53.057 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:53.057 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.370111828999999, 2024-09-13T07:01:53.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011993248, 2024-09-13T07:01:53.057 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.057 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.057 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3975 2024-09-13T07:01:53.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.057 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:53.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:53.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654395+0000", 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654398+0000", 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8619999999999996e-06 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654408+0000", 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8209999999999997e-06 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654416+0000", 2024-09-13T07:01:53.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0180000000000001e-06 2024-09-13T07:01:53.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666002+0000", 2024-09-13T07:01:53.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011586008 2024-09-13T07:01:53.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666074+0000", 2024-09-13T07:01:53.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1500000000000003e-05 2024-09-13T07:01:53.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666388+0000", 2024-09-13T07:01:53.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00031467300000000001 2024-09-13T07:01:53.061 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.061 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.061 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.061 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4022 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:53.061 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:01:53.061 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.370083333, 2024-09-13T07:01:53.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014992557, 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4022 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654423+0000", 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654427+0000", 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0960000000000003e-06 2024-09-13T07:01:53.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654437+0000", 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7070000000000004e-06 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654445+0000", 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9049999999999997e-06 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666423+0000", 2024-09-13T07:01:53.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011978223 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666521+0000", 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7832999999999997e-05 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669416+0000", 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002895393 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.066 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4023 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:53.066 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:01:53.066 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.370054447000001, 2024-09-13T07:01:53.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015452399, 2024-09-13T07:01:53.066 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.066 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.066 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.066 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.066 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.066 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4023 2024-09-13T07:01:53.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.066 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654452+0000", 2024-09-13T07:01:53.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654456+0000", 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7570000000000002e-06 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654465+0000", 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7140000000000005e-06 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654486+0000", 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0290000000000001e-05 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669455+0000", 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014969651 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669722+0000", 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026677899999999997 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669905+0000", 2024-09-13T07:01:53.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018285999999999999 2024-09-13T07:01:53.070 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.070 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.070 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.070 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4024 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:53.070 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:53.070 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.370013647, 2024-09-13T07:01:53.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015682979, 2024-09-13T07:01:53.070 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.070 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.071 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.071 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.071 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.071 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4024 2024-09-13T07:01:53.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.071 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:53.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654497+0000", 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4800000000000003e-06 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654507+0000", 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8749999999999995e-06 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654515+0000", 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0350000000000001e-06 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669936+0000", 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015420916 2024-09-13T07:01:53.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670030+0000", 2024-09-13T07:01:53.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3712e-05 2024-09-13T07:01:53.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670176+0000", 2024-09-13T07:01:53.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000146613 2024-09-13T07:01:53.074 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.074 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.074 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.075 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4025 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:53.075 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:53.075 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.369984638999998, 2024-09-13T07:01:53.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016683559000000001, 2024-09-13T07:01:53.075 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.075 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.075 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.075 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.075 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.075 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4025 2024-09-13T07:01:53.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.075 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:53.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:53.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:01:53.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654526+0000", 2024-09-13T07:01:53.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-13T07:01:53.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654535+0000", 2024-09-13T07:01:53.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4199999999999996e-06 2024-09-13T07:01:53.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654543+0000", 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670307+0000", 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015763482999999998 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670399+0000", 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2068e-05 2024-09-13T07:01:53.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671206+0000", 2024-09-13T07:01:53.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000807352 2024-09-13T07:01:53.079 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.079 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.079 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.079 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4026 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 13238272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:53.079 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:53.079 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.369956909999999, 2024-09-13T07:01:53.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016876344000000001, 2024-09-13T07:01:53.080 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.080 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.080 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.080 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.080 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.080 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4026 2024-09-13T07:01:53.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.080 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:53.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:53.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:01:53.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654554+0000", 2024-09-13T07:01:53.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1119999999999998e-06 2024-09-13T07:01:53.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654564+0000", 2024-09-13T07:01:53.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0327e-05 2024-09-13T07:01:53.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654571+0000", 2024-09-13T07:01:53.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3440000000000003e-06 2024-09-13T07:01:53.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671239+0000", 2024-09-13T07:01:53.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016668039999999999 2024-09-13T07:01:53.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671327+0000", 2024-09-13T07:01:53.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7818999999999994e-05 2024-09-13T07:01:53.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671427+0000", 2024-09-13T07:01:53.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9324e-05 2024-09-13T07:01:53.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.084 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.084 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4027 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:53.084 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:01:53.084 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.369928148, 2024-09-13T07:01:53.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017835597000000002, 2024-09-13T07:01:53.085 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.085 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.085 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4027 2024-09-13T07:01:53.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.085 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:01:53.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:01:53.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654578+0000", 2024-09-13T07:01:53.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654590+0000", 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1595e-05 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654601+0000", 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0827e-05 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654609+0000", 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4919999999999993e-06 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672157+0000", 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017547770000000001 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672234+0000", 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7509999999999995e-05 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672415+0000", 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018005500000000001 2024-09-13T07:01:53.088 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11064 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.607882+0000", 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.416625550999999, 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012098661, 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.089 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11064 2024-09-13T07:01:53.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.090 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607882+0000", 2024-09-13T07:01:53.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607882+0000", 2024-09-13T07:01:53.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607881+0000", 2024-09-13T07:01:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607885+0000", 2024-09-13T07:01:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5530000000000003e-06 2024-09-13T07:01:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607896+0000", 2024-09-13T07:01:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1073999999999999e-05 2024-09-13T07:01:53.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.607905+0000", 2024-09-13T07:01:53.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8310000000000007e-06 2024-09-13T07:01:53.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.617863+0000", 2024-09-13T07:01:53.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099584400000000007 2024-09-13T07:01:53.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.617953+0000", 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0050999999999996e-05 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.619980+0000", 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002026721 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11151 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:53.093 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608582+0000", 2024-09-13T07:01:53.094 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.415924996999999, 2024-09-13T07:01:53.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012261311, 2024-09-13T07:01:53.094 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.094 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.094 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.094 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.094 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.094 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11151 2024-09-13T07:01:53.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.094 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608582+0000", 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608582+0000", 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608581+0000", 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608586+0000", 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1289999999999998e-06 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608596+0000", 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0787e-05 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608618+0000", 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1971e-05 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620453+0000", 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011834829999999999 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620542+0000", 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8947000000000001e-05 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620843+0000", 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00030128199999999999 2024-09-13T07:01:53.097 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11152 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608627+0000", 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.415880156, 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012436806, 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.098 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11152 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608627+0000", 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608627+0000", 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608626+0000", 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608631+0000", 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4109999999999998e-06 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608642+0000", 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1158000000000001e-05 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608650+0000", 2024-09-13T07:01:53.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5110000000000003e-06 2024-09-13T07:01:53.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620871+0000", 2024-09-13T07:01:53.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012221019 2024-09-13T07:01:53.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620956+0000", 2024-09-13T07:01:53.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5223999999999996e-05 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621064+0000", 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010727599999999999 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11153 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608658+0000", 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.415849041, 2024-09-13T07:01:53.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012691595, 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11153 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608658+0000", 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608658+0000", 2024-09-13T07:01:53.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608657+0000", 2024-09-13T07:01:53.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608661+0000", 2024-09-13T07:01:53.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7629999999999998e-06 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608672+0000", 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0732e-05 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608680+0000", 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8429999999999993e-06 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621106+0000", 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01242594 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621190+0000", 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4005000000000006e-05 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621350+0000", 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000699999999999 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.106 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11173 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608785+0000", 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.415722495000001, 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012946599, 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11173 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.107 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608785+0000", 2024-09-13T07:01:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608785+0000", 2024-09-13T07:01:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608784+0000", 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608788+0000", 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5070000000000002e-06 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608799+0000", 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0431000000000001e-05 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608807+0000", 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5220000000000007e-06 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621397+0000", 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01258975 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621478+0000", 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0408000000000007e-05 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621731+0000", 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025365899999999999 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11174 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.415691577, 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01318603, 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.111 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.112 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.112 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.112 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.112 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11174 2024-09-13T07:01:53.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.112 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:01:53.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608819+0000", 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9740000000000004e-06 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608829+0000", 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0383999999999999e-05 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608838+0000", 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6719999999999993e-06 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621751+0000", 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012913017000000001 2024-09-13T07:01:53.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621809+0000", 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8505000000000003e-05 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622002+0000", 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192141 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.116 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11175 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:53.116 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:01:53.116 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.415661717000001, 2024-09-13T07:01:53.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013984661000000001, 2024-09-13T07:01:53.116 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.116 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:53.116 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.116 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.116 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.116 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11175 2024-09-13T07:01:53.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.116 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608849+0000", 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8449999999999996e-06 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608859+0000", 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0335e-05 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608868+0000", 2024-09-13T07:01:53.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6070000000000008e-06 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622108+0000", 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013240676 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622223+0000", 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115277 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622830+0000", 2024-09-13T07:01:53.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00060653900000000001 2024-09-13T07:01:53.120 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.120 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.120 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.120 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12393 3.b 3:d3c3e30e:::rbd_data.11945fc173dd.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:53.120 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:01:53.120 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.752042844, 2024-09-13T07:01:53.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027522214, 2024-09-13T07:01:53.120 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:53.120 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:53.120 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12393 2024-09-13T07:01:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:01:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272462+0000", 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272471+0000", 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8510000000000005e-06 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:53.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272486+0000", 2024-09-13T07:01:53.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5409000000000001e-05 2024-09-13T07:01:53.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272494+0000", 2024-09-13T07:01:53.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0020000000000006e-06 2024-09-13T07:01:53.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:53.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272575+0000", 2024-09-13T07:01:53.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0467000000000001e-05 2024-09-13T07:01:53.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:53.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272710+0000", 2024-09-13T07:01:53.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135554 2024-09-13T07:01:53.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:53.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.279539+0000", 2024-09-13T07:01:53.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068285439999999998 2024-09-13T07:01:53.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:53.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299872+0000", 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020332507 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299920+0000", 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8009000000000001e-05 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299956+0000", 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6554999999999998e-05 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:53.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:53.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299986+0000", 2024-09-13T07:01:53.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0331000000000001e-05 2024-09-13T07:01:53.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.126 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:53.126 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:53.126 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:53.127 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:01:53.128 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:01:53.129 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:53.125+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:53.229 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:53.225+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:53.248 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone14 from 0xcbd2568 (overlap 0xc7259b) to 0x8b671f 2024-09-13T07:01:53.248 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:53.249 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:53.259 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:01:53.330 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:53.329+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:53.360 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:53.361 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:53.361 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:53.361 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:53.361 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:53.371 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:01:53.398 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-13T07:01:53.431 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:53.429+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:53.480 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:53.480 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:53.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:53.480 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:53.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.481 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1524 3.14 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:53.481 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:01:53.481 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.433368928, 2024-09-13T07:01:53.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061190880000000003, 2024-09-13T07:01:53.481 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.481 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:53.481 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.481 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.481 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.481 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1524 2024-09-13T07:01:53.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.481 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:01:53.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:01:53.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044767+0000", 2024-09-13T07:01:53.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:53.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052670+0000", 2024-09-13T07:01:53.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079022450000000004 2024-09-13T07:01:53.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052682+0000", 2024-09-13T07:01:53.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2336e-05 2024-09-13T07:01:53.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052696+0000", 2024-09-13T07:01:53.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-13T07:01:53.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052755+0000", 2024-09-13T07:01:53.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9298e-05 2024-09-13T07:01:53.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052947+0000", 2024-09-13T07:01:53.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019153900000000001 2024-09-13T07:01:53.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:53.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.053694+0000", 2024-09-13T07:01:53.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074663099999999999 2024-09-13T07:01:53.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:53.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.102148+0000", 2024-09-13T07:01:53.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048454329999999997 2024-09-13T07:01:53.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:53.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105816+0000", 2024-09-13T07:01:53.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036675950000000001 2024-09-13T07:01:53.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105921+0000", 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105874 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105959+0000", 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7991000000000003e-05 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.487 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16541 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 5758976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.488 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.984769+0000", 2024-09-13T07:01:53.488 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4933686789999996, 2024-09-13T07:01:53.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047534260000000002, 2024-09-13T07:01:53.488 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.488 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.488 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16541 2024-09-13T07:01:53.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.488 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984769+0000", 2024-09-13T07:01:53.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984769+0000", 2024-09-13T07:01:53.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984768+0000", 2024-09-13T07:01:53.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984773+0000", 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.357e-06 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984786+0000", 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3943e-05 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984799+0000", 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2472e-05 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032038+0000", 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047239194999999998 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032173+0000", 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013521 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032303+0000", 2024-09-13T07:01:53.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012971799999999999 2024-09-13T07:01:53.492 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.492 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.492 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.492 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16542 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 5783552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.492 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.984806+0000", 2024-09-13T07:01:53.492 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4933314879999999, 2024-09-13T07:01:53.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047678218000000001, 2024-09-13T07:01:53.492 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.492 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.493 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.493 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.493 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.493 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16542 2024-09-13T07:01:53.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.493 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984806+0000", 2024-09-13T07:01:53.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984806+0000", 2024-09-13T07:01:53.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984805+0000", 2024-09-13T07:01:53.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:53.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984811+0000", 2024-09-13T07:01:53.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:01:53.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984827+0000", 2024-09-13T07:01:53.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6667000000000002e-05 2024-09-13T07:01:53.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984835+0000", 2024-09-13T07:01:53.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.165e-06 2024-09-13T07:01:53.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032327+0000", 2024-09-13T07:01:53.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047492151000000003 2024-09-13T07:01:53.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032407+0000", 2024-09-13T07:01:53.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9782000000000003e-05 2024-09-13T07:01:53.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032484+0000", 2024-09-13T07:01:53.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.771e-05 2024-09-13T07:01:53.496 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16543 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.984842+0000", 2024-09-13T07:01:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4932959600000002, 2024-09-13T07:01:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047953233999999997, 2024-09-13T07:01:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.498 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16543 2024-09-13T07:01:53.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.498 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984842+0000", 2024-09-13T07:01:53.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984842+0000", 2024-09-13T07:01:53.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984841+0000", 2024-09-13T07:01:53.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984845+0000", 2024-09-13T07:01:53.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7689999999999998e-06 2024-09-13T07:01:53.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984855+0000", 2024-09-13T07:01:53.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0484e-05 2024-09-13T07:01:53.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.984869+0000", 2024-09-13T07:01:53.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4004e-05 2024-09-13T07:01:53.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032542+0000", 2024-09-13T07:01:53.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047672288 2024-09-13T07:01:53.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032667+0000", 2024-09-13T07:01:53.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012577000000000001 2024-09-13T07:01:53.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.032795+0000", 2024-09-13T07:01:53.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000127458 2024-09-13T07:01:53.502 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.502 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.502 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.503 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16556 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6127616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.503 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988454+0000", 2024-09-13T07:01:53.503 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4896834270000001, 2024-09-13T07:01:53.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047410069999999999, 2024-09-13T07:01:53.503 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.503 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.503 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16556 2024-09-13T07:01:53.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.503 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988454+0000", 2024-09-13T07:01:53.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988454+0000", 2024-09-13T07:01:53.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988454+0000", 2024-09-13T07:01:53.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988455+0000", 2024-09-13T07:01:53.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.491e-06 2024-09-13T07:01:53.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988458+0000", 2024-09-13T07:01:53.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0419999999999998e-06 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988461+0000", 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0070000000000002e-06 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.035709+0000", 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047248050999999999 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.035786+0000", 2024-09-13T07:01:53.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6477999999999995e-05 2024-09-13T07:01:53.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.035864+0000", 2024-09-13T07:01:53.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8246999999999998e-05 2024-09-13T07:01:53.507 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.507 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.507 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.507 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16557 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6152192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988466+0000", 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4896715870000001, 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047718719999999999, 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16557 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988466+0000", 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988466+0000", 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988466+0000", 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988467+0000", 2024-09-13T07:01:53.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.48e-06 2024-09-13T07:01:53.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988476+0000", 2024-09-13T07:01:53.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4549999999999995e-06 2024-09-13T07:01:53.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988484+0000", 2024-09-13T07:01:53.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6649999999999992e-06 2024-09-13T07:01:53.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.035920+0000", 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047435828999999999 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036042+0000", 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012243000000000001 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036185+0000", 2024-09-13T07:01:53.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014214100000000001 2024-09-13T07:01:53.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.512 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.512 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16558 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6176768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.512 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988487+0000", 2024-09-13T07:01:53.512 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4896503379999997, 2024-09-13T07:01:53.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047874958000000002, 2024-09-13T07:01:53.512 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.512 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.512 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16558 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988487+0000", 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988487+0000", 2024-09-13T07:01:53.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988487+0000", 2024-09-13T07:01:53.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988488+0000", 2024-09-13T07:01:53.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.525e-06 2024-09-13T07:01:53.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988493+0000", 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5719999999999996e-06 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988498+0000", 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036208+0000", 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047710474000000003 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036285+0000", 2024-09-13T07:01:53.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6798000000000006e-05 2024-09-13T07:01:53.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036362+0000", 2024-09-13T07:01:53.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7107999999999994e-05 2024-09-13T07:01:53.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.516 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16559 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988549+0000", 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4895885260000004, 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048124801000000002, 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16559 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988549+0000", 2024-09-13T07:01:53.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988549+0000", 2024-09-13T07:01:53.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988548+0000", 2024-09-13T07:01:53.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988552+0000", 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5980000000000001e-06 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988564+0000", 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.181e-05 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988573+0000", 2024-09-13T07:01:53.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4069999999999999e-06 2024-09-13T07:01:53.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036419+0000", 2024-09-13T07:01:53.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047845461999999998 2024-09-13T07:01:53.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036543+0000", 2024-09-13T07:01:53.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012405000000000001 2024-09-13T07:01:53.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036674+0000", 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013107599999999999 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16560 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6225920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988579+0000", 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4895587780000001, 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048274951000000003, 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16560 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988579+0000", 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988579+0000", 2024-09-13T07:01:53.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988578+0000", 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988581+0000", 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0589999999999998e-06 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988593+0000", 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2058e-05 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988605+0000", 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1206000000000001e-05 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036697+0000", 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048092477000000002 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036778+0000", 2024-09-13T07:01:53.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0766000000000006e-05 2024-09-13T07:01:53.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036854+0000", 2024-09-13T07:01:53.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5852999999999994e-05 2024-09-13T07:01:53.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.525 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.525 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16561 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6250496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.526 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988610+0000", 2024-09-13T07:01:53.526 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4895276050000001, 2024-09-13T07:01:53.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048548085999999997, 2024-09-13T07:01:53.526 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.526 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.526 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16561 2024-09-13T07:01:53.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.526 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988610+0000", 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988610+0000", 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988610+0000", 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988614+0000", 2024-09-13T07:01:53.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0949999999999998e-06 2024-09-13T07:01:53.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988625+0000", 2024-09-13T07:01:53.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1347e-05 2024-09-13T07:01:53.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988631+0000", 2024-09-13T07:01:53.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0639999999999999e-06 2024-09-13T07:01:53.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.036910+0000", 2024-09-13T07:01:53.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048278910000000001 2024-09-13T07:01:53.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.037030+0000", 2024-09-13T07:01:53.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119912 2024-09-13T07:01:53.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.037158+0000", 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128121 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16562 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6275072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.988636+0000", 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4895018350000004, 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048699601000000002, 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.531 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16562 2024-09-13T07:01:53.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.531 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988636+0000", 2024-09-13T07:01:53.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988636+0000", 2024-09-13T07:01:53.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988635+0000", 2024-09-13T07:01:53.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988638+0000", 2024-09-13T07:01:53.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9440000000000001e-06 2024-09-13T07:01:53.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988652+0000", 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3638000000000001e-05 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.988662+0000", 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0447e-05 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.037180+0000", 2024-09-13T07:01:53.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048517376000000001 2024-09-13T07:01:53.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.037261+0000", 2024-09-13T07:01:53.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1143000000000006e-05 2024-09-13T07:01:53.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.037335+0000", 2024-09-13T07:01:53.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4466000000000002e-05 2024-09-13T07:01:53.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.534 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.535 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16576 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.535 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992577+0000", 2024-09-13T07:01:53.535 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4855605929999998, 2024-09-13T07:01:53.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047360526, 2024-09-13T07:01:53.535 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.535 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.535 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.535 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.535 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.535 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16576 2024-09-13T07:01:53.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.536 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992577+0000", 2024-09-13T07:01:53.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992577+0000", 2024-09-13T07:01:53.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992576+0000", 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992585+0000", 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3999999999999992e-06 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992591+0000", 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.409e-06 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992601+0000", 2024-09-13T07:01:53.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8649999999999996e-06 2024-09-13T07:01:53.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.039790+0000", 2024-09-13T07:01:53.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047188859 2024-09-13T07:01:53.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.039865+0000", 2024-09-13T07:01:53.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5117999999999995e-05 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.039937+0000", 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2298000000000005e-05 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16577 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6643712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992607+0000", 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4855308679999997, 2024-09-13T07:01:53.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047460953, 2024-09-13T07:01:53.540 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.540 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.540 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16577 2024-09-13T07:01:53.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.540 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992607+0000", 2024-09-13T07:01:53.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992607+0000", 2024-09-13T07:01:53.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992606+0000", 2024-09-13T07:01:53.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992609+0000", 2024-09-13T07:01:53.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0249999999999998e-06 2024-09-13T07:01:53.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992621+0000", 2024-09-13T07:01:53.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1379999999999999e-05 2024-09-13T07:01:53.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992626+0000", 2024-09-13T07:01:53.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2240000000000003e-06 2024-09-13T07:01:53.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.039968+0000", 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047342238000000002 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040019+0000", 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0948000000000001e-05 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040068+0000", 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8628000000000003e-05 2024-09-13T07:01:53.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.544 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.544 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.544 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16578 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6668288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.544 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992631+0000", 2024-09-13T07:01:53.544 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4855068940000002, 2024-09-13T07:01:53.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047631675999999998, 2024-09-13T07:01:53.544 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.544 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.545 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16578 2024-09-13T07:01:53.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.545 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992631+0000", 2024-09-13T07:01:53.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992631+0000", 2024-09-13T07:01:53.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992630+0000", 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992633+0000", 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992645+0000", 2024-09-13T07:01:53.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2169e-05 2024-09-13T07:01:53.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992654+0000", 2024-09-13T07:01:53.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8370000000000003e-06 2024-09-13T07:01:53.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040107+0000", 2024-09-13T07:01:53.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047452544999999999 2024-09-13T07:01:53.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040187+0000", 2024-09-13T07:01:53.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0382000000000004e-05 2024-09-13T07:01:53.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040262+0000", 2024-09-13T07:01:53.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5244e-05 2024-09-13T07:01:53.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.548 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.548 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16579 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6692864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992659+0000", 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4854780539999997, 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047731126999999998, 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16579 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992659+0000", 2024-09-13T07:01:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992659+0000", 2024-09-13T07:01:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992659+0000", 2024-09-13T07:01:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992662+0000", 2024-09-13T07:01:53.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0800000000000002e-06 2024-09-13T07:01:53.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992673+0000", 2024-09-13T07:01:53.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1338999999999999e-05 2024-09-13T07:01:53.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992682+0000", 2024-09-13T07:01:53.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5609999999999999e-06 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040294+0000", 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047611852000000003 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040345+0000", 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1001999999999997e-05 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040391+0000", 2024-09-13T07:01:53.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5719000000000001e-05 2024-09-13T07:01:53.553 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.553 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.553 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.553 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16580 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.553 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992728+0000", 2024-09-13T07:01:53.553 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4854098640000002, 2024-09-13T07:01:53.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047850404999999999, 2024-09-13T07:01:53.553 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.553 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16580 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992728+0000", 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992728+0000", 2024-09-13T07:01:53.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992727+0000", 2024-09-13T07:01:53.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992731+0000", 2024-09-13T07:01:53.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3469999999999999e-06 2024-09-13T07:01:53.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992740+0000", 2024-09-13T07:01:53.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3279999999999995e-06 2024-09-13T07:01:53.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992752+0000", 2024-09-13T07:01:53.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2272e-05 2024-09-13T07:01:53.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040430+0000", 2024-09-13T07:01:53.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047678164000000002 2024-09-13T07:01:53.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040506+0000", 2024-09-13T07:01:53.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5539999999999998e-05 2024-09-13T07:01:53.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040578+0000", 2024-09-13T07:01:53.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2083999999999996e-05 2024-09-13T07:01:53.558 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.558 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.558 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.558 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16581 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6742016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.558 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992756+0000", 2024-09-13T07:01:53.558 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4853813410000001, 2024-09-13T07:01:53.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047949586000000002, 2024-09-13T07:01:53.558 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.558 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.559 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.559 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16581 2024-09-13T07:01:53.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.559 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992756+0000", 2024-09-13T07:01:53.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992756+0000", 2024-09-13T07:01:53.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992756+0000", 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992758+0000", 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.057e-06 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992770+0000", 2024-09-13T07:01:53.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1936000000000001e-05 2024-09-13T07:01:53.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992777+0000", 2024-09-13T07:01:53.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6479999999999997e-06 2024-09-13T07:01:53.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040607+0000", 2024-09-13T07:01:53.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047830841999999998 2024-09-13T07:01:53.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040657+0000", 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9181000000000002e-05 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040706+0000", 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9190000000000002e-05 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.562 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16582 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6766592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992780+0000", 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4853575169999997, 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048118462000000001, 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16582 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992780+0000", 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992780+0000", 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992780+0000", 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992782+0000", 2024-09-13T07:01:53.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1059999999999998e-06 2024-09-13T07:01:53.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992785+0000", 2024-09-13T07:01:53.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2990000000000001e-06 2024-09-13T07:01:53.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992793+0000", 2024-09-13T07:01:53.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3450000000000006e-06 2024-09-13T07:01:53.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040746+0000", 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047952616000000003 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040825+0000", 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9340999999999998e-05 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040898+0000", 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3195000000000006e-05 2024-09-13T07:01:53.566 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16583 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6791168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992796+0000", 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4853412849999996, 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048227689999999997, 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.567 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16583 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992796+0000", 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992796+0000", 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992796+0000", 2024-09-13T07:01:53.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:53.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992798+0000", 2024-09-13T07:01:53.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.782e-06 2024-09-13T07:01:53.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992801+0000", 2024-09-13T07:01:53.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1030000000000002e-06 2024-09-13T07:01:53.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992804+0000", 2024-09-13T07:01:53.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9469999999999999e-06 2024-09-13T07:01:53.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040929+0000", 2024-09-13T07:01:53.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048125384 2024-09-13T07:01:53.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.040979+0000", 2024-09-13T07:01:53.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9625e-05 2024-09-13T07:01:53.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.041024+0000", 2024-09-13T07:01:53.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5105999999999998e-05 2024-09-13T07:01:53.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.571 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.571 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16584 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:53.571 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.992806+0000", 2024-09-13T07:01:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4853313090000002, 2024-09-13T07:01:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048406193, 2024-09-13T07:01:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16584 2024-09-13T07:01:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992806+0000", 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992806+0000", 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992806+0000", 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992807+0000", 2024-09-13T07:01:53.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.514e-06 2024-09-13T07:01:53.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992811+0000", 2024-09-13T07:01:53.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0359999999999998e-06 2024-09-13T07:01:53.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.992813+0000", 2024-09-13T07:01:53.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.914e-06 2024-09-13T07:01:53.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.041061+0000", 2024-09-13T07:01:53.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048247440000000003 2024-09-13T07:01:53.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.041139+0000", 2024-09-13T07:01:53.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7723000000000001e-05 2024-09-13T07:01:53.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.041212+0000", 2024-09-13T07:01:53.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3792000000000001e-05 2024-09-13T07:01:53.576 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.576 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.576 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.576 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:53.576 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:53.576 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:53.577 INFO:teuthology.orchestra.run.smithi118.stdout:1212 read 0x841fc75 thru 0x8428e8c (0x9218 bytes) 2024-09-13T07:01:53.577 INFO:teuthology.orchestra.run.smithi118.stdout:1214 punch from 0xa496a5d to 0xa49ef67, (0x850a bytes) 2024-09-13T07:01:53.578 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:01:53.578 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:53.574+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:53.581 INFO:teuthology.orchestra.run.smithi118.stdout:1215 read 0x1f665fb thru 0x1f6c5b1 (0x5fb7 bytes) 2024-09-13T07:01:53.582 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:53.582 INFO:teuthology.orchestra.run.smithi118.stdout:1218 read 0x31ad4a2 thru 0x31aff15 (0x2a74 bytes) 2024-09-13T07:01:53.583 INFO:teuthology.orchestra.run.smithi118.stdout:1219 read 0x4e99ed7 thru 0x4e9cb7e (0x2ca8 bytes) 2024-09-13T07:01:53.586 INFO:teuthology.orchestra.run.smithi118.stdout:1220 trunc from 0xcbd2568 to 0xbea0061 2024-09-13T07:01:53.620 INFO:teuthology.orchestra.run.smithi118.stdout:1221 read 0x71a6811 thru 0x71af723 (0x8f13 bytes) 2024-09-13T07:01:53.620 INFO:teuthology.orchestra.run.smithi118.stdout:1224 write 0x944d321 thru 0x9451486 (0x4166 bytes) 2024-09-13T07:01:53.640 INFO:teuthology.orchestra.run.smithi118.stdout:1226 trunc from 0xbea0061 to 0x26375d6 2024-09-13T07:01:53.678 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:53.677+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:53.756 INFO:teuthology.orchestra.run.smithi118.stdout:1227 read 0x2492906 thru 0x2499049 (0x6744 bytes) 2024-09-13T07:01:53.756 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:53.756 INFO:teuthology.orchestra.run.smithi118.stdout:1229 punch from 0xa24e60 to 0xa2aa75, (0x5c15 bytes) 2024-09-13T07:01:53.756 INFO:teuthology.orchestra.run.smithi118.stdout:1231 writesame 0x12515c8 thru 0x125e492 data_size 0x1(0xcecb bytes) 2024-09-13T07:01:53.758 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:53.758 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:53.768 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:01:53.780 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:53.777+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:53.797 INFO:teuthology.orchestra.run.smithi118.stdout:1232 read 0x25a3c2e thru 0x25b0ece (0xd2a1 bytes) 2024-09-13T07:01:53.798 INFO:teuthology.orchestra.run.smithi118.stdout:1233 read 0x8509e9 thru 0x8594c6 (0x8ade bytes) 2024-09-13T07:01:53.799 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:53.828 INFO:teuthology.orchestra.run.smithi118.stdout:1235 write 0xb2fc18d thru 0xb3091e4 (0xd058 bytes) 2024-09-13T07:01:53.835 INFO:teuthology.orchestra.run.smithi118.stdout:1236 read 0xdbadcd thru 0xdc6400 (0xb634 bytes) 2024-09-13T07:01:53.836 INFO:teuthology.orchestra.run.smithi118.stdout:1237 write 0xa44649c thru 0xa44d3f4 (0x6f59 bytes) 2024-09-13T07:01:53.842 INFO:teuthology.orchestra.run.smithi118.stdout:1238 read 0x50668a thru 0x512530 (0xbea7 bytes) 2024-09-13T07:01:53.848 INFO:teuthology.orchestra.run.smithi118.stdout:1239 trunc from 0xb3091e5 to 0x8c6b5af 2024-09-13T07:01:53.864 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:53.864 INFO:teuthology.orchestra.run.smithi118.stdout:1242 read 0x76959a8 thru 0x76a1181 (0xb7da bytes) 2024-09-13T07:01:53.864 INFO:teuthology.orchestra.run.smithi118.stdout:1243 writesame 0x406a17f thru 0x406abc5 data_size 0x1(0xa47 bytes) 2024-09-13T07:01:53.870 INFO:teuthology.orchestra.run.smithi118.stdout:1244 read 0x67b558e thru 0x67bb853 (0x62c6 bytes) 2024-09-13T07:01:53.870 INFO:teuthology.orchestra.run.smithi118.stdout:1245 punch from 0x3c3dddc to 0x3c44383, (0x65a7 bytes) 2024-09-13T07:01:53.871 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:53.871 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:53.871 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:53.871 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:53.871 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:53.881 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:01:53.882 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:53.878+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:53.882 INFO:teuthology.orchestra.run.smithi118.stdout:1246 trunc from 0x8c6b5af to 0x3adb1dc 2024-09-13T07:01:53.934 INFO:teuthology.orchestra.run.smithi118.stdout:1247 write 0xba08402 thru 0xba08cb7 (0x8b6 bytes) 2024-09-13T07:01:53.939 INFO:teuthology.orchestra.run.smithi118.stdout:1248 punch from 0x266cbf4 to 0x266d0f3, (0x4ff bytes) 2024-09-13T07:01:53.939 INFO:teuthology.orchestra.run.smithi118.stdout:1249 punch from 0x2fc90b5 to 0x2fcc27a, (0x31c5 bytes) 2024-09-13T07:01:53.939 INFO:teuthology.orchestra.run.smithi118.stdout:1250 writesame 0xe5b0b9 thru 0xe68846 data_size 0x33(0xd78e bytes) 2024-09-13T07:01:53.957 INFO:teuthology.orchestra.run.smithi118.stdout:1251 trunc from 0xba08cb8 to 0x4ccba09 2024-09-13T07:01:53.982 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:53.978+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:53.993 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:53.994 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:53.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:53.994 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:53.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.994 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1531 3.c 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:53.994 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:53.994 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.941982680999999, 2024-09-13T07:01:53.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063168486999999995, 2024-09-13T07:01:53.994 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:53.994 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:53.994 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:53.994 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:53.994 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1531 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:53.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049511+0000", 2024-09-13T07:01:53.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:53.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:53.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069234+0000", 2024-09-13T07:01:53.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019723920999999998 2024-09-13T07:01:53.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:53.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069269+0000", 2024-09-13T07:01:53.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4446999999999997e-05 2024-09-13T07:01:53.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:53.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069286+0000", 2024-09-13T07:01:53.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7405999999999999e-05 2024-09-13T07:01:53.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:53.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069370+0000", 2024-09-13T07:01:53.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-05 2024-09-13T07:01:53.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069511+0000", 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014134299999999999 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070463+0000", 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095167899999999998 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105076+0000", 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034613139000000001 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:53.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112562+0000", 2024-09-13T07:01:53.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074861429999999998 2024-09-13T07:01:53.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:53.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112631+0000", 2024-09-13T07:01:53.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:01:53.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:53.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:53.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:53.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112680+0000", 2024-09-13T07:01:53.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9607000000000001e-05 2024-09-13T07:01:53.999 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.000 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.000 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7628 3.1f 3:fdf2e50b:::rbd_data.1194c5936dae.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1032192~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:01:54.000 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:01:54.000 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.229056524000001, 2024-09-13T07:01:54.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.630407088, 2024-09-13T07:01:54.000 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.000 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.001 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7628 2024-09-13T07:01:54.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.001 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:01:54.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:01:54.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762435+0000", 2024-09-13T07:01:54.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:01:54.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762543+0000", 2024-09-13T07:01:54.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010850500000000001 2024-09-13T07:01:54.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762597+0000", 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4262000000000003e-05 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762626+0000", 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9176e-05 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762761+0000", 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013405599999999999 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762776+0000", 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5679999999999999e-05 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388492+0000", 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.62571606300000004 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for missing object", 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388614+0000", 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012126 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390837+0000", 2024-09-13T07:01:54.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002223806 2024-09-13T07:01:54.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390963+0000", 2024-09-13T07:01:54.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012550300000000001 2024-09-13T07:01:54.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-13T07:01:54.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391127+0000", 2024-09-13T07:01:54.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163995 2024-09-13T07:01:54.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391989+0000", 2024-09-13T07:01:54.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086230499999999995 2024-09-13T07:01:54.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392662+0000", 2024-09-13T07:01:54.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067260799999999999 2024-09-13T07:01:54.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392728+0000", 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6501000000000006e-05 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392845+0000", 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116665 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15862 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3440640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.007 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.772590+0000", 2024-09-13T07:01:54.008 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2189045869999999, 2024-09-13T07:01:54.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048639120000000001, 2024-09-13T07:01:54.008 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.008 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.008 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15862 2024-09-13T07:01:54.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.008 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772590+0000", 2024-09-13T07:01:54.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772590+0000", 2024-09-13T07:01:54.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772589+0000", 2024-09-13T07:01:54.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772592+0000", 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772598+0000", 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.463e-06 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772603+0000", 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5450000000000003e-06 2024-09-13T07:01:54.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821064+0000", 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048461046000000001 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821137+0000", 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2507000000000002e-05 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821229+0000", 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2082000000000004e-05 2024-09-13T07:01:54.011 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.012 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.012 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.012 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15863 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3465216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.012 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.772608+0000", 2024-09-13T07:01:54.012 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2188863129999996, 2024-09-13T07:01:54.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048769738, 2024-09-13T07:01:54.012 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.012 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.013 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15863 2024-09-13T07:01:54.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.013 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772608+0000", 2024-09-13T07:01:54.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772608+0000", 2024-09-13T07:01:54.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772607+0000", 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772611+0000", 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3759999999999999e-06 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772616+0000", 2024-09-13T07:01:54.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3970000000000002e-06 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772621+0000", 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8459999999999999e-06 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821253+0000", 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048632149 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821308+0000", 2024-09-13T07:01:54.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4639999999999999e-05 2024-09-13T07:01:54.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821378+0000", 2024-09-13T07:01:54.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9987999999999995e-05 2024-09-13T07:01:54.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.016 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.016 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15870 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3637248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.016 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2175979100000003, 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048771508999999998, 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15870 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:01:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:01:54.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:01:54.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773899+0000", 2024-09-13T07:01:54.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1949999999999999e-06 2024-09-13T07:01:54.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773904+0000", 2024-09-13T07:01:54.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.356e-06 2024-09-13T07:01:54.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773909+0000", 2024-09-13T07:01:54.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0849999999999996e-06 2024-09-13T07:01:54.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822505+0000", 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048595933000000001 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822579+0000", 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4276999999999994e-05 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822668+0000", 2024-09-13T07:01:54.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8522000000000004e-05 2024-09-13T07:01:54.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.021 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.021 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15871 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.021 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:01:54.021 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2175809400000004, 2024-09-13T07:01:54.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048913976999999997, 2024-09-13T07:01:54.021 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.021 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.022 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.022 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15871 2024-09-13T07:01:54.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.022 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:01:54.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:01:54.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773916+0000", 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0369999999999999e-06 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773921+0000", 2024-09-13T07:01:54.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.383e-06 2024-09-13T07:01:54.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773927+0000", 2024-09-13T07:01:54.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.147e-06 2024-09-13T07:01:54.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822692+0000", 2024-09-13T07:01:54.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048765583000000001 2024-09-13T07:01:54.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822753+0000", 2024-09-13T07:01:54.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0637999999999998e-05 2024-09-13T07:01:54.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822827+0000", 2024-09-13T07:01:54.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4630999999999997e-05 2024-09-13T07:01:54.025 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.025 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.025 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.026 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15884 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.026 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.776551+0000", 2024-09-13T07:01:54.026 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2149436810000003, 2024-09-13T07:01:54.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048644055999999998, 2024-09-13T07:01:54.026 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.026 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.026 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15884 2024-09-13T07:01:54.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.026 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776551+0000", 2024-09-13T07:01:54.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776551+0000", 2024-09-13T07:01:54.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776550+0000", 2024-09-13T07:01:54.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776553+0000", 2024-09-13T07:01:54.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1520000000000001e-06 2024-09-13T07:01:54.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776559+0000", 2024-09-13T07:01:54.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4890000000000003e-06 2024-09-13T07:01:54.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776564+0000", 2024-09-13T07:01:54.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0390000000000004e-06 2024-09-13T07:01:54.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825037+0000", 2024-09-13T07:01:54.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048472701 2024-09-13T07:01:54.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825109+0000", 2024-09-13T07:01:54.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1884000000000005e-05 2024-09-13T07:01:54.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825195+0000", 2024-09-13T07:01:54.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6187000000000007e-05 2024-09-13T07:01:54.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.030 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.030 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15885 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 4005888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.030 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.776569+0000", 2024-09-13T07:01:54.030 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2149256770000001, 2024-09-13T07:01:54.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048792239000000001, 2024-09-13T07:01:54.031 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.031 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.031 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15885 2024-09-13T07:01:54.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.031 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776569+0000", 2024-09-13T07:01:54.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776569+0000", 2024-09-13T07:01:54.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776568+0000", 2024-09-13T07:01:54.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776571+0000", 2024-09-13T07:01:54.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0419999999999998e-06 2024-09-13T07:01:54.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776577+0000", 2024-09-13T07:01:54.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5709999999999997e-06 2024-09-13T07:01:54.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776582+0000", 2024-09-13T07:01:54.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9540000000000003e-06 2024-09-13T07:01:54.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825231+0000", 2024-09-13T07:01:54.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048649031000000002 2024-09-13T07:01:54.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825288+0000", 2024-09-13T07:01:54.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7148999999999998e-05 2024-09-13T07:01:54.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825361+0000", 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3133999999999995e-05 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15907 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2108982670000001, 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048713259000000002, 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15907 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780598+0000", 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7e-06 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780604+0000", 2024-09-13T07:01:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-06 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780609+0000", 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6750000000000001e-06 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829162+0000", 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048553600000000002 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829226+0000", 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3547000000000007e-05 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829309+0000", 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3547000000000005e-05 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15908 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.780613+0000", 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2108813539999996, 2024-09-13T07:01:54.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048901932000000002, 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15908 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780613+0000", 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780613+0000", 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780612+0000", 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780615+0000", 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0120000000000001e-06 2024-09-13T07:01:54.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780621+0000", 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8320000000000002e-06 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780626+0000", 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829346+0000", 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048720439999999997 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829419+0000", 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2347000000000004e-05 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829515+0000", 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6181000000000006e-05 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15928 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.043 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2068501100000004, 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048610352000000003, 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15928 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:01:54.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:01:54.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:01:54.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784646+0000", 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3479999999999998e-06 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784652+0000", 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.485e-06 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784657+0000", 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8189999999999999e-06 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833039+0000", 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048382890999999997 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833116+0000", 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6371000000000005e-05 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833255+0000", 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138784 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15929 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784661+0000", 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2068335499999998, 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048815364, 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15929 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.048 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784661+0000", 2024-09-13T07:01:54.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784661+0000", 2024-09-13T07:01:54.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784660+0000", 2024-09-13T07:01:54.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784663+0000", 2024-09-13T07:01:54.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5639999999999999e-06 2024-09-13T07:01:54.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784668+0000", 2024-09-13T07:01:54.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4079999999999997e-06 2024-09-13T07:01:54.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784673+0000", 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5120000000000002e-06 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833284+0000", 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048611047999999997 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833382+0000", 2024-09-13T07:01:54.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7571999999999993e-05 2024-09-13T07:01:54.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833476+0000", 2024-09-13T07:01:54.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4653000000000003e-05 2024-09-13T07:01:54.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.052 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.052 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15930 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.052 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784677+0000", 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2068171230000004, 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048947388000000001, 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15930 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784677+0000", 2024-09-13T07:01:54.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784677+0000", 2024-09-13T07:01:54.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784676+0000", 2024-09-13T07:01:54.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784680+0000", 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1930000000000002e-06 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784685+0000", 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784690+0000", 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6949999999999999e-06 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833496+0000", 2024-09-13T07:01:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048806041000000001 2024-09-13T07:01:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833541+0000", 2024-09-13T07:01:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4886999999999999e-05 2024-09-13T07:01:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833625+0000", 2024-09-13T07:01:54.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4073000000000006e-05 2024-09-13T07:01:54.057 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.057 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.057 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.057 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15931 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.057 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784704+0000", 2024-09-13T07:01:54.057 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2067906690000001, 2024-09-13T07:01:54.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049199247000000002, 2024-09-13T07:01:54.057 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.057 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.057 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.057 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.058 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.058 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15931 2024-09-13T07:01:54.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.058 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784704+0000", 2024-09-13T07:01:54.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784704+0000", 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784703+0000", 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784706+0000", 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1e-06 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784712+0000", 2024-09-13T07:01:54.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8730000000000004e-06 2024-09-13T07:01:54.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784718+0000", 2024-09-13T07:01:54.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3850000000000001e-06 2024-09-13T07:01:54.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833671+0000", 2024-09-13T07:01:54.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048952993 2024-09-13T07:01:54.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833787+0000", 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011663300000000001 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833903+0000", 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115747 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.062 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15951 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.062 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788591+0000", 2024-09-13T07:01:54.062 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2029038500000002, 2024-09-13T07:01:54.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048616683000000001, 2024-09-13T07:01:54.062 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.062 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.064 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15951 2024-09-13T07:01:54.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.064 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788591+0000", 2024-09-13T07:01:54.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788591+0000", 2024-09-13T07:01:54.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788590+0000", 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788593+0000", 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5720000000000001e-06 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788598+0000", 2024-09-13T07:01:54.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4840000000000003e-06 2024-09-13T07:01:54.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788603+0000", 2024-09-13T07:01:54.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8659999999999997e-06 2024-09-13T07:01:54.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837089+0000", 2024-09-13T07:01:54.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048485815000000002 2024-09-13T07:01:54.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837143+0000", 2024-09-13T07:01:54.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3794e-05 2024-09-13T07:01:54.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837207+0000", 2024-09-13T07:01:54.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4566999999999994e-05 2024-09-13T07:01:54.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.067 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15952 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788607+0000", 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2028872970000002, 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048772226000000002, 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15952 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788607+0000", 2024-09-13T07:01:54.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788607+0000", 2024-09-13T07:01:54.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788606+0000", 2024-09-13T07:01:54.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788609+0000", 2024-09-13T07:01:54.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7640000000000001e-06 2024-09-13T07:01:54.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788615+0000", 2024-09-13T07:01:54.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8490000000000002e-06 2024-09-13T07:01:54.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788620+0000", 2024-09-13T07:01:54.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3859999999999998e-06 2024-09-13T07:01:54.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837240+0000", 2024-09-13T07:01:54.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048619860000000001 2024-09-13T07:01:54.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837301+0000", 2024-09-13T07:01:54.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0304999999999999e-05 2024-09-13T07:01:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837379+0000", 2024-09-13T07:01:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8689999999999994e-05 2024-09-13T07:01:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15953 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:01:54.072 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2028579109999997, 2024-09-13T07:01:54.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04888609, 2024-09-13T07:01:54.073 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.073 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.073 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15953 2024-09-13T07:01:54.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.073 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:01:54.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788639+0000", 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4740000000000001e-06 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788646+0000", 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788652+0000", 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7289999999999997e-06 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837400+0000", 2024-09-13T07:01:54.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048747946 2024-09-13T07:01:54.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837455+0000", 2024-09-13T07:01:54.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5621000000000002e-05 2024-09-13T07:01:54.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837523+0000", 2024-09-13T07:01:54.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7158999999999999e-05 2024-09-13T07:01:54.076 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.077 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.077 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.077 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15954 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1507328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.077 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:01:54.077 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2028381340000003, 2024-09-13T07:01:54.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049039567999999999, 2024-09-13T07:01:54.077 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.077 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.078 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15954 2024-09-13T07:01:54.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.078 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:01:54.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:01:54.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:01:54.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788659+0000", 2024-09-13T07:01:54.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1889999999999999e-06 2024-09-13T07:01:54.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788664+0000", 2024-09-13T07:01:54.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-13T07:01:54.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788670+0000", 2024-09-13T07:01:54.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6999999999999996e-06 2024-09-13T07:01:54.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837552+0000", 2024-09-13T07:01:54.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048882402999999998 2024-09-13T07:01:54.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837615+0000", 2024-09-13T07:01:54.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2354000000000006e-05 2024-09-13T07:01:54.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837696+0000", 2024-09-13T07:01:54.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1286999999999996e-05 2024-09-13T07:01:54.081 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.081 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.081 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15977 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.792640+0000", 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.19885412, 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048609181000000001, 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15977 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.792640+0000", 2024-09-13T07:01:54.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.792640+0000", 2024-09-13T07:01:54.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.792639+0000", 2024-09-13T07:01:54.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.792643+0000", 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1879999999999998e-06 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.792648+0000", 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0889999999999999e-06 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.792652+0000", 2024-09-13T07:01:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6539999999999998e-06 2024-09-13T07:01:54.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.841124+0000", 2024-09-13T07:01:54.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048471619000000001 2024-09-13T07:01:54.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.841178+0000", 2024-09-13T07:01:54.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3693999999999998e-05 2024-09-13T07:01:54.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.841249+0000", 2024-09-13T07:01:54.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1729000000000004e-05 2024-09-13T07:01:54.086 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.086 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.086 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.086 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:01:54.086 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:01:54.086 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8729062150000004, 2024-09-13T07:01:54.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:01:54.086 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.086 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:01:54.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:01:54.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:54.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:01:54.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:01:54.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:01:54.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:01:54.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:01:54.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:01:54.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:01:54.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:01:54.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:01:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:01:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:01:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:01:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:01:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:01:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.093 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.093 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:54.095 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:01:54.095 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:54.090+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:54.098 INFO:teuthology.orchestra.run.smithi118.stdout:1252 writesame 0xbb96775 thru 0xbb97aa8 data_size 0x2(0x1334 bytes) 2024-09-13T07:01:54.098 INFO:teuthology.orchestra.run.smithi118.stdout:1254 read 0x2906eab thru 0x290f0f1 (0x8247 bytes) 2024-09-13T07:01:54.098 INFO:teuthology.orchestra.run.smithi118.stdout:1256 write 0x8df38b6 thru 0x8dff50f (0xbc5a bytes) 2024-09-13T07:01:54.098 INFO:teuthology.orchestra.run.smithi118.stdout:1257 read 0x8a3fbd2 thru 0x8a4dbaa (0xdfd9 bytes) 2024-09-13T07:01:54.098 INFO:teuthology.orchestra.run.smithi118.stdout:1258 write 0xcaf781e thru 0xcaf7f05 (0x6e8 bytes) 2024-09-13T07:01:54.098 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:54.098 INFO:teuthology.orchestra.run.smithi118.stdout:1260 punch from 0x7fd5bd7 to 0x7fdfd5d, (0xa186 bytes) 2024-09-13T07:01:54.098 INFO:teuthology.orchestra.run.smithi118.stdout:1262 punch from 0x4217094 to 0x421bc74, (0x4be0 bytes) 2024-09-13T07:01:54.098 INFO:teuthology.orchestra.run.smithi118.stdout:1263 read 0xbaf3807 thru 0xbafff9b (0xc795 bytes) 2024-09-13T07:01:54.098 INFO:teuthology.orchestra.run.smithi118.stdout:1264 read 0xa453d5a thru 0xa45a6ff (0x69a6 bytes) 2024-09-13T07:01:54.098 INFO:teuthology.orchestra.run.smithi118.stdout:1266 write 0x587b6d8 thru 0x58867f2 (0xb11b bytes) 2024-09-13T07:01:54.099 INFO:teuthology.orchestra.run.smithi118.stdout:1268 trunc from 0xcaf7f06 to 0xc36f270 2024-09-13T07:01:54.099 INFO:teuthology.orchestra.run.smithi118.stdout:1269 punch from 0x95d78ce to 0x95d8fa5, (0x16d7 bytes) 2024-09-13T07:01:54.099 INFO:teuthology.orchestra.run.smithi118.stdout:1271 write 0xa5ba04 thru 0xa60675 (0x4c72 bytes) 2024-09-13T07:01:54.099 INFO:teuthology.orchestra.run.smithi118.stdout:1272 read 0x9495642 thru 0x94a0f57 (0xb916 bytes) 2024-09-13T07:01:54.099 INFO:teuthology.orchestra.run.smithi118.stdout:1275 writesame 0x7ca0970 thru 0x7cabc82 data_size 0x6f(0xb313 bytes) 2024-09-13T07:01:54.099 INFO:teuthology.orchestra.run.smithi118.stdout:1279 read 0xa1faa9c thru 0xa1ff21a (0x477f bytes) 2024-09-13T07:01:54.099 INFO:teuthology.orchestra.run.smithi118.stdout:1280 read 0xa52a72b thru 0xa533057 (0x892d bytes) 2024-09-13T07:01:54.099 INFO:teuthology.orchestra.run.smithi118.stdout:1281 write 0x2694bfa thru 0x269e0b2 (0x94b9 bytes) 2024-09-13T07:01:54.099 INFO:teuthology.orchestra.run.smithi118.stdout:1282 punch from 0x8d9a5 to 0x8fbc1, (0x221c bytes) 2024-09-13T07:01:54.099 INFO:teuthology.orchestra.run.smithi118.stdout:1283 write 0x396e608 thru 0x397e3a5 (0xfd9e bytes) 2024-09-13T07:01:54.099 INFO:teuthology.orchestra.run.smithi118.stdout:1284 punch from 0x218015a to 0x2181803, (0x16a9 bytes) 2024-09-13T07:01:54.099 INFO:teuthology.orchestra.run.smithi118.stdout:1285 read 0x3123d3a thru 0x3127180 (0x3447 bytes) 2024-09-13T07:01:54.099 INFO:teuthology.orchestra.run.smithi118.stdout:1286 write 0x1b12962 thru 0x1b199b5 (0x7054 bytes) 2024-09-13T07:01:54.100 INFO:teuthology.orchestra.run.smithi118.stdout:1287 read 0x7e18ffb thru 0x7e26f28 (0xdf2e bytes) 2024-09-13T07:01:54.100 INFO:teuthology.orchestra.run.smithi118.stdout:1288 write 0x4cb7334 thru 0x4cba5b1 (0x327e bytes) 2024-09-13T07:01:54.100 INFO:teuthology.orchestra.run.smithi118.stdout:1290 read 0xc145298 thru 0xc146902 (0x166b bytes) 2024-09-13T07:01:54.100 INFO:teuthology.orchestra.run.smithi118.stdout:1292 read 0x9c222e0 thru 0x9c24ebc (0x2bdd bytes) 2024-09-13T07:01:54.100 INFO:teuthology.orchestra.run.smithi118.stdout:1295 writesame 0x5075a7d thru 0x50782ee data_size 0x2(0x2872 bytes) 2024-09-13T07:01:54.100 INFO:teuthology.orchestra.run.smithi118.stdout:1296 write 0xa2d7a3 thru 0xa3d70e (0xff6c bytes) 2024-09-13T07:01:54.100 INFO:teuthology.orchestra.run.smithi118.stdout:1297 read 0x3db418b thru 0x3dbd394 (0x920a bytes) 2024-09-13T07:01:54.104 INFO:teuthology.orchestra.run.smithi118.stdout:1298 trunc from 0xc36f270 to 0x9e33cc6 2024-09-13T07:01:54.113 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:01:54.128 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:54.139 INFO:teuthology.orchestra.run.smithi118.stdout:1302 write 0xa9e9db6 thru 0xa9f1eb9 (0x8104 bytes) 2024-09-13T07:01:54.144 INFO:teuthology.orchestra.run.smithi118.stdout:1303 write 0x6176b84 thru 0x6181977 (0xadf4 bytes) 2024-09-13T07:01:54.147 INFO:teuthology.orchestra.run.smithi118.stdout:1304 read 0x559eda1 thru 0x55a2a17 (0x3c77 bytes) 2024-09-13T07:01:54.148 INFO:teuthology.orchestra.run.smithi118.stdout:1305 punch from 0x74ec19b to 0x74f37da, (0x763f bytes) 2024-09-13T07:01:54.148 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:54.169 INFO:teuthology.orchestra.run.smithi118.stdout:1307 trunc from 0xa9f1eba to 0xa409f5 2024-09-13T07:01:54.192 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:54.195 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:54.193+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:54.203 INFO:teuthology.orchestra.run.smithi118.stdout:1313 writesame 0x177c154 thru 0x177f840 data_size 0x2b(0x36ed bytes) 2024-09-13T07:01:54.206 INFO:teuthology.orchestra.run.smithi118.stdout:1314 read 0xa47410 thru 0xa53989 (0xc57a bytes) 2024-09-13T07:01:54.207 INFO:teuthology.orchestra.run.smithi118.stdout:1315 read 0x91a441 thru 0x9230ce (0x8c8e bytes) 2024-09-13T07:01:54.208 INFO:teuthology.orchestra.run.smithi118.stdout:1316 write 0xe9377 thru 0xf8896 (0xf520 bytes) 2024-09-13T07:01:54.211 INFO:teuthology.orchestra.run.smithi118.stdout:1317 read 0x5c4f75 thru 0x5c58ad (0x939 bytes) 2024-09-13T07:01:54.212 INFO:teuthology.orchestra.run.smithi118.stdout:1318 read 0x80f19 thru 0x887d1 (0x78b9 bytes) 2024-09-13T07:01:54.221 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:54.221 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:54.221 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:54.221 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:54.230 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:01:54.244 INFO:teuthology.orchestra.run.smithi118.stdout:1319 write 0xa2064ce thru 0xa20f3c4 (0x8ef7 bytes) 2024-09-13T07:01:54.250 INFO:teuthology.orchestra.run.smithi118.stdout:1320 trunc from 0xa20f3c5 to 0xa0e44a0 2024-09-13T07:01:54.262 INFO:teuthology.orchestra.run.smithi118.stdout:1321 clone 16 order 19 su 4096 sc 9 2024-09-13T07:01:54.296 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:54.293+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:54.338 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:54.338 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:54.338 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:54.339 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:54.339 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:54.348 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:01:54.397 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:54.397+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:54.447 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:54.448 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:54.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:54.448 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:54.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.448 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:54.448 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:54.448 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.387130829, 2024-09-13T07:01:54.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:01:54.448 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.448 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.448 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.449 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:01:54.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.449 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:54.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:54.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:01:54.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:01:54.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:01:54.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:01:54.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:01:54.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:01:54.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:54.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:01:54.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:01:54.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:01:54.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:01:54.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:01:54.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:01:54.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:01:54.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:01:54.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1534 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14365141] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.380303278, 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040478761000000002, 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.454 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.455 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.455 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.455 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1534 2024-09-13T07:01:54.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.455 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:54.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065747+0000", 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075258+0000", 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095108710000000006 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075288+0000", 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0085000000000001e-05 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075305+0000", 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7748e-05 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075367+0000", 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1867999999999998e-05 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075520+0000", 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015298800000000001 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.076247+0000", 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072720499999999997 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105157+0000", 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028909642999999999 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106124+0000", 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096729400000000005 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106173+0000", 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8196999999999999e-05 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106227+0000", 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4388999999999997e-05 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.459 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1557 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.305537608000002, 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012339057, 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1557 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.460 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:54.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:54.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140502+0000", 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999881 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140526+0000", 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4049999999999998e-05 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140563+0000", 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6569000000000002e-05 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140580+0000", 2024-09-13T07:01:54.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7725000000000001e-05 2024-09-13T07:01:54.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140942+0000", 2024-09-13T07:01:54.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036136300000000002 2024-09-13T07:01:54.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.141130+0000", 2024-09-13T07:01:54.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018814999999999999 2024-09-13T07:01:54.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:54.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.141433+0000", 2024-09-13T07:01:54.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030329899999999998 2024-09-13T07:01:54.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.147925+0000", 2024-09-13T07:01:54.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064915240000000003 2024-09-13T07:01:54.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152757+0000", 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048328709999999999 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152802+0000", 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4314000000000003e-05 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152853+0000", 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1149000000000001e-05 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1558 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.305439870000001, 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012353627000000001, 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1558 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:54.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:54.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140610+0000", 2024-09-13T07:01:54.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:54.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140619+0000", 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8650000000000007e-06 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140634+0000", 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4515000000000001e-05 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140646+0000", 2024-09-13T07:01:54.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1736000000000001e-05 2024-09-13T07:01:54.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.144258+0000", 2024-09-13T07:01:54.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036123499999999998 2024-09-13T07:01:54.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.145034+0000", 2024-09-13T07:01:54.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00077638900000000005 2024-09-13T07:01:54.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:54.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.145395+0000", 2024-09-13T07:01:54.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036117499999999998 2024-09-13T07:01:54.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.149775+0000", 2024-09-13T07:01:54.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043795639999999999 2024-09-13T07:01:54.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152906+0000", 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031306929999999999 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152941+0000", 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4919000000000002e-05 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152965+0000", 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4641e-05 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.473 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1676 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.105161666000001, 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067554429999999999, 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1676 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.474 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:54.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:54.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340889+0000", 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340896+0000", 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1420000000000004e-06 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340909+0000", 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3329000000000001e-05 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340920+0000", 2024-09-13T07:01:54.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1029999999999999e-05 2024-09-13T07:01:54.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341653+0000", 2024-09-13T07:01:54.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073243400000000001 2024-09-13T07:01:54.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341825+0000", 2024-09-13T07:01:54.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017243099999999999 2024-09-13T07:01:54.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.347645+0000", 2024-09-13T07:01:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058200639999999998 2024-09-13T07:01:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1854 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,zero 4194304~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.848041946999999, 2024-09-13T07:01:54.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070961979999999997, 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1854 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:54.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598007+0000", 2024-09-13T07:01:54.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:54.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598022+0000", 2024-09-13T07:01:54.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-13T07:01:54.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598062+0000", 2024-09-13T07:01:54.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9552999999999999e-05 2024-09-13T07:01:54.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598081+0000", 2024-09-13T07:01:54.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9247999999999999e-05 2024-09-13T07:01:54.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598256+0000", 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000174482 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598365+0000", 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109108 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.605106+0000", 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067409189999999997 2024-09-13T07:01:54.482 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1855 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.847933667, 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074110900000000004, 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.483 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1855 2024-09-13T07:01:54.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.484 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:54.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:54.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598117+0000", 2024-09-13T07:01:54.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598125+0000", 2024-09-13T07:01:54.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3110000000000006e-06 2024-09-13T07:01:54.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598147+0000", 2024-09-13T07:01:54.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2484999999999999e-05 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598159+0000", 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1939999999999999e-05 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.599204+0000", 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010449680000000001 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.599409+0000", 2024-09-13T07:01:54.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020456700000000001 2024-09-13T07:01:54.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.605529+0000", 2024-09-13T07:01:54.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061199790000000002 2024-09-13T07:01:54.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.487 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.487 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7595 3.a 3:54f0339c:::rbd_data.1194c5936dae.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 564664~8776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:01:54.487 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.687839+0000", 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.758212397000001, 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013545, 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7595 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.687839+0000", 2024-09-13T07:01:54.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.687839+0000", 2024-09-13T07:01:54.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.687836+0000", 2024-09-13T07:01:54.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:54.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690295+0000", 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024590800000000002 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690412+0000", 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116197 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690445+0000", 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2947999999999997e-05 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690548+0000", 2024-09-13T07:01:54.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103193 2024-09-13T07:01:54.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690733+0000", 2024-09-13T07:01:54.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018520799999999999 2024-09-13T07:01:54.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:54.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691055+0000", 2024-09-13T07:01:54.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032229699999999998 2024-09-13T07:01:54.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.695214+0000", 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041589050000000001 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701310+0000", 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060959370000000001 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701352+0000", 2024-09-13T07:01:54.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:01:54.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701384+0000", 2024-09-13T07:01:54.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1612999999999997e-05 2024-09-13T07:01:54.493 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.493 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.493 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.493 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7596 3.10 3:085a9411:::rbd_data.1194c5936dae.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 557056~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:01:54.493 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.690504+0000", 2024-09-13T07:01:54.493 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.755547624999998, 2024-09-13T07:01:54.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011135499, 2024-09-13T07:01:54.494 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.494 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.494 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7596 2024-09-13T07:01:54.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.494 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690504+0000", 2024-09-13T07:01:54.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690504+0000", 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690503+0000", 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691953+0000", 2024-09-13T07:01:54.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014497920000000001 2024-09-13T07:01:54.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691978+0000", 2024-09-13T07:01:54.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5384999999999998e-05 2024-09-13T07:01:54.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691991+0000", 2024-09-13T07:01:54.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.261e-05 2024-09-13T07:01:54.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.692271+0000", 2024-09-13T07:01:54.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028019399999999999 2024-09-13T07:01:54.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.692541+0000", 2024-09-13T07:01:54.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026980400000000001 2024-09-13T07:01:54.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:54.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.693006+0000", 2024-09-13T07:01:54.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046576499999999998 2024-09-13T07:01:54.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701109+0000", 2024-09-13T07:01:54.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081030630000000006 2024-09-13T07:01:54.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701549+0000", 2024-09-13T07:01:54.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043989499999999999 2024-09-13T07:01:54.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701606+0000", 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6292999999999998e-05 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701639+0000", 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3741000000000001e-05 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.499 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11218 3.10 3:0ba898fc:::rbd_data.1194a7cedc35.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.629190+0000", 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.816861127999999, 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070817470000000002, 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11218 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.629190+0000", 2024-09-13T07:01:54.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.629190+0000", 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.629188+0000", 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631056+0000", 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018678989999999999 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631082+0000", 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6156000000000002e-05 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631093+0000", 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0696999999999999e-05 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631216+0000", 2024-09-13T07:01:54.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012328299999999999 2024-09-13T07:01:54.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631435+0000", 2024-09-13T07:01:54.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021897299999999999 2024-09-13T07:01:54.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:54.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631884+0000", 2024-09-13T07:01:54.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044842900000000003 2024-09-13T07:01:54.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636066+0000", 2024-09-13T07:01:54.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041825059999999999 2024-09-13T07:01:54.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636133+0000", 2024-09-13T07:01:54.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7000000000000002e-05 2024-09-13T07:01:54.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636213+0000", 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9671000000000003e-05 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636272+0000", 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9126000000000003e-05 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:18674 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:head [stat,truncate 524288] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.276568+0000", 2024-09-13T07:01:54.505 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.1694830490000001, 2024-09-13T07:01:54.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069483679999999999, 2024-09-13T07:01:54.506 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.506 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.506 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 18674 2024-09-13T07:01:54.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.506 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276568+0000", 2024-09-13T07:01:54.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276568+0000", 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276567+0000", 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276574+0000", 2024-09-13T07:01:54.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2230000000000001e-06 2024-09-13T07:01:54.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276586+0000", 2024-09-13T07:01:54.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2109999999999999e-05 2024-09-13T07:01:54.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276595+0000", 2024-09-13T07:01:54.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3960000000000006e-06 2024-09-13T07:01:54.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276651+0000", 2024-09-13T07:01:54.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6740999999999998e-05 2024-09-13T07:01:54.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276714+0000", 2024-09-13T07:01:54.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3015999999999994e-05 2024-09-13T07:01:54.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:54.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276825+0000", 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110841 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.280710+0000", 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038845809999999998 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.283439+0000", 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027293700000000001 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.283474+0000", 2024-09-13T07:01:54.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4885999999999998e-05 2024-09-13T07:01:54.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.283517+0000", 2024-09-13T07:01:54.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2867000000000001e-05 2024-09-13T07:01:54.511 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.511 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.511 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.511 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20105 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:54.512 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667095+0000", 2024-09-13T07:01:54.512 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.77895687999999996, 2024-09-13T07:01:54.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086114899999999994, 2024-09-13T07:01:54.512 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.512 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.512 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.512 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.512 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.512 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20105 2024-09-13T07:01:54.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.512 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667095+0000", 2024-09-13T07:01:54.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667095+0000", 2024-09-13T07:01:54.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667094+0000", 2024-09-13T07:01:54.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667097+0000", 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4049999999999998e-06 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667102+0000", 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7450000000000001e-06 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667107+0000", 2024-09-13T07:01:54.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.669578+0000", 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024708180000000001 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.669641+0000", 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3089000000000005e-05 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675706+0000", 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060652359999999999 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20489 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667588+0000", 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.77846309800000002, 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068857950000000001, 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.517 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20489 2024-09-13T07:01:54.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.517 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667588+0000", 2024-09-13T07:01:54.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667588+0000", 2024-09-13T07:01:54.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667588+0000", 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667590+0000", 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2079999999999999e-06 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667596+0000", 2024-09-13T07:01:54.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.553e-06 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667608+0000", 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1703e-05 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674346+0000", 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067382320000000002 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674414+0000", 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8090999999999996e-05 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674474+0000", 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0337999999999997e-05 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20490 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667612+0000", 2024-09-13T07:01:54.520 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.77843920600000005, 2024-09-13T07:01:54.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069912159999999998, 2024-09-13T07:01:54.521 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.521 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.521 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20490 2024-09-13T07:01:54.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.521 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667612+0000", 2024-09-13T07:01:54.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667612+0000", 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667612+0000", 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667614+0000", 2024-09-13T07:01:54.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.486e-06 2024-09-13T07:01:54.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667620+0000", 2024-09-13T07:01:54.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.48e-06 2024-09-13T07:01:54.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667629+0000", 2024-09-13T07:01:54.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1249999999999999e-06 2024-09-13T07:01:54.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674487+0000", 2024-09-13T07:01:54.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068580710000000003 2024-09-13T07:01:54.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674530+0000", 2024-09-13T07:01:54.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3402999999999998e-05 2024-09-13T07:01:54.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674604+0000", 2024-09-13T07:01:54.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3028000000000006e-05 2024-09-13T07:01:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20491 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667634+0000", 2024-09-13T07:01:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.77841771500000001, 2024-09-13T07:01:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071287809999999998, 2024-09-13T07:01:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20491 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667634+0000", 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667634+0000", 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667633+0000", 2024-09-13T07:01:54.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667636+0000", 2024-09-13T07:01:54.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4719999999999998e-06 2024-09-13T07:01:54.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667642+0000", 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9159999999999998e-06 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667646+0000", 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4460000000000003e-06 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674624+0000", 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069781849999999996 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674691+0000", 2024-09-13T07:01:54.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6686000000000005e-05 2024-09-13T07:01:54.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674763+0000", 2024-09-13T07:01:54.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1440999999999996e-05 2024-09-13T07:01:54.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.529 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.529 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20492 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:54.529 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667650+0000", 2024-09-13T07:01:54.530 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.77840134699999997, 2024-09-13T07:01:54.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072483499999999998, 2024-09-13T07:01:54.530 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.530 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.530 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20492 2024-09-13T07:01:54.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.530 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667650+0000", 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667650+0000", 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667650+0000", 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667652+0000", 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0020000000000001e-06 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667657+0000", 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3349999999999997e-06 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667661+0000", 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3250000000000001e-06 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674775+0000", 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007113556 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674823+0000", 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8214999999999998e-05 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674899+0000", 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5257000000000001e-05 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.533 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.534 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20493 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:54.534 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667665+0000", 2024-09-13T07:01:54.534 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.778386256, 2024-09-13T07:01:54.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073907210000000003, 2024-09-13T07:01:54.534 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.534 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.534 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20493 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667665+0000", 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667665+0000", 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667665+0000", 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667672+0000", 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4490000000000002e-06 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667678+0000", 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9290000000000003e-06 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667683+0000", 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674920+0000", 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072375169999999997 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674988+0000", 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7968999999999999e-05 2024-09-13T07:01:54.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675056+0000", 2024-09-13T07:01:54.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-13T07:01:54.538 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.538 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.538 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.538 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20494 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:54.538 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667687+0000", 2024-09-13T07:01:54.538 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.77836410199999995, 2024-09-13T07:01:54.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074669300000000001, 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20494 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667687+0000", 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667687+0000", 2024-09-13T07:01:54.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667687+0000", 2024-09-13T07:01:54.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667689+0000", 2024-09-13T07:01:54.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-13T07:01:54.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667695+0000", 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6019999999999999e-06 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667699+0000", 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5430000000000004e-06 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675069+0000", 2024-09-13T07:01:54.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007369942 2024-09-13T07:01:54.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675109+0000", 2024-09-13T07:01:54.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9869000000000001e-05 2024-09-13T07:01:54.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675154+0000", 2024-09-13T07:01:54.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5064000000000001e-05 2024-09-13T07:01:54.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.542 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20495 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667703+0000", 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.77834831299999996, 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0075960799999999998, 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20495 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.543 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667703+0000", 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667703+0000", 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667703+0000", 2024-09-13T07:01:54.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667705+0000", 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2139999999999999e-06 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667710+0000", 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667715+0000", 2024-09-13T07:01:54.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3889999999999998e-06 2024-09-13T07:01:54.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675167+0000", 2024-09-13T07:01:54.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074520109999999997 2024-09-13T07:01:54.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675211+0000", 2024-09-13T07:01:54.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4107999999999998e-05 2024-09-13T07:01:54.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675299+0000", 2024-09-13T07:01:54.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8617999999999998e-05 2024-09-13T07:01:54.547 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.547 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.547 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.547 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20496 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:54.547 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667725+0000", 2024-09-13T07:01:54.547 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.77832607300000001, 2024-09-13T07:01:54.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077447180000000003, 2024-09-13T07:01:54.548 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.548 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.548 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20496 2024-09-13T07:01:54.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.548 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667725+0000", 2024-09-13T07:01:54.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667725+0000", 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667725+0000", 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667727+0000", 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6139999999999999e-06 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667733+0000", 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4049999999999999e-06 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667737+0000", 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675320+0000", 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0075824869999999997 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675389+0000", 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9035000000000006e-05 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675470+0000", 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1291000000000005e-05 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.552 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.552 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:54.553 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:01:54.554 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:54.550+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:54.640 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:01:54.665 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:54.665 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:54.665 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:54.666 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:54.677 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:01:54.685 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:54.685+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:54.755 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:54.750+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:54.785 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:54.785 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:54.785 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:54.785 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:54.785 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:54.792 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:54.795 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:01:54.813 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:54.834 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:54.853 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:54.856 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:54.853+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:54.873 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:54.893 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:54.902 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:54.902 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:54.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:54.902 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:54.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.902 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:54.902 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:54.902 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.387107499000003, 2024-09-13T07:01:54.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018207798000000001, 2024-09-13T07:01:54.903 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.903 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:54.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:54.903 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-13T07:01:54.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.903 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:54.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:54.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513664+0000", 2024-09-13T07:01:54.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513674+0000", 2024-09-13T07:01:54.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8770000000000005e-06 2024-09-13T07:01:54.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513695+0000", 2024-09-13T07:01:54.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1319e-05 2024-09-13T07:01:54.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513714+0000", 2024-09-13T07:01:54.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.914e-05 2024-09-13T07:01:54.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526929+0000", 2024-09-13T07:01:54.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013215146000000001 2024-09-13T07:01:54.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.527080+0000", 2024-09-13T07:01:54.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000150824 2024-09-13T07:01:54.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:54.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.527312+0000", 2024-09-13T07:01:54.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023218100000000001 2024-09-13T07:01:54.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530679+0000", 2024-09-13T07:01:54.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003366529 2024-09-13T07:01:54.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531659+0000", 2024-09-13T07:01:54.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0009803940000000001 2024-09-13T07:01:54.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531784+0000", 2024-09-13T07:01:54.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012519399999999999 2024-09-13T07:01:54.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531873+0000", 2024-09-13T07:01:54.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8307999999999995e-05 2024-09-13T07:01:54.908 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.908 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.908 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.909 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:54.909 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:54.909 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.387045229999998, 2024-09-13T07:01:54.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018374253, 2024-09-13T07:01:54.909 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.909 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:54.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:54.909 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-13T07:01:54.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.910 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:54.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:54.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513726+0000", 2024-09-13T07:01:54.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513933+0000", 2024-09-13T07:01:54.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020722500000000001 2024-09-13T07:01:54.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513984+0000", 2024-09-13T07:01:54.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0719e-05 2024-09-13T07:01:54.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513999+0000", 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4877e-05 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528581+0000", 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014581604999999999 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528674+0000", 2024-09-13T07:01:54.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3021000000000003e-05 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528803+0000", 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012964199999999999 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531891+0000", 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00308764 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531990+0000", 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9407999999999994e-05 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532064+0000", 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3793000000000003e-05 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532101+0000", 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7357000000000002e-05 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.914 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.386755970999999, 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018326366, 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.915 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:54.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:54.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514015+0000", 2024-09-13T07:01:54.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:54.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514026+0000", 2024-09-13T07:01:54.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1053000000000001e-05 2024-09-13T07:01:54.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514047+0000", 2024-09-13T07:01:54.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0687e-05 2024-09-13T07:01:54.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514065+0000", 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8799000000000001e-05 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529495+0000", 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015429877 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529642+0000", 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014682399999999999 2024-09-13T07:01:54.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529871+0000", 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022936399999999999 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532183+0000", 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023112889999999998 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532199+0000", 2024-09-13T07:01:54.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6082e-05 2024-09-13T07:01:54.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532269+0000", 2024-09-13T07:01:54.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9812000000000002e-05 2024-09-13T07:01:54.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532343+0000", 2024-09-13T07:01:54.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4141999999999996e-05 2024-09-13T07:01:54.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.920 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.386693735000001, 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018420999, 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.921 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514078+0000", 2024-09-13T07:01:54.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514088+0000", 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0433e-05 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514104+0000", 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5616000000000001e-05 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514133+0000", 2024-09-13T07:01:54.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9453000000000001e-05 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530751+0000", 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016617996999999999 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530926+0000", 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017493599999999999 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531174+0000", 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024812100000000002 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532358+0000", 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00118396 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532421+0000", 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2749999999999994e-05 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532467+0000", 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6427000000000001e-05 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532500+0000", 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2329000000000002e-05 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1174 3.13 3:cb2ed8dc:::rbd_data.11943a018f34.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 6419344~27893] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.016975760000001, 2024-09-13T07:01:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041813481999999999, 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1174 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:01:54.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883795+0000", 2024-09-13T07:01:54.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:54.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894234+0000", 2024-09-13T07:01:54.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010439133 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894276+0000", 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429000000000003e-05 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894296+0000", 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9279999999999998e-05 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894477+0000", 2024-09-13T07:01:54.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018160600000000001 2024-09-13T07:01:54.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894889+0000", 2024-09-13T07:01:54.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041202399999999998 2024-09-13T07:01:54.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:54.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.898212+0000", 2024-09-13T07:01:54.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033228400000000001 2024-09-13T07:01:54.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.922541+0000", 2024-09-13T07:01:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024329052 2024-09-13T07:01:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925458+0000", 2024-09-13T07:01:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002916891 2024-09-13T07:01:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925531+0000", 2024-09-13T07:01:54.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2781999999999995e-05 2024-09-13T07:01:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925610+0000", 2024-09-13T07:01:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9355000000000002e-05 2024-09-13T07:01:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:01:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:01:54.932 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.846599958999999, 2024-09-13T07:01:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:01:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:01:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:01:54.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:01:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:01:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:01:54.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:01:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:01:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:01:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:01:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:01:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:01:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1535 3.12 e27/22 3:4d1f9422:::rbd_object_map.11943a018f34:head v 27'27, mlcod=27'25)", 2024-09-13T07:01:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:01:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.832643128000001, 2024-09-13T07:01:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027873574000000002, 2024-09-13T07:01:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.937 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:01:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:01:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068128+0000", 2024-09-13T07:01:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068140+0000", 2024-09-13T07:01:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2279000000000001e-05 2024-09-13T07:01:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068175+0000", 2024-09-13T07:01:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4872000000000001e-05 2024-09-13T07:01:54.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068193+0000", 2024-09-13T07:01:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7540000000000001e-05 2024-09-13T07:01:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068302+0000", 2024-09-13T07:01:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010928 2024-09-13T07:01:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068351+0000", 2024-09-13T07:01:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036000000000003e-05 2024-09-13T07:01:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.095950+0000", 2024-09-13T07:01:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027598988000000001 2024-09-13T07:01:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.096003+0000", 2024-09-13T07:01:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2722000000000001e-05 2024-09-13T07:01:54.941 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11179 3.e 3:739b9dce:::rbd_data.1194897e7c0a.0000000000000004:c [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.606210+0000", 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.294562461000002, 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01661491, 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.942 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11179 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606210+0000", 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606210+0000", 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606208+0000", 2024-09-13T07:01:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606216+0000", 2024-09-13T07:01:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3139999999999998e-06 2024-09-13T07:01:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606237+0000", 2024-09-13T07:01:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1742e-05 2024-09-13T07:01:54.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606251+0000", 2024-09-13T07:01:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3152999999999999e-05 2024-09-13T07:01:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.622117+0000", 2024-09-13T07:01:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015865898999999999 2024-09-13T07:01:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.622199+0000", 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2262000000000006e-05 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.622825+0000", 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00062611399999999999 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11180 3.e 3:739b9dce:::rbd_data.1194897e7c0a.0000000000000004:c [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.606262+0000", 2024-09-13T07:01:54.946 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.294510344999999, 2024-09-13T07:01:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016844517, 2024-09-13T07:01:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11180 2024-09-13T07:01:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606262+0000", 2024-09-13T07:01:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606262+0000", 2024-09-13T07:01:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606261+0000", 2024-09-13T07:01:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606267+0000", 2024-09-13T07:01:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.127e-06 2024-09-13T07:01:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606284+0000", 2024-09-13T07:01:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6623999999999999e-05 2024-09-13T07:01:54.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606297+0000", 2024-09-13T07:01:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3118e-05 2024-09-13T07:01:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.622869+0000", 2024-09-13T07:01:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016571931000000002 2024-09-13T07:01:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.622970+0000", 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100832 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.623107+0000", 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000136842 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11181 3.e 3:739b9dce:::rbd_data.1194897e7c0a.0000000000000004:c [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.606309+0000", 2024-09-13T07:01:54.951 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.294463708999999, 2024-09-13T07:01:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017040290999999999, 2024-09-13T07:01:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11181 2024-09-13T07:01:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606309+0000", 2024-09-13T07:01:54.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606309+0000", 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606308+0000", 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606314+0000", 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9499999999999998e-06 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606330+0000", 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6291e-05 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.606345+0000", 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4924e-05 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.623135+0000", 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016790623000000001 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.623228+0000", 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2801999999999997e-05 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.623349+0000", 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120784 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.955 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:12392 3.11 3:8b43190b:::rbd_data.11945fc173dd.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:40.270727+0000", 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.630045815999999, 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023367789, 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12392 2024-09-13T07:01:54.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270727+0000", 2024-09-13T07:01:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270727+0000", 2024-09-13T07:01:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270724+0000", 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270740+0000", 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6467999999999999e-05 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270778+0000", 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8345e-05 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270795+0000", 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6171000000000002e-05 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270852+0000", 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7365999999999999e-05 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270951+0000", 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8908999999999998e-05 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.274009+0000", 2024-09-13T07:01:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030581110000000001 2024-09-13T07:01:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.293838+0000", 2024-09-13T07:01:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019829368 2024-09-13T07:01:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.293936+0000", 2024-09-13T07:01:54.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8066000000000006e-05 2024-09-13T07:01:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.294022+0000", 2024-09-13T07:01:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5908000000000005e-05 2024-09-13T07:01:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.294094+0000", 2024-09-13T07:01:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2044000000000003e-05 2024-09-13T07:01:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:12614 3.11 3:8b43190b:::rbd_data.11945fc173dd.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:40.730488+0000", 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.170284897, 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023878531000000001, 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12614 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.962 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730488+0000", 2024-09-13T07:01:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730488+0000", 2024-09-13T07:01:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730486+0000", 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730498+0000", 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1960999999999999e-05 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730524+0000", 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6301000000000001e-05 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730544+0000", 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9760999999999999e-05 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730630+0000", 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6768999999999995e-05 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730723+0000", 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2071999999999995e-05 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.735004+0000", 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042816770000000002 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.754112+0000", 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019107902999999999 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.754210+0000", 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8191999999999998e-05 2024-09-13T07:01:54.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.754295+0000", 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4401999999999996e-05 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.754366+0000", 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1320000000000002e-05 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16733 3.12 3:4dd7f56d:::rbd_data.1194b0f2ab1d.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:50.142664+0000", 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.7581087970000002, 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019628573, 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16733 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142664+0000", 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142664+0000", 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142662+0000", 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142675+0000", 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3356000000000001e-05 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142709+0000", 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3767999999999999e-05 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142723+0000", 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4357e-05 2024-09-13T07:01:54.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142882+0000", 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015923599999999999 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.143106+0000", 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000223453 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.146873+0000", 2024-09-13T07:01:54.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037672550000000002 2024-09-13T07:01:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.162202+0000", 2024-09-13T07:01:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01532932 2024-09-13T07:01:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.162211+0000", 2024-09-13T07:01:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7530000000000007e-06 2024-09-13T07:01:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.162246+0000", 2024-09-13T07:01:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4542999999999998e-05 2024-09-13T07:01:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.162292+0000", 2024-09-13T07:01:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6635000000000003e-05 2024-09-13T07:01:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.973 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16762 3.d 3:b3b4cf0b:::rbd_data.1194b0f2ab1d.0000000000000045:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-13T07:01:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:50.199245+0000", 2024-09-13T07:01:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.7015275150000004, 2024-09-13T07:01:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021081807000000001, 2024-09-13T07:01:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16762 2024-09-13T07:01:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199245+0000", 2024-09-13T07:01:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199245+0000", 2024-09-13T07:01:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199242+0000", 2024-09-13T07:01:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:54.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199256+0000", 2024-09-13T07:01:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3754000000000001e-05 2024-09-13T07:01:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199287+0000", 2024-09-13T07:01:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0948999999999998e-05 2024-09-13T07:01:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199302+0000", 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5291999999999999e-05 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199399+0000", 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6747000000000001e-05 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199741+0000", 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000341713 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.205391+0000", 2024-09-13T07:01:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056499189999999998 2024-09-13T07:01:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220249+0000", 2024-09-13T07:01:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014857977 2024-09-13T07:01:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220257+0000", 2024-09-13T07:01:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:01:54.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220296+0000", 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9001000000000001e-05 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220327+0000", 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0667000000000003e-05 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.979 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16763 3.15 3:ad1c703d:::rbd_data.1194b0f2ab1d.0000000000000046:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-13T07:01:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:50.199333+0000", 2024-09-13T07:01:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.7014398069999999, 2024-09-13T07:01:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021534991999999999, 2024-09-13T07:01:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16763 2024-09-13T07:01:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199333+0000", 2024-09-13T07:01:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199333+0000", 2024-09-13T07:01:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199331+0000", 2024-09-13T07:01:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:54.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199339+0000", 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0229999999999992e-06 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199351+0000", 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2320000000000001e-05 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199361+0000", 2024-09-13T07:01:54.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3039999999999994e-06 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199488+0000", 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000126993 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199812+0000", 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032460699999999998 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:54.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.205326+0000", 2024-09-13T07:01:54.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055137900000000002 2024-09-13T07:01:54.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:54.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220772+0000", 2024-09-13T07:01:54.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015445667999999999 2024-09-13T07:01:54.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:54.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220795+0000", 2024-09-13T07:01:54.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3266999999999999e-05 2024-09-13T07:01:54.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220840+0000", 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4620000000000003e-05 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220868+0000", 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7866000000000001e-05 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.985 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19454 3.f 3:f48ffbdf:::rbd_data.1194d2359c4.0000000000000012:12 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:54.986 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.591839+0000", 2024-09-13T07:01:54.986 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.3089334510000001, 2024-09-13T07:01:54.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016675986, 2024-09-13T07:01:54.986 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.986 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.986 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.986 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.986 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.986 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19454 2024-09-13T07:01:54.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.986 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591839+0000", 2024-09-13T07:01:54.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591839+0000", 2024-09-13T07:01:54.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591839+0000", 2024-09-13T07:01:54.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591840+0000", 2024-09-13T07:01:54.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.781e-06 2024-09-13T07:01:54.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591846+0000", 2024-09-13T07:01:54.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2279999999999998e-06 2024-09-13T07:01:54.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591849+0000", 2024-09-13T07:01:54.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.433e-06 2024-09-13T07:01:54.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608355+0000", 2024-09-13T07:01:54.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016506145 2024-09-13T07:01:54.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608416+0000", 2024-09-13T07:01:54.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0278999999999997e-05 2024-09-13T07:01:54.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608515+0000", 2024-09-13T07:01:54.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9444999999999994e-05 2024-09-13T07:01:54.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.990 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.990 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19455 3.f 3:f48ffbdf:::rbd_data.1194d2359c4.0000000000000012:12 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:54.991 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.591853+0000", 2024-09-13T07:01:54.991 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.3089193219999999, 2024-09-13T07:01:54.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016778435000000001, 2024-09-13T07:01:54.991 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.991 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.991 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.991 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.991 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.991 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19455 2024-09-13T07:01:54.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.991 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591853+0000", 2024-09-13T07:01:54.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591853+0000", 2024-09-13T07:01:54.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591852+0000", 2024-09-13T07:01:54.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:54.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591856+0000", 2024-09-13T07:01:54.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2849999999999999e-06 2024-09-13T07:01:54.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591865+0000", 2024-09-13T07:01:54.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9979999999999992e-06 2024-09-13T07:01:54.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591871+0000", 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6279999999999999e-06 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608531+0000", 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016659514 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608577+0000", 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6668000000000001e-05 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608632+0000", 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4091000000000001e-05 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19456 3.f 3:f48ffbdf:::rbd_data.1194d2359c4.0000000000000012:12 [sparse-read 4653056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.591894+0000", 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.3088780689999999, 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016932905000000002, 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:54.995 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:54.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:54.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:54.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:54.996 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19456 2024-09-13T07:01:54.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.996 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:54.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:54.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591894+0000", 2024-09-13T07:01:54.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591894+0000", 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591894+0000", 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591897+0000", 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7519999999999998e-06 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591905+0000", 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9650000000000008e-06 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591919+0000", 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4073000000000001e-05 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608646+0000", 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016726502000000001 2024-09-13T07:01:54.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608709+0000", 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2765e-05 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608827+0000", 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118554 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:54.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.000 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19457 3.f 3:f48ffbdf:::rbd_data.1194d2359c4.0000000000000012:12 [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:55.000 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.591927+0000", 2024-09-13T07:01:55.000 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.308845802, 2024-09-13T07:01:55.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017043903999999999, 2024-09-13T07:01:55.000 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:55.000 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:55.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:55.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:55.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:55.000 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19457 2024-09-13T07:01:55.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.000 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:55.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591927+0000", 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591927+0000", 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591926+0000", 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:55.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:55.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591930+0000", 2024-09-13T07:01:55.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3840000000000001e-06 2024-09-13T07:01:55.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:55.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591938+0000", 2024-09-13T07:01:55.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.14e-06 2024-09-13T07:01:55.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:55.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591945+0000", 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6260000000000003e-06 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608846+0000", 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016901177 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608897+0000", 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0118999999999999e-05 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:55.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608971+0000", 2024-09-13T07:01:55.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3936000000000005e-05 2024-09-13T07:01:55.004 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:55.004 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:55.004 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:55.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.004 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19516 3.11 3:8f71fce3:::rbd_header.119437d9d50d:head [watch reconnect cookie 140335576756256 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:55.004 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.657585+0000", 2024-09-13T07:01:55.004 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.243187016, 2024-09-13T07:01:55.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028756766999999999, 2024-09-13T07:01:55.004 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:55.004 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:55.005 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:55.005 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:55.005 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:55.005 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19516 2024-09-13T07:01:55.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.005 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:55.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:55.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657585+0000", 2024-09-13T07:01:55.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:55.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:55.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657585+0000", 2024-09-13T07:01:55.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:55.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:55.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657584+0000", 2024-09-13T07:01:55.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:55.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:55.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657592+0000", 2024-09-13T07:01:55.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5599999999999996e-06 2024-09-13T07:01:55.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657605+0000", 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2866e-05 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657614+0000", 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1050000000000001e-06 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657870+0000", 2024-09-13T07:01:55.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000256331 2024-09-13T07:01:55.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:55.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657958+0000", 2024-09-13T07:01:55.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8077000000000004e-05 2024-09-13T07:01:55.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:55.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:55.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:55.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.686342+0000", 2024-09-13T07:01:55.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028384063000000001 2024-09-13T07:01:55.008 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:55.008 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:55.008 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:55.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:55.009 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:55.009 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:55.010 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:55.010 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:55.011 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:01:55.012 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:55.012 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:55.012 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:55.012 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:55.012 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:55.012 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:55.012 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:55.012 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:55.013 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:55.009+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:55.111 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:55.109+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:55.129 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:55.129 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:55.142 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:01:55.212 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:55.209+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:55.254 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:55.254 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:55.257 INFO:teuthology.orchestra.run.smithi118.stdout:flattening image image_client.0-clone15 2024-09-13T07:01:55.264 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:01:55.313 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:55.309+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:55.377 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:55.377 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:55.387 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:01:55.414 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:55.410+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:55.456 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-13T07:01:55.500 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:55.501 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:55.506 INFO:teuthology.orchestra.run.smithi118.stdout:1322 write 0xeca3185 thru 0xecad9b1 (0xa82d bytes) 2024-09-13T07:01:55.510 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:01:55.516 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:55.513+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:55.532 INFO:teuthology.orchestra.run.smithi118.stdout:1323 writesame 0xc92fda thru 0xc9831c data_size 0x57(0x5343 bytes) 2024-09-13T07:01:55.578 INFO:teuthology.orchestra.run.smithi118.stdout:1324 read 0xac1c5a thru 0xacfd15 (0xe0bc bytes) 2024-09-13T07:01:55.579 INFO:teuthology.orchestra.run.smithi118.stdout:1325 read 0x1302f13 thru 0x13041c0 (0x12ae bytes) 2024-09-13T07:01:55.579 INFO:teuthology.orchestra.run.smithi118.stdout:1327 write 0xd44674c thru 0xd45621a (0xfacf bytes) 2024-09-13T07:01:55.596 INFO:teuthology.orchestra.run.smithi118.stdout:1328 write 0xb21d68c thru 0xb228f27 (0xb89c bytes) 2024-09-13T07:01:55.613 INFO:teuthology.orchestra.run.smithi118.stdout:1329 read 0x7976916 thru 0x7981a11 (0xb0fc bytes) 2024-09-13T07:01:55.613 INFO:teuthology.orchestra.run.smithi118.stdout:1331 writesame 0x7938304 thru 0x79386ff data_size 0x1(0x3fc bytes) 2024-09-13T07:01:55.616 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:55.614+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:55.619 INFO:teuthology.orchestra.run.smithi118.stdout:1332 write 0xc290782 thru 0xc296bbc (0x643b bytes) 2024-09-13T07:01:55.629 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:55.630 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:55.635 INFO:teuthology.orchestra.run.smithi118.stdout:1333 read 0xcd015b7 thru 0xcd09d0a (0x8754 bytes) 2024-09-13T07:01:55.635 INFO:teuthology.orchestra.run.smithi118.stdout:1336 read 0x5732c87 thru 0x5734308 (0x1682 bytes) 2024-09-13T07:01:55.635 INFO:teuthology.orchestra.run.smithi118.stdout:1337 writesame 0x20a5cc1 thru 0x20ac267 data_size 0x35(0x65a7 bytes) 2024-09-13T07:01:55.642 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:01:55.697 INFO:teuthology.orchestra.run.smithi118.stdout:1338 write 0xdb6afd9 thru 0xdb6cdfb (0x1e23 bytes) 2024-09-13T07:01:55.703 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:55.704 INFO:teuthology.orchestra.run.smithi118.stdout:1340 write 0x8cf617f thru 0x8cfa230 (0x40b2 bytes) 2024-09-13T07:01:55.717 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:55.714+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:55.729 INFO:teuthology.orchestra.run.smithi118.stdout:1341 read 0xc5b9cdb thru 0xc5c8761 (0xea87 bytes) 2024-09-13T07:01:55.730 INFO:teuthology.orchestra.run.smithi118.stdout:1342 read 0x268160a thru 0x26850e2 (0x3ad9 bytes) 2024-09-13T07:01:55.731 INFO:teuthology.orchestra.run.smithi118.stdout:1344 writesame 0x4abc4e5 thru 0x4ac1329 data_size 0x1(0x4e45 bytes) 2024-09-13T07:01:55.748 INFO:teuthology.orchestra.run.smithi118.stdout:1345 punch from 0x9107aba to 0x9115172, (0xd6b8 bytes) 2024-09-13T07:01:55.748 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:55.757 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:55.757 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:55.760 INFO:teuthology.orchestra.run.smithi118.stdout:1347 trunc from 0xecad9b2 to 0x7663783 2024-09-13T07:01:55.770 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:01:55.798 INFO:teuthology.orchestra.run.smithi118.stdout:1348 read 0xbf94a0 thru 0xbfea99 (0x55fa bytes) 2024-09-13T07:01:55.798 INFO:teuthology.orchestra.run.smithi118.stdout:1349 read 0x560e1b thru 0x5647f1 (0x39d7 bytes) 2024-09-13T07:01:55.800 INFO:teuthology.orchestra.run.smithi118.stdout:1350 punch from 0x11172b5 to 0x11227af, (0xb4fa bytes) 2024-09-13T07:01:55.815 INFO:teuthology.orchestra.run.smithi118.stdout:1351 write 0xb42397e thru 0xb425676 (0x1cf9 bytes) 2024-09-13T07:01:55.818 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:55.814+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:55.826 INFO:teuthology.orchestra.run.smithi118.stdout:1353 read 0xa3b123f thru 0xa3bae94 (0x9c56 bytes) 2024-09-13T07:01:55.826 INFO:teuthology.orchestra.run.smithi118.stdout:1354 punch from 0x9ac51ff to 0x9aced41, (0x9b42 bytes) 2024-09-13T07:01:55.827 INFO:teuthology.orchestra.run.smithi118.stdout:1355 write 0xe81cae thru 0xe854d4 (0x3827 bytes) 2024-09-13T07:01:55.860 INFO:teuthology.orchestra.run.smithi118.stdout:1356 write 0x4aaef31 thru 0x4ab97cc (0xa89c bytes) 2024-09-13T07:01:55.870 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:55.870 INFO:teuthology.orchestra.run.smithi118.stdout:1359 punch from 0x89b579e to 0x89bf11f, (0x9981 bytes) 2024-09-13T07:01:55.871 INFO:teuthology.orchestra.run.smithi118.stdout:1361 clone 17 order 24 su 65536 sc 15 2024-09-13T07:01:55.877 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:55.877 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:55.887 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:01:55.919 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:55.917+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:55.991 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:55.991 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:56.001 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:01:56.041 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:56.041+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:56.121 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:56.118+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:56.127 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:01:56.156 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:56.156 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:56.166 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:01:56.222 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:01:56.221+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:01:56.234 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone16 from 0xb425677 (overlap 0x7663783) to 0x23843da 2024-09-13T07:01:56.288 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:56.288 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:56.300 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:01:56.322 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:56.318+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:56.423 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:56.422+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:56.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:01:56.441+0000 7fc26e59c700 0 --2- 172.21.15.118:0/597952038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2700af720 0x7fc27006c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T07:01:56.479 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:56.479 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:56.493 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:01:56.525 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:56.522+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:56.619 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:56.619 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:56.626 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:56.622+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:56.632 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:01:56.727 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:56.725+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:56.739 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:56.739 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:56.749 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:01:56.767 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-13T07:01:56.828 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:56.825+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:56.855 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:56.855 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:56.861 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:01:56.861 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:01:56.861 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T07:01:56.862 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T07:01:56.865 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:01:56.929 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:56.925+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:56.972 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:56.972 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:56.981 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:01:57.013 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:01:57.030 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:57.029+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:57.077 INFO:teuthology.orchestra.run.smithi118.stdout:1362 punch from 0xad5f429 to 0xad5f7a5, (0x37c bytes) 2024-09-13T07:01:57.083 INFO:teuthology.orchestra.run.smithi118.stdout:1364 read 0x9e5f503 thru 0x9e66273 (0x6d71 bytes) 2024-09-13T07:01:57.083 INFO:teuthology.orchestra.run.smithi118.stdout:1365 punch from 0x4d89d to 0x5c97f, (0xf0e2 bytes) 2024-09-13T07:01:57.086 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:57.086 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:57.096 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:01:57.098 INFO:teuthology.orchestra.run.smithi118.stdout:1366 write 0xed52326 thru 0xed5b071 (0x8d4c bytes) 2024-09-13T07:01:57.133 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:57.133+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:57.138 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:01:57.168 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:57.171 INFO:teuthology.orchestra.run.smithi118.stdout:1367 punch from 0x3e0a08 to 0x3e3db2, (0x33aa bytes) 2024-09-13T07:01:57.176 INFO:teuthology.orchestra.run.smithi118.stdout:1368 trunc from 0xed5b072 to 0xc3e5a3e 2024-09-13T07:01:57.194 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:57.223 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:57.224 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:57.224 INFO:teuthology.orchestra.run.smithi118.stdout: "epoch": 58, 2024-09-13T07:01:57.224 INFO:teuthology.orchestra.run.smithi118.stdout: "fsid": "25bb30d1-9541-4e40-8659-18d83bb1bd8a", 2024-09-13T07:01:57.224 INFO:teuthology.orchestra.run.smithi118.stdout: "created": "2024-09-13T07:00:49.323883+0000", 2024-09-13T07:01:57.224 INFO:teuthology.orchestra.run.smithi118.stdout: "modified": "2024-09-13T07:01:57.123230+0000", 2024-09-13T07:01:57.224 INFO:teuthology.orchestra.run.smithi118.stdout: "last_up_change": "2024-09-13T07:01:00.146832+0000", 2024-09-13T07:01:57.224 INFO:teuthology.orchestra.run.smithi118.stdout: "last_in_change": "2024-09-13T07:01:46.465676+0000", 2024-09-13T07:01:57.224 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T07:01:57.224 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_num": 5806080, 2024-09-13T07:01:57.224 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_set": [ 2024-09-13T07:01:57.224 INFO:teuthology.orchestra.run.smithi118.stdout: "nodeep-scrub", 2024-09-13T07:01:57.224 INFO:teuthology.orchestra.run.smithi118.stdout: "noscrub", 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: "pglog_hardlimit", 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: "purged_snapdirs", 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_deletes", 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: "sortbitwise" 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_version": 4, 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_snapshot": "", 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_max": 3, 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: "max_osd": 8, 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: "require_min_compat_client": "luminous", 2024-09-13T07:01:57.225 INFO:teuthology.orchestra.run.smithi118.stdout: "min_compat_client": "jewel", 2024-09-13T07:01:57.226 INFO:teuthology.orchestra.run.smithi118.stdout: "require_osd_release": "reef", 2024-09-13T07:01:57.226 INFO:teuthology.orchestra.run.smithi118.stdout: "allow_crimson": false, 2024-09-13T07:01:57.226 INFO:teuthology.orchestra.run.smithi118.stdout: "pools": [ 2024-09-13T07:01:57.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.226 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 1, 2024-09-13T07:01:57.226 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": ".mgr", 2024-09-13T07:01:57.226 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:00.675365+0000", 2024-09-13T07:01:57.226 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 1, 2024-09-13T07:01:57.226 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool", 2024-09-13T07:01:57.226 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:01:57.226 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:01:57.226 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:01:57.226 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 1, 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 1, 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 1, 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 1, 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 1, 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:01:57.227 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:01:57.228 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:01:57.228 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:01:57.228 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:01:57.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.228 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "16", 2024-09-13T07:01:57.228 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:01:57.228 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:01:57.228 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:01:57.228 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:01:57.228 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:01:57.228 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 0, 2024-09-13T07:01:57.228 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 0, 2024-09-13T07:01:57.228 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:01:57.229 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:01:57.230 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:01:57.231 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:01:57.231 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:01:57.231 INFO:teuthology.orchestra.run.smithi118.stdout: "options": { 2024-09-13T07:01:57.231 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_max": 32, 2024-09-13T07:01:57.231 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_min": 1 2024-09-13T07:01:57.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.231 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:01:57.231 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": {} 2024-09-13T07:01:57.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.231 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:01:57.231 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 8, 2024-09-13T07:01:57.231 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 8, 2024-09-13T07:01:57.231 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.25, 2024-09-13T07:01:57.232 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 2, 2024-09-13T07:01:57.232 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 2, 2024-09-13T07:01:57.232 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 1, 2024-09-13T07:01:57.232 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 1, 2024-09-13T07:01:57.232 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 1 2024-09-13T07:01:57.232 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.232 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 2, 2024-09-13T07:01:57.232 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": "rbd", 2024-09-13T07:01:57.232 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:07.404263+0000", 2024-09-13T07:01:57.232 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 8193, 2024-09-13T07:01:57.232 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 8, 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 8, 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 8, 2024-09-13T07:01:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 8, 2024-09-13T07:01:57.234 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 8, 2024-09-13T07:01:57.234 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:01:57.234 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:01:57.234 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:01:57.234 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:01:57.234 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:01:57.234 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:01:57.234 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:01:57.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.234 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "20", 2024-09-13T07:01:57.234 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:01:57.234 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:01:57.234 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 3, 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 20, 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:01:57.235 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:01:57.236 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: "options": {}, 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd": {} 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 2, 2024-09-13T07:01:57.237 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 2, 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.75, 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 1.5, 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 1.5, 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 1, 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 1, 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 1 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 3, 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": "pool_client.0", 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:17.321011+0000", 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 8193, 2024-09-13T07:01:57.238 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 32, 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 32, 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 32, 2024-09-13T07:01:57.239 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 32, 2024-09-13T07:01:57.240 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 32, 2024-09-13T07:01:57.240 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:01:57.240 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:01:57.240 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:01:57.240 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:01:57.240 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:01:57.240 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:01:57.240 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:01:57.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.240 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "57", 2024-09-13T07:01:57.240 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:01:57.240 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:01:57.240 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 20, 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 57, 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:01:57.241 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:01:57.242 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: "options": {}, 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd": {} 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 1.75, 2024-09-13T07:01:57.243 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 1.75, 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.75, 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 1.309999942779541, 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 1.309999942779541, 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 1, 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 1, 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 1 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: "osds": [ 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 0, 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "21736eee-00cd-4eef-9725-eca7776957b4", 2024-09-13T07:01:57.244 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 0, 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0, 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 22, 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.245 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6817", 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6819", 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.246 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6821", 2024-09-13T07:01:57.247 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:01:57.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.247 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.247 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6823", 2024-09-13T07:01:57.247 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:01:57.247 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.247 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.247 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:01:57.247 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.247 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6829", 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6831", 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.248 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6825", 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6827", 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6819/3731386737", 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6823/3731386737", 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6831/3731386737", 2024-09-13T07:01:57.249 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6827/3731386737", 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 1, 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "05818db2-841d-406d-be17-8fbdab4fff69", 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:01:57.250 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:01:57.251 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:01:57.251 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 48, 2024-09-13T07:01:57.251 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:01:57.251 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:01:57.251 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:01:57.251 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.251 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.251 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6816", 2024-09-13T07:01:57.251 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:01:57.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.251 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6818", 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6820", 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.252 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6822", 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6828", 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.253 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6830", 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6824", 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.254 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6826", 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6818/876310814", 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6822/876310814", 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6830/876310814", 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6826/876310814", 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 2, 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "2387761c-42d2-481b-b5b5-b16d3a943b22", 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 0, 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0, 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 17, 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:01:57.256 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6808", 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6809", 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.257 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6810", 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6811", 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.258 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6814", 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6815", 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.259 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6812", 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6813", 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.260 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6809/690188762", 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6811/690188762", 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6815/690188762", 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6813/690188762", 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 3, 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "56500e90-f76b-47c8-a649-893400359c53", 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:01:57.261 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 48, 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6800", 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:01:57.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6801", 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6802", 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:01:57.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.264 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.264 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6803", 2024-09-13T07:01:57.264 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:01:57.264 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.264 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.264 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:01:57.264 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.264 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.264 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6806", 2024-09-13T07:01:57.264 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6807", 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6804", 2024-09-13T07:01:57.265 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:01:57.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.266 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.266 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6805", 2024-09-13T07:01:57.266 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:01:57.266 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.266 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.266 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6801/2198175481", 2024-09-13T07:01:57.266 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6803/2198175481", 2024-09-13T07:01:57.266 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6807/2198175481", 2024-09-13T07:01:57.266 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6805/2198175481", 2024-09-13T07:01:57.266 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 4, 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "f594a612-aa07-4565-a09e-08368cf397b6", 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:01:57.267 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:01:57.268 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 48, 2024-09-13T07:01:57.268 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:01:57.268 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:01:57.268 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:01:57.268 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.268 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.268 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6810", 2024-09-13T07:01:57.268 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:01:57.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.268 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.268 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6813", 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6817", 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.269 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6819", 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6827", 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.270 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6830", 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6822", 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.271 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6825", 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6813/1835601651", 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6819/1835601651", 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6830/1835601651", 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6825/1835601651", 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.273 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 5, 2024-09-13T07:01:57.273 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "1013b81f-e6b4-430b-938d-88759d43b881", 2024-09-13T07:01:57.273 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:01:57.273 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:01:57.273 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:01:57.273 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:01:57.273 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:01:57.273 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:01:57.273 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:01:57.273 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 51, 2024-09-13T07:01:57.273 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:01:57.273 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:01:57.273 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6800", 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6801", 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:01:57.274 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6802", 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6803", 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:01:57.275 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.276 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.276 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6806", 2024-09-13T07:01:57.276 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:01:57.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.276 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.276 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6807", 2024-09-13T07:01:57.276 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:01:57.276 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.276 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.276 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:01:57.277 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.277 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.277 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6804", 2024-09-13T07:01:57.277 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:01:57.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.277 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.277 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6805", 2024-09-13T07:01:57.277 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:01:57.277 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.277 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6801/2853696331", 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6803/2853696331", 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6807/2853696331", 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6805/2853696331", 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 6, 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "367f5f5b-5919-4534-b88c-229f327e4950", 2024-09-13T07:01:57.278 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:01:57.279 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:01:57.279 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:01:57.279 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:01:57.279 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:01:57.279 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:01:57.279 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:01:57.279 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 48, 2024-09-13T07:01:57.279 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:01:57.279 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:01:57.279 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:01:57.279 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.279 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6808", 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6812", 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.280 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.281 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6815", 2024-09-13T07:01:57.281 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:01:57.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.281 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.281 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6818", 2024-09-13T07:01:57.281 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:01:57.281 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.281 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.281 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:01:57.281 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6828", 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6831", 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:01:57.282 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6821", 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6824", 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6812/131309660", 2024-09-13T07:01:57.283 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6818/131309660", 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6831/131309660", 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6824/131309660", 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 7, 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "90d9e486-e1ae-4305-80a6-6f483e534643", 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:01:57.284 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:01:57.285 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:01:57.285 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:01:57.285 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:01:57.285 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 51, 2024-09-13T07:01:57.285 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:01:57.285 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:01:57.285 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:01:57.285 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.285 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.285 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6809", 2024-09-13T07:01:57.285 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:01:57.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6811", 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6814", 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:01:57.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6816", 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6826", 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:01:57.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6829", 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6820", 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:01:57.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.289 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:01:57.289 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6823", 2024-09-13T07:01:57.289 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:01:57.289 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.289 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.289 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6811/652173699", 2024-09-13T07:01:57.289 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6816/652173699", 2024-09-13T07:01:57.289 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6829/652173699", 2024-09-13T07:01:57.289 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6823/652173699", 2024-09-13T07:01:57.289 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:01:57.289 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_xinfo": [ 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 0, 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 54166, 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.059233+0000", 2024-09-13T07:01:57.290 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 1, 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.083130+0000", 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 2, 2024-09-13T07:01:57.291 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:01:57.292 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:01:57.292 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:01:57.292 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:01:57.292 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 65536, 2024-09-13T07:01:57.292 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.848896+0000", 2024-09-13T07:01:57.292 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:01:57.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.292 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 3, 2024-09-13T07:01:57.292 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:01:57.292 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:01:57.292 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:01:57.292 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.847456+0000", 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 4, 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.914135+0000", 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:01:57.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 5, 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.834284+0000", 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 6, 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:01:57.294 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.003715+0000", 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 7, 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:01:57.295 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.033514+0000", 2024-09-13T07:01:57.296 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:01:57.296 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.296 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:01:57.296 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap": [], 2024-09-13T07:01:57.296 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap_items": [], 2024-09-13T07:01:57.296 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap_primaries": [], 2024-09-13T07:01:57.296 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_temp": [], 2024-09-13T07:01:57.296 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_temp": [], 2024-09-13T07:01:57.296 INFO:teuthology.orchestra.run.smithi118.stdout: "blocklist": {}, 2024-09-13T07:01:57.296 INFO:teuthology.orchestra.run.smithi118.stdout: "range_blocklist": {}, 2024-09-13T07:01:57.296 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profiles": { 2024-09-13T07:01:57.296 INFO:teuthology.orchestra.run.smithi118.stdout: "default": { 2024-09-13T07:01:57.296 INFO:teuthology.orchestra.run.smithi118.stdout: "crush-failure-domain": "osd", 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: "k": "2", 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: "m": "1", 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: "plugin": "jerasure", 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: "technique": "reed_sol_van" 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps_queue": [], 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: "new_removed_snaps": [], 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: "new_purged_snaps": [], 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_node_flags": {}, 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: "device_class_flags": {}, 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode": { 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode_enabled": false, 2024-09-13T07:01:57.297 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_bucket_count": 0, 2024-09-13T07:01:57.298 INFO:teuthology.orchestra.run.smithi118.stdout: "degraded_stretch_mode": 0, 2024-09-13T07:01:57.298 INFO:teuthology.orchestra.run.smithi118.stdout: "recovering_stretch_mode": 0, 2024-09-13T07:01:57.298 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode_bucket": 0 2024-09-13T07:01:57.298 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:57.298 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:57.299 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:57.300 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:57.300 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:57.301 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:01:57.301 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:57.294+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:57.302 INFO:tasks.thrashosds.thrasher:j is {'epoch': 58, 'fsid': '25bb30d1-9541-4e40-8659-18d83bb1bd8a', 'created': '2024-09-13T07:00:49.323883+0000', 'modified': '2024-09-13T07:01:57.123230+0000', 'last_up_change': '2024-09-13T07:01:00.146832+0000', 'last_in_change': '2024-09-13T07:01:46.465676+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T07:01:00.675365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T07:01:07.404263+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T07:01:17.321011+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 20, 'snap_epoch': 57, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 0.75, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '21736eee-00cd-4eef-9725-eca7776957b4', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6817', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6819', 'nonce': 3731386737}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6821', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6823', 'nonce': 3731386737}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6829', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6831', 'nonce': 3731386737}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6825', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6827', 'nonce': 3731386737}]}, 'public_addr': '172.21.15.118:6819/3731386737', 'cluster_addr': '172.21.15.118:6823/3731386737', 'heartbeat_back_addr': '172.21.15.118:6831/3731386737', 'heartbeat_front_addr': '172.21.15.118:6827/3731386737', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '05818db2-841d-406d-be17-8fbdab4fff69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6816', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6818', 'nonce': 876310814}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6820', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6822', 'nonce': 876310814}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6828', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6830', 'nonce': 876310814}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6824', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6826', 'nonce': 876310814}]}, 'public_addr': '172.21.15.118:6818/876310814', 'cluster_addr': '172.21.15.118:6822/876310814', 'heartbeat_back_addr': '172.21.15.118:6830/876310814', 'heartbeat_front_addr': '172.21.15.118:6826/876310814', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2387761c-42d2-481b-b5b5-b16d3a943b22', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6808', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6809', 'nonce': 690188762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6810', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6811', 'nonce': 690188762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6814', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6815', 'nonce': 690188762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6812', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6813', 'nonce': 690188762}]}, 'public_addr': '172.21.15.118:6809/690188762', 'cluster_addr': '172.21.15.118:6811/690188762', 'heartbeat_back_addr': '172.21.15.118:6815/690188762', 'heartbeat_front_addr': '172.21.15.118:6813/690188762', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '56500e90-f76b-47c8-a649-893400359c53', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6800', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6801', 'nonce': 2198175481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6802', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6803', 'nonce': 2198175481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6806', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6807', 'nonce': 2198175481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6804', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6805', 'nonce': 2198175481}]}, 'public_addr': '172.21.15.118:6801/2198175481', 'cluster_addr': '172.21.15.118:6803/2198175481', 'heartbeat_back_addr': '172.21.15.118:6807/2198175481', 'heartbeat_front_addr': '172.21.15.118:6805/2198175481', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f594a612-aa07-4565-a09e-08368cf397b6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 1835601651}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6817', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 1835601651}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6827', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6830', 'nonce': 1835601651}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 1835601651}]}, 'public_addr': '172.21.15.164:6813/1835601651', 'cluster_addr': '172.21.15.164:6819/1835601651', 'heartbeat_back_addr': '172.21.15.164:6830/1835601651', 'heartbeat_front_addr': '172.21.15.164:6825/1835601651', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1013b81f-e6b4-430b-938d-88759d43b881', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 2853696331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 2853696331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 2853696331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 2853696331}]}, 'public_addr': '172.21.15.164:6801/2853696331', 'cluster_addr': '172.21.15.164:6803/2853696331', 'heartbeat_back_addr': '172.21.15.164:6807/2853696331', 'heartbeat_front_addr': '172.21.15.164:6805/2853696331', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '367f5f5b-5919-4534-b88c-229f327e4950', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6812', 'nonce': 131309660}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6815', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6818', 'nonce': 131309660}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 131309660}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6821', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6824', 'nonce': 131309660}]}, 'public_addr': '172.21.15.164:6812/131309660', 'cluster_addr': '172.21.15.164:6818/131309660', 'heartbeat_back_addr': '172.21.15.164:6831/131309660', 'heartbeat_front_addr': '172.21.15.164:6824/131309660', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '90d9e486-e1ae-4305-80a6-6f483e534643', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6809', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 652173699}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6816', 'nonce': 652173699}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 652173699}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 652173699}]}, 'public_addr': '172.21.15.164:6811/652173699', 'cluster_addr': '172.21.15.164:6816/652173699', 'heartbeat_back_addr': '172.21.15.164:6829/652173699', 'heartbeat_front_addr': '172.21.15.164:6823/652173699', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 54166, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.059233+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.083130+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.848896+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.847456+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.914135+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.834284+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.003715+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.033514+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T07:01:57.302 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:01:57.303 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:57.303 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:57.327 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:57.357 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:57.392 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:57.394 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:57.394 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:57.394 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:57.394 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:57.394 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:57.394 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:57.394 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:57.395 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:57.400 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:57.397+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:57.452 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:57.452 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:57.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:57.452 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:57.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.452 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1531 3.c 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:57.452 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:57.453 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.400704440999998, 2024-09-13T07:01:57.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063168486999999995, 2024-09-13T07:01:57.453 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.453 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:57.453 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.453 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.453 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.453 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1531 2024-09-13T07:01:57.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.453 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049511+0000", 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069234+0000", 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019723920999999998 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069269+0000", 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4446999999999997e-05 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069286+0000", 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7405999999999999e-05 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069370+0000", 2024-09-13T07:01:57.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-05 2024-09-13T07:01:57.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069511+0000", 2024-09-13T07:01:57.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014134299999999999 2024-09-13T07:01:57.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:57.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070463+0000", 2024-09-13T07:01:57.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095167899999999998 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105076+0000", 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034613139000000001 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112562+0000", 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074861429999999998 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:57.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112631+0000", 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112680+0000", 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9607000000000001e-05 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7628 3.1f 3:fdf2e50b:::rbd_data.1194c5936dae.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1032192~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:01:57.458 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.687778284, 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.630407088, 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7628 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:01:57.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762435+0000", 2024-09-13T07:01:57.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:01:57.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762543+0000", 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010850500000000001 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762597+0000", 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4262000000000003e-05 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762626+0000", 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9176e-05 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762761+0000", 2024-09-13T07:01:57.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013405599999999999 2024-09-13T07:01:57.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:01:57.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762776+0000", 2024-09-13T07:01:57.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5679999999999999e-05 2024-09-13T07:01:57.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388492+0000", 2024-09-13T07:01:57.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.62571606300000004 2024-09-13T07:01:57.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for missing object", 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388614+0000", 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012126 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390837+0000", 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002223806 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390963+0000", 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012550300000000001 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391127+0000", 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163995 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391989+0000", 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086230499999999995 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392662+0000", 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067260799999999999 2024-09-13T07:01:57.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392728+0000", 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6501000000000006e-05 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392845+0000", 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116665 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.466 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15862 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3440640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.466 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.772590+0000", 2024-09-13T07:01:57.466 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6776263470000004, 2024-09-13T07:01:57.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048639120000000001, 2024-09-13T07:01:57.466 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.466 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.466 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15862 2024-09-13T07:01:57.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.466 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772590+0000", 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772590+0000", 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772589+0000", 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:57.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772592+0000", 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772598+0000", 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.463e-06 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772603+0000", 2024-09-13T07:01:57.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5450000000000003e-06 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821064+0000", 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048461046000000001 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821137+0000", 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2507000000000002e-05 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821229+0000", 2024-09-13T07:01:57.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2082000000000004e-05 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15863 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3465216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.772608+0000", 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.677608073, 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048769738, 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.470 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15863 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772608+0000", 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772608+0000", 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772607+0000", 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772611+0000", 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3759999999999999e-06 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772616+0000", 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3970000000000002e-06 2024-09-13T07:01:57.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772621+0000", 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8459999999999999e-06 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821253+0000", 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048632149 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821308+0000", 2024-09-13T07:01:57.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4639999999999999e-05 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821378+0000", 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9987999999999995e-05 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15870 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3637248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6763196699999998, 2024-09-13T07:01:57.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048771508999999998, 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15870 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773899+0000", 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1949999999999999e-06 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773904+0000", 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.356e-06 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773909+0000", 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0849999999999996e-06 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822505+0000", 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048595933000000001 2024-09-13T07:01:57.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822579+0000", 2024-09-13T07:01:57.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4276999999999994e-05 2024-09-13T07:01:57.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822668+0000", 2024-09-13T07:01:57.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8522000000000004e-05 2024-09-13T07:01:57.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.478 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15871 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6763027000000008, 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048913976999999997, 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15871 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:57.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773916+0000", 2024-09-13T07:01:57.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0369999999999999e-06 2024-09-13T07:01:57.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773921+0000", 2024-09-13T07:01:57.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.383e-06 2024-09-13T07:01:57.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773927+0000", 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.147e-06 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822692+0000", 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048765583000000001 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822753+0000", 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0637999999999998e-05 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822827+0000", 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4630999999999997e-05 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15884 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.776551+0000", 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6736654410000007, 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048644055999999998, 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.483 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15884 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776551+0000", 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776551+0000", 2024-09-13T07:01:57.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776550+0000", 2024-09-13T07:01:57.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:57.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776553+0000", 2024-09-13T07:01:57.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1520000000000001e-06 2024-09-13T07:01:57.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776559+0000", 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4890000000000003e-06 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776564+0000", 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0390000000000004e-06 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825037+0000", 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048472701 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825109+0000", 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1884000000000005e-05 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825195+0000", 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6187000000000007e-05 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15885 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 4005888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.487 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.776569+0000", 2024-09-13T07:01:57.488 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6736474369999996, 2024-09-13T07:01:57.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048792239000000001, 2024-09-13T07:01:57.488 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.488 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.488 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15885 2024-09-13T07:01:57.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.488 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776569+0000", 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776569+0000", 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776568+0000", 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776571+0000", 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0419999999999998e-06 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776577+0000", 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5709999999999997e-06 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776582+0000", 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9540000000000003e-06 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825231+0000", 2024-09-13T07:01:57.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048649031000000002 2024-09-13T07:01:57.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825288+0000", 2024-09-13T07:01:57.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7148999999999998e-05 2024-09-13T07:01:57.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825361+0000", 2024-09-13T07:01:57.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3133999999999995e-05 2024-09-13T07:01:57.491 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15907 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6696200270000006, 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048713259000000002, 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.492 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15907 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780598+0000", 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7e-06 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780604+0000", 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-06 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780609+0000", 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6750000000000001e-06 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829162+0000", 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048553600000000002 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829226+0000", 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3547000000000007e-05 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829309+0000", 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3547000000000005e-05 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15908 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.780613+0000", 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6696031140000009, 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048901932000000002, 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.496 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15908 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780613+0000", 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780613+0000", 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780612+0000", 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780615+0000", 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0120000000000001e-06 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780621+0000", 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8320000000000002e-06 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780626+0000", 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829346+0000", 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048720439999999997 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829419+0000", 2024-09-13T07:01:57.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2347000000000004e-05 2024-09-13T07:01:57.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829515+0000", 2024-09-13T07:01:57.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6181000000000006e-05 2024-09-13T07:01:57.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.500 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.500 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15928 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6655718700000008, 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048610352000000003, 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15928 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:01:57.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784646+0000", 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3479999999999998e-06 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784652+0000", 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.485e-06 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784657+0000", 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8189999999999999e-06 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833039+0000", 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048382890999999997 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833116+0000", 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6371000000000005e-05 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833255+0000", 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138784 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.504 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15929 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784661+0000", 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6655553100000002, 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048815364, 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15929 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.505 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784661+0000", 2024-09-13T07:01:57.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784661+0000", 2024-09-13T07:01:57.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784660+0000", 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784663+0000", 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5639999999999999e-06 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784668+0000", 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4079999999999997e-06 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784673+0000", 2024-09-13T07:01:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5120000000000002e-06 2024-09-13T07:01:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833284+0000", 2024-09-13T07:01:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048611047999999997 2024-09-13T07:01:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833382+0000", 2024-09-13T07:01:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7571999999999993e-05 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833476+0000", 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4653000000000003e-05 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15930 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784677+0000", 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.665538883, 2024-09-13T07:01:57.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048947388000000001, 2024-09-13T07:01:57.510 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.510 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.510 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15930 2024-09-13T07:01:57.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.510 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784677+0000", 2024-09-13T07:01:57.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784677+0000", 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784676+0000", 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784680+0000", 2024-09-13T07:01:57.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1930000000000002e-06 2024-09-13T07:01:57.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784685+0000", 2024-09-13T07:01:57.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:01:57.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784690+0000", 2024-09-13T07:01:57.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6949999999999999e-06 2024-09-13T07:01:57.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833496+0000", 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048806041000000001 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833541+0000", 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4886999999999999e-05 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833625+0000", 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4073000000000006e-05 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.513 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.514 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.514 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15931 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.514 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784704+0000", 2024-09-13T07:01:57.514 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6655124289999996, 2024-09-13T07:01:57.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049199247000000002, 2024-09-13T07:01:57.514 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.514 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.514 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.514 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.514 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.514 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15931 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784704+0000", 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784704+0000", 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784703+0000", 2024-09-13T07:01:57.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:57.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784706+0000", 2024-09-13T07:01:57.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1e-06 2024-09-13T07:01:57.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784712+0000", 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8730000000000004e-06 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784718+0000", 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3850000000000001e-06 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833671+0000", 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048952993 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833787+0000", 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011663300000000001 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833903+0000", 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115747 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.518 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15951 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788591+0000", 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6616256099999998, 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048616683000000001, 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15951 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788591+0000", 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788591+0000", 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788590+0000", 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788593+0000", 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5720000000000001e-06 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788598+0000", 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4840000000000003e-06 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788603+0000", 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8659999999999997e-06 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837089+0000", 2024-09-13T07:01:57.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048485815000000002 2024-09-13T07:01:57.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837143+0000", 2024-09-13T07:01:57.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3794e-05 2024-09-13T07:01:57.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837207+0000", 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4566999999999994e-05 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15952 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788607+0000", 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6616090569999997, 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048772226000000002, 2024-09-13T07:01:57.523 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.524 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.524 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15952 2024-09-13T07:01:57.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.524 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788607+0000", 2024-09-13T07:01:57.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788607+0000", 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788606+0000", 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788609+0000", 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7640000000000001e-06 2024-09-13T07:01:57.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788615+0000", 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8490000000000002e-06 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788620+0000", 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3859999999999998e-06 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837240+0000", 2024-09-13T07:01:57.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048619860000000001 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837301+0000", 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0304999999999999e-05 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837379+0000", 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8689999999999994e-05 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15953 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:01:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6615796710000001, 2024-09-13T07:01:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04888609, 2024-09-13T07:01:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15953 2024-09-13T07:01:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:01:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788639+0000", 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4740000000000001e-06 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788646+0000", 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788652+0000", 2024-09-13T07:01:57.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7289999999999997e-06 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837400+0000", 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048747946 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837455+0000", 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5621000000000002e-05 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837523+0000", 2024-09-13T07:01:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7158999999999999e-05 2024-09-13T07:01:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15954 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1507328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:01:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6615598939999998, 2024-09-13T07:01:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049039567999999999, 2024-09-13T07:01:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15954 2024-09-13T07:01:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:01:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788659+0000", 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1889999999999999e-06 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788664+0000", 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788670+0000", 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6999999999999996e-06 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837552+0000", 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048882402999999998 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837615+0000", 2024-09-13T07:01:57.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2354000000000006e-05 2024-09-13T07:01:57.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837696+0000", 2024-09-13T07:01:57.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1286999999999996e-05 2024-09-13T07:01:57.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.536 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15977 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.792640+0000", 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6575758799999996, 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048609181000000001, 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15977 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.792640+0000", 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.792640+0000", 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.792639+0000", 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.792643+0000", 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1879999999999998e-06 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.792648+0000", 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0889999999999999e-06 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.792652+0000", 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6539999999999998e-06 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.841124+0000", 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048471619000000001 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.841178+0000", 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3693999999999998e-05 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.841249+0000", 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1729000000000004e-05 2024-09-13T07:01:57.540 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.331627975, 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.541 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:01:57.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:01:57.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:01:57.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:01:57.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:01:57.547 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.547 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.547 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.547 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.547 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.547 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:57.548 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:57.545+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:57.549 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:01:57.649 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:57.649+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:57.712 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:57.712 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:57.712 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:57.712 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:57.725 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:01:57.747 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:01:57.750 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:01:57.750 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:57.749+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:57.764 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-09-13T07:01:57.132593+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632176011,"num_objects":553,"num_object_clones":144,"num_object_copies":1106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":553,"num_whiteouts":185,"num_read":36324,"num_read_kb":110887,"num_write":4212,"num_write_kb":76964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12139577,"num_keys_recovered":59,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3539,"ondisk_log_size":3539,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":243244,"kb_used_data":52464,"kb_used_omap":9,"kb_used_meta":190710,"kb_avail":565987796,"statfs":{"total":579820584960,"available":579571503104,"internally_reserved":0,"allocated":53723136,"data_stored":57592690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9556,"internal_metadata":195287724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":115751082,"num_objects":134,"num_object_clones":51,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":33,"num_read":8736,"num_read_kb":16696,"num_write":793,"num_write_kb":5519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.031897"},"pg_stats":[{"pgid":"3.1f","version":"56'64","reported_seq":626,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.118823+0000","last_change":"2024-09-13T07:01:41.516000+0000","last_active":"2024-09-13T07:01:55.118823+0000","last_peered":"2024-09-13T07:01:55.118823+0000","last_clean":"2024-09-13T07:01:55.118823+0000","last_became_active":"2024-09-13T07:01:41.437584+0000","last_became_peered":"2024-09-13T07:01:41.437584+0000","last_unstale":"2024-09-13T07:01:55.118823+0000","last_undegraded":"2024-09-13T07:01:55.118823+0000","last_fullsized":"2024-09-13T07:01:55.118823+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:47.432372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013321120000000001,"stat_sum":{"num_bytes":32784702,"num_objects":21,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":7,"num_read":448,"num_read_kb":2375,"num_write":56,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":834141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"56'60","reported_seq":456,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.534399+0000","last_change":"2024-09-13T07:01:21.291845+0000","last_active":"2024-09-13T07:01:55.534399+0000","last_peered":"2024-09-13T07:01:55.534399+0000","last_clean":"2024-09-13T07:01:55.534399+0000","last_became_active":"2024-09-13T07:01:19.281907+0000","last_became_peered":"2024-09-13T07:01:19.281907+0000","last_unstale":"2024-09-13T07:01:55.534399+0000","last_undegraded":"2024-09-13T07:01:55.534399+0000","last_fullsized":"2024-09-13T07:01:55.534399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:43.514663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001950586,"stat_sum":{"num_bytes":22845837,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":394,"num_read_kb":566,"num_write":56,"num_write_kb":845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"56'117","reported_seq":311,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.596751+0000","last_change":"2024-09-13T07:01:21.293288+0000","last_active":"2024-09-13T07:01:55.596751+0000","last_peered":"2024-09-13T07:01:55.596751+0000","last_clean":"2024-09-13T07:01:55.596751+0000","last_became_active":"2024-09-13T07:01:19.276445+0000","last_became_peered":"2024-09-13T07:01:19.276445+0000","last_unstale":"2024-09-13T07:01:55.596751+0000","last_undegraded":"2024-09-13T07:01:55.596751+0000","last_fullsized":"2024-09-13T07:01:55.596751+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:16:37.731881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076373900000000004,"stat_sum":{"num_bytes":12337877,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":602,"num_read_kb":591,"num_write":176,"num_write_kb":477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"56'74","reported_seq":780,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.460368+0000","last_change":"2024-09-13T07:01:21.292234+0000","last_active":"2024-09-13T07:01:55.460368+0000","last_peered":"2024-09-13T07:01:55.460368+0000","last_clean":"2024-09-13T07:01:55.460368+0000","last_became_active":"2024-09-13T07:01:19.276459+0000","last_became_peered":"2024-09-13T07:01:19.276459+0000","last_unstale":"2024-09-13T07:01:55.460368+0000","last_undegraded":"2024-09-13T07:01:55.460368+0000","last_fullsized":"2024-09-13T07:01:55.460368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:50:50.173262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050866599999999996,"stat_sum":{"num_bytes":16026199,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":1317,"num_read_kb":2414,"num_write":65,"num_write_kb":234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"56'56","reported_seq":476,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.530074+0000","last_change":"2024-09-13T07:01:34.715025+0000","last_active":"2024-09-13T07:01:55.530074+0000","last_peered":"2024-09-13T07:01:55.530074+0000","last_clean":"2024-09-13T07:01:55.530074+0000","last_became_active":"2024-09-13T07:01:34.676996+0000","last_became_peered":"2024-09-13T07:01:34.676996+0000","last_unstale":"2024-09-13T07:01:55.530074+0000","last_undegraded":"2024-09-13T07:01:55.530074+0000","last_fullsized":"2024-09-13T07:01:55.530074+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:26:51.849086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083111000000000003,"stat_sum":{"num_bytes":19050476,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":400,"num_read_kb":1987,"num_write":50,"num_write_kb":1394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":67840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"56'95","reported_seq":720,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.530179+0000","last_change":"2024-09-13T07:01:21.297776+0000","last_active":"2024-09-13T07:01:55.530179+0000","last_peered":"2024-09-13T07:01:55.530179+0000","last_clean":"2024-09-13T07:01:55.530179+0000","last_became_active":"2024-09-13T07:01:19.275470+0000","last_became_peered":"2024-09-13T07:01:19.275470+0000","last_unstale":"2024-09-13T07:01:55.530179+0000","last_undegraded":"2024-09-13T07:01:55.530179+0000","last_fullsized":"2024-09-13T07:01:55.530179+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:15.860822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001799911,"stat_sum":{"num_bytes":12348570,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":878,"num_read_kb":877,"num_write":124,"num_write_kb":438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"56'153","reported_seq":789,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.679036+0000","last_change":"2024-09-13T07:01:21.300421+0000","last_active":"2024-09-13T07:01:55.679036+0000","last_peered":"2024-09-13T07:01:55.679036+0000","last_clean":"2024-09-13T07:01:55.679036+0000","last_became_active":"2024-09-13T07:01:19.280734+0000","last_became_peered":"2024-09-13T07:01:19.280734+0000","last_unstale":"2024-09-13T07:01:55.679036+0000","last_undegraded":"2024-09-13T07:01:55.679036+0000","last_fullsized":"2024-09-13T07:01:55.679036+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:02.332240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055727200000000004,"stat_sum":{"num_bytes":23405601,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":6,"num_read":981,"num_read_kb":1654,"num_write":196,"num_write_kb":1668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"56'96","reported_seq":958,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.518764+0000","last_change":"2024-09-13T07:01:35.443178+0000","last_active":"2024-09-13T07:01:55.518764+0000","last_peered":"2024-09-13T07:01:55.518764+0000","last_clean":"2024-09-13T07:01:55.518764+0000","last_became_active":"2024-09-13T07:01:35.423891+0000","last_became_peered":"2024-09-13T07:01:35.423891+0000","last_unstale":"2024-09-13T07:01:55.518764+0000","last_undegraded":"2024-09-13T07:01:55.518764+0000","last_fullsized":"2024-09-13T07:01:55.518764+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:20:08.704987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082396300000000004,"stat_sum":{"num_bytes":6273191,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":2397,"num_read_kb":3111,"num_write":84,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":14503,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"56'270","reported_seq":1381,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.665556+0000","last_change":"2024-09-13T07:01:50.079211+0000","last_active":"2024-09-13T07:01:55.665556+0000","last_peered":"2024-09-13T07:01:55.665556+0000","last_clean":"2024-09-13T07:01:55.665556+0000","last_became_active":"2024-09-13T07:01:50.078679+0000","last_became_peered":"2024-09-13T07:01:50.078679+0000","last_unstale":"2024-09-13T07:01:55.665556+0000","last_undegraded":"2024-09-13T07:01:55.665556+0000","last_fullsized":"2024-09-13T07:01:55.665556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:30:15.075968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001182647,"stat_sum":{"num_bytes":25876228,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":2373,"num_read_kb":11276,"num_write":408,"num_write_kb":4226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2756020,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"56'113","reported_seq":2513,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.663084+0000","last_change":"2024-09-13T07:01:21.293012+0000","last_active":"2024-09-13T07:01:55.663084+0000","last_peered":"2024-09-13T07:01:55.663084+0000","last_clean":"2024-09-13T07:01:55.663084+0000","last_became_active":"2024-09-13T07:01:19.277380+0000","last_became_peered":"2024-09-13T07:01:19.277380+0000","last_unstale":"2024-09-13T07:01:55.663084+0000","last_undegraded":"2024-09-13T07:01:55.663084+0000","last_fullsized":"2024-09-13T07:01:55.663084+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:05:57.870409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010463009999999999,"stat_sum":{"num_bytes":9755627,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":2656,"num_read_kb":5757,"num_write":140,"num_write_kb":461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"56'74","reported_seq":547,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.460368+0000","last_change":"2024-09-13T07:01:21.314353+0000","last_active":"2024-09-13T07:01:55.460368+0000","last_peered":"2024-09-13T07:01:55.460368+0000","last_clean":"2024-09-13T07:01:55.460368+0000","last_became_active":"2024-09-13T07:01:19.280203+0000","last_became_peered":"2024-09-13T07:01:19.280203+0000","last_unstale":"2024-09-13T07:01:55.460368+0000","last_undegraded":"2024-09-13T07:01:55.460368+0000","last_fullsized":"2024-09-13T07:01:55.460368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:38.598517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024699230000000002,"stat_sum":{"num_bytes":9508597,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":1004,"num_read_kb":759,"num_write":68,"num_write_kb":251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"53'142","reported_seq":530,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T07:01:54.116178+0000","last_change":"2024-09-13T07:01:21.298190+0000","last_active":"2024-09-13T07:01:54.116178+0000","last_peered":"2024-09-13T07:01:54.116178+0000","last_clean":"2024-09-13T07:01:54.116178+0000","last_became_active":"2024-09-13T07:01:19.276431+0000","last_became_peered":"2024-09-13T07:01:19.276431+0000","last_unstale":"2024-09-13T07:01:54.116178+0000","last_undegraded":"2024-09-13T07:01:54.116178+0000","last_fullsized":"2024-09-13T07:01:54.116178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:50:28.618968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001937825,"stat_sum":{"num_bytes":28715903,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":793,"num_read_kb":5792,"num_write":182,"num_write_kb":7689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":52,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T07:01:54.130371+0000","last_change":"2024-09-13T07:01:11.221939+0000","last_active":"2024-09-13T07:01:54.130371+0000","last_peered":"2024-09-13T07:01:54.130371+0000","last_clean":"2024-09-13T07:01:54.130371+0000","last_became_active":"2024-09-13T07:01:09.212302+0000","last_became_peered":"2024-09-13T07:01:09.212302+0000","last_unstale":"2024-09-13T07:01:54.130371+0000","last_undegraded":"2024-09-13T07:01:54.130371+0000","last_fullsized":"2024-09-13T07:01:54.130371+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:49.220240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"56'32","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.563710+0000","last_change":"2024-09-13T07:01:47.512765+0000","last_active":"2024-09-13T07:01:55.563710+0000","last_peered":"2024-09-13T07:01:55.563710+0000","last_clean":"2024-09-13T07:01:55.563710+0000","last_became_active":"2024-09-13T07:01:47.491472+0000","last_became_peered":"2024-09-13T07:01:47.491472+0000","last_unstale":"2024-09-13T07:01:55.563710+0000","last_undegraded":"2024-09-13T07:01:55.563710+0000","last_fullsized":"2024-09-13T07:01:55.563710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:06:58.026055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038364399999999999,"stat_sum":{"num_bytes":1970366,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":6,"num_read":13,"num_read_kb":8,"num_write":27,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":20355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T07:01:54.114224+0000","last_change":"2024-09-13T07:01:11.232122+0000","last_active":"2024-09-13T07:01:54.114224+0000","last_peered":"2024-09-13T07:01:54.114224+0000","last_clean":"2024-09-13T07:01:54.114224+0000","last_became_active":"2024-09-13T07:01:09.213058+0000","last_became_peered":"2024-09-13T07:01:09.213058+0000","last_unstale":"2024-09-13T07:01:54.114224+0000","last_undegraded":"2024-09-13T07:01:54.114224+0000","last_fullsized":"2024-09-13T07:01:54.114224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:10.839167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067353900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"56'151","reported_seq":423,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.630143+0000","last_change":"2024-09-13T07:01:21.291928+0000","last_active":"2024-09-13T07:01:55.630143+0000","last_peered":"2024-09-13T07:01:55.630143+0000","last_clean":"2024-09-13T07:01:55.630143+0000","last_became_active":"2024-09-13T07:01:19.286391+0000","last_became_peered":"2024-09-13T07:01:19.286391+0000","last_unstale":"2024-09-13T07:01:55.630143+0000","last_undegraded":"2024-09-13T07:01:55.630143+0000","last_fullsized":"2024-09-13T07:01:55.630143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:58:51.082865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016908909999999999,"stat_sum":{"num_bytes":25606189,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":750,"num_read_kb":2343,"num_write":199,"num_write_kb":481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T07:01:54.110130+0000","last_change":"2024-09-13T07:01:47.489709+0000","last_active":"2024-09-13T07:01:54.110130+0000","last_peered":"2024-09-13T07:01:54.110130+0000","last_clean":"2024-09-13T07:01:54.110130+0000","last_became_active":"2024-09-13T07:01:47.489419+0000","last_became_peered":"2024-09-13T07:01:47.489419+0000","last_unstale":"2024-09-13T07:01:54.110130+0000","last_undegraded":"2024-09-13T07:01:54.110130+0000","last_fullsized":"2024-09-13T07:01:54.110130+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:41.389847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"54'102","reported_seq":276,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T07:01:54.114053+0000","last_change":"2024-09-13T07:01:41.478005+0000","last_active":"2024-09-13T07:01:54.114053+0000","last_peered":"2024-09-13T07:01:54.114053+0000","last_clean":"2024-09-13T07:01:54.114053+0000","last_became_active":"2024-09-13T07:01:41.438741+0000","last_became_peered":"2024-09-13T07:01:41.438741+0000","last_unstale":"2024-09-13T07:01:54.114053+0000","last_undegraded":"2024-09-13T07:01:54.114053+0000","last_fullsized":"2024-09-13T07:01:54.114053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:03.836314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085579199999999997,"stat_sum":{"num_bytes":20887922,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":10,"num_read":345,"num_read_kb":247,"num_write":123,"num_write_kb":304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":3786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":60,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T07:01:54.114121+0000","last_change":"2024-09-13T07:01:47.486387+0000","last_active":"2024-09-13T07:01:54.114121+0000","last_peered":"2024-09-13T07:01:54.114121+0000","last_clean":"2024-09-13T07:01:54.114121+0000","last_became_active":"2024-09-13T07:01:47.485603+0000","last_became_peered":"2024-09-13T07:01:47.485603+0000","last_unstale":"2024-09-13T07:01:54.114121+0000","last_undegraded":"2024-09-13T07:01:54.114121+0000","last_fullsized":"2024-09-13T07:01:54.114121+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:57.144719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"56'60","reported_seq":603,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.460693+0000","last_change":"2024-09-13T07:01:21.293697+0000","last_active":"2024-09-13T07:01:55.460693+0000","last_peered":"2024-09-13T07:01:55.460693+0000","last_clean":"2024-09-13T07:01:55.460693+0000","last_became_active":"2024-09-13T07:01:19.279719+0000","last_became_peered":"2024-09-13T07:01:19.279719+0000","last_unstale":"2024-09-13T07:01:55.460693+0000","last_undegraded":"2024-09-13T07:01:55.460693+0000","last_fullsized":"2024-09-13T07:01:55.460693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:05:06.593984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085976499999999997,"stat_sum":{"num_bytes":24298336,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":607,"num_read_kb":3754,"num_write":57,"num_write_kb":417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"56'93","reported_seq":1228,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.673034+0000","last_change":"2024-09-13T07:01:47.540796+0000","last_active":"2024-09-13T07:01:55.673034+0000","last_peered":"2024-09-13T07:01:55.673034+0000","last_clean":"2024-09-13T07:01:55.673034+0000","last_became_active":"2024-09-13T07:01:47.491983+0000","last_became_peered":"2024-09-13T07:01:47.491983+0000","last_unstale":"2024-09-13T07:01:55.673034+0000","last_undegraded":"2024-09-13T07:01:55.673034+0000","last_fullsized":"2024-09-13T07:01:55.673034+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:18.382486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017265959999999999,"stat_sum":{"num_bytes":17531169,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":1348,"num_read_kb":13599,"num_write":149,"num_write_kb":4301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":3083106,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":54,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T07:01:54.110537+0000","last_change":"2024-09-13T07:01:11.228726+0000","last_active":"2024-09-13T07:01:54.110537+0000","last_peered":"2024-09-13T07:01:54.110537+0000","last_clean":"2024-09-13T07:01:54.110537+0000","last_became_active":"2024-09-13T07:01:09.211518+0000","last_became_peered":"2024-09-13T07:01:09.211518+0000","last_unstale":"2024-09-13T07:01:54.110537+0000","last_undegraded":"2024-09-13T07:01:54.110537+0000","last_fullsized":"2024-09-13T07:01:54.110537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:21.453172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"56'137","reported_seq":1735,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.690419+0000","last_change":"2024-09-13T07:01:44.465048+0000","last_active":"2024-09-13T07:01:55.690419+0000","last_peered":"2024-09-13T07:01:55.690419+0000","last_clean":"2024-09-13T07:01:55.690419+0000","last_became_active":"2024-09-13T07:01:44.463948+0000","last_became_peered":"2024-09-13T07:01:44.463948+0000","last_unstale":"2024-09-13T07:01:55.690419+0000","last_undegraded":"2024-09-13T07:01:55.690419+0000","last_fullsized":"2024-09-13T07:01:55.690419+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:34.829699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065804299999999997,"stat_sum":{"num_bytes":35290558,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":2200,"num_read_kb":4973,"num_write":159,"num_write_kb":9136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1395197,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T07:01:54.113156+0000","last_change":"2024-09-13T07:01:47.491502+0000","last_active":"2024-09-13T07:01:54.113156+0000","last_peered":"2024-09-13T07:01:54.113156+0000","last_clean":"2024-09-13T07:01:54.113156+0000","last_became_active":"2024-09-13T07:01:47.491145+0000","last_became_peered":"2024-09-13T07:01:47.491145+0000","last_unstale":"2024-09-13T07:01:54.113156+0000","last_undegraded":"2024-09-13T07:01:54.113156+0000","last_fullsized":"2024-09-13T07:01:54.113156+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:36:07.497138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"54'90","reported_seq":2276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.436520+0000","last_change":"2024-09-13T07:01:47.571765+0000","last_active":"2024-09-13T07:01:55.436520+0000","last_peered":"2024-09-13T07:01:55.436520+0000","last_clean":"2024-09-13T07:01:55.436520+0000","last_became_active":"2024-09-13T07:01:47.493229+0000","last_became_peered":"2024-09-13T07:01:47.493229+0000","last_unstale":"2024-09-13T07:01:55.436520+0000","last_undegraded":"2024-09-13T07:01:55.436520+0000","last_fullsized":"2024-09-13T07:01:55.436520+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:50:50.993168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044811299999999998,"stat_sum":{"num_bytes":21389127,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":2639,"num_read_kb":10492,"num_write":74,"num_write_kb":670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":106600,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T07:01:54.109330+0000","last_change":"2024-09-13T07:01:11.859918+0000","last_active":"2024-09-13T07:01:54.109330+0000","last_peered":"2024-09-13T07:01:54.109330+0000","last_clean":"2024-09-13T07:01:54.109330+0000","last_became_active":"2024-09-13T07:01:09.210971+0000","last_became_peered":"2024-09-13T07:01:09.210971+0000","last_unstale":"2024-09-13T07:01:54.109330+0000","last_undegraded":"2024-09-13T07:01:54.109330+0000","last_fullsized":"2024-09-13T07:01:54.109330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:30.775691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"56'134","reported_seq":1065,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.508522+0000","last_change":"2024-09-13T07:01:21.290419+0000","last_active":"2024-09-13T07:01:55.508522+0000","last_peered":"2024-09-13T07:01:55.508522+0000","last_clean":"2024-09-13T07:01:55.508522+0000","last_became_active":"2024-09-13T07:01:19.282935+0000","last_became_peered":"2024-09-13T07:01:19.282935+0000","last_unstale":"2024-09-13T07:01:55.508522+0000","last_undegraded":"2024-09-13T07:01:55.508522+0000","last_fullsized":"2024-09-13T07:01:55.508522+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:36:53.773908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064294199999999997,"stat_sum":{"num_bytes":17722323,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":2524,"num_read_kb":3858,"num_write":136,"num_write_kb":2331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":65,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T07:01:54.110294+0000","last_change":"2024-09-13T07:01:41.443501+0000","last_active":"2024-09-13T07:01:54.110294+0000","last_peered":"2024-09-13T07:01:54.110294+0000","last_clean":"2024-09-13T07:01:54.110294+0000","last_became_active":"2024-09-13T07:01:41.436739+0000","last_became_peered":"2024-09-13T07:01:41.436739+0000","last_unstale":"2024-09-13T07:01:54.110294+0000","last_undegraded":"2024-09-13T07:01:54.110294+0000","last_fullsized":"2024-09-13T07:01:54.110294+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:33:04.362119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T07:01:54.109426+0000","last_change":"2024-09-13T07:01:50.080564+0000","last_active":"2024-09-13T07:01:54.109426+0000","last_peered":"2024-09-13T07:01:54.109426+0000","last_clean":"2024-09-13T07:01:54.109426+0000","last_became_active":"2024-09-13T07:01:50.080072+0000","last_became_peered":"2024-09-13T07:01:50.080072+0000","last_unstale":"2024-09-13T07:01:54.109426+0000","last_undegraded":"2024-09-13T07:01:54.109426+0000","last_fullsized":"2024-09-13T07:01:54.109426+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:14:58.662626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"56'219","reported_seq":304,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.574541+0000","last_change":"2024-09-13T07:01:21.291091+0000","last_active":"2024-09-13T07:01:55.574541+0000","last_peered":"2024-09-13T07:01:55.574541+0000","last_clean":"2024-09-13T07:01:55.574541+0000","last_became_active":"2024-09-13T07:01:19.280796+0000","last_became_peered":"2024-09-13T07:01:19.280796+0000","last_unstale":"2024-09-13T07:01:55.574541+0000","last_undegraded":"2024-09-13T07:01:55.574541+0000","last_fullsized":"2024-09-13T07:01:55.574541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:20:10.332964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001264996,"stat_sum":{"num_bytes":2665148,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":984,"num_read_kb":813,"num_write":351,"num_write_kb":789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"56'38","reported_seq":185,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.530048+0000","last_change":"2024-09-13T07:01:41.467037+0000","last_active":"2024-09-13T07:01:55.530048+0000","last_peered":"2024-09-13T07:01:55.530048+0000","last_clean":"2024-09-13T07:01:55.530048+0000","last_became_active":"2024-09-13T07:01:41.438743+0000","last_became_peered":"2024-09-13T07:01:41.438743+0000","last_unstale":"2024-09-13T07:01:55.530048+0000","last_undegraded":"2024-09-13T07:01:55.530048+0000","last_fullsized":"2024-09-13T07:01:55.530048+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:11:01.568164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022509729999999999,"stat_sum":{"num_bytes":5803544,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":68,"num_read_kb":1047,"num_write":35,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":26238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"56'114","reported_seq":337,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.260453+0000","last_change":"2024-09-13T07:01:22.025365+0000","last_active":"2024-09-13T07:01:55.260453+0000","last_peered":"2024-09-13T07:01:55.260453+0000","last_clean":"2024-09-13T07:01:55.260453+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:01:55.260453+0000","last_undegraded":"2024-09-13T07:01:55.260453+0000","last_fullsized":"2024-09-13T07:01:55.260453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:11.537644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064478199999999999,"stat_sum":{"num_bytes":12772941,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":256,"num_read_kb":2484,"num_write":105,"num_write_kb":12116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"56'128","reported_seq":862,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.458316+0000","last_change":"2024-09-13T07:01:47.564025+0000","last_active":"2024-09-13T07:01:55.458316+0000","last_peered":"2024-09-13T07:01:55.458316+0000","last_clean":"2024-09-13T07:01:55.458316+0000","last_became_active":"2024-09-13T07:01:47.492956+0000","last_became_peered":"2024-09-13T07:01:47.492956+0000","last_unstale":"2024-09-13T07:01:55.458316+0000","last_undegraded":"2024-09-13T07:01:55.458316+0000","last_fullsized":"2024-09-13T07:01:55.458316+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:23:37.432022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001533932,"stat_sum":{"num_bytes":32735412,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":1465,"num_read_kb":4989,"num_write":124,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1474279,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"56'168","reported_seq":1738,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.664460+0000","last_change":"2024-09-13T07:01:21.292633+0000","last_active":"2024-09-13T07:01:55.664460+0000","last_peered":"2024-09-13T07:01:55.664460+0000","last_clean":"2024-09-13T07:01:55.664460+0000","last_became_active":"2024-09-13T07:01:19.279813+0000","last_became_peered":"2024-09-13T07:01:19.279813+0000","last_unstale":"2024-09-13T07:01:55.664460+0000","last_undegraded":"2024-09-13T07:01:55.664460+0000","last_fullsized":"2024-09-13T07:01:55.664460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:39.697691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072600099999999999,"stat_sum":{"num_bytes":29807592,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":4,"num_read":3272,"num_read_kb":3678,"num_write":193,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"56'109","reported_seq":1127,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.461453+0000","last_change":"2024-09-13T07:01:21.313310+0000","last_active":"2024-09-13T07:01:55.461453+0000","last_peered":"2024-09-13T07:01:55.461453+0000","last_clean":"2024-09-13T07:01:55.461453+0000","last_became_active":"2024-09-13T07:01:19.282520+0000","last_became_peered":"2024-09-13T07:01:19.282520+0000","last_unstale":"2024-09-13T07:01:55.461453+0000","last_undegraded":"2024-09-13T07:01:55.461453+0000","last_fullsized":"2024-09-13T07:01:55.461453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:41:44.801487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018248170000000001,"stat_sum":{"num_bytes":35828064,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":1260,"num_read_kb":4878,"num_write":94,"num_write_kb":1476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"56'62","reported_seq":639,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.591661+0000","last_change":"2024-09-13T07:01:21.313674+0000","last_active":"2024-09-13T07:01:55.591661+0000","last_peered":"2024-09-13T07:01:55.591661+0000","last_clean":"2024-09-13T07:01:55.591661+0000","last_became_active":"2024-09-13T07:01:19.281215+0000","last_became_peered":"2024-09-13T07:01:19.281215+0000","last_unstale":"2024-09-13T07:01:55.591661+0000","last_undegraded":"2024-09-13T07:01:55.591661+0000","last_fullsized":"2024-09-13T07:01:55.591661+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:22.161596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019878309999999998,"stat_sum":{"num_bytes":20517901,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":567,"num_read_kb":4415,"num_write":65,"num_write_kb":2099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"56'56","reported_seq":510,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.531480+0000","last_change":"2024-09-13T07:01:21.314048+0000","last_active":"2024-09-13T07:01:55.531480+0000","last_peered":"2024-09-13T07:01:55.531480+0000","last_clean":"2024-09-13T07:01:55.531480+0000","last_became_active":"2024-09-13T07:01:19.281169+0000","last_became_peered":"2024-09-13T07:01:19.281169+0000","last_unstale":"2024-09-13T07:01:55.531480+0000","last_undegraded":"2024-09-13T07:01:55.531480+0000","last_fullsized":"2024-09-13T07:01:55.531480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:49:46.525500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023438180000000001,"stat_sum":{"num_bytes":22996059,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":451,"num_read_kb":185,"num_write":49,"num_write_kb":339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"56'51","reported_seq":396,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.610782+0000","last_change":"2024-09-13T07:01:47.526764+0000","last_active":"2024-09-13T07:01:55.610782+0000","last_peered":"2024-09-13T07:01:55.610782+0000","last_clean":"2024-09-13T07:01:55.610782+0000","last_became_active":"2024-09-13T07:01:47.488355+0000","last_became_peered":"2024-09-13T07:01:47.488355+0000","last_unstale":"2024-09-13T07:01:55.610782+0000","last_undegraded":"2024-09-13T07:01:55.610782+0000","last_fullsized":"2024-09-13T07:01:55.610782+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:21:34.227554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061928799999999996,"stat_sum":{"num_bytes":10880563,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":282,"num_read_kb":3587,"num_write":52,"num_write_kb":1820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1706762,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"56'115","reported_seq":999,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.661758+0000","last_change":"2024-09-13T07:01:21.313825+0000","last_active":"2024-09-13T07:01:55.661758+0000","last_peered":"2024-09-13T07:01:55.661758+0000","last_clean":"2024-09-13T07:01:55.661758+0000","last_became_active":"2024-09-13T07:01:19.279714+0000","last_became_peered":"2024-09-13T07:01:19.279714+0000","last_unstale":"2024-09-13T07:01:55.661758+0000","last_undegraded":"2024-09-13T07:01:55.661758+0000","last_fullsized":"2024-09-13T07:01:55.661758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:06:09.717433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002154132,"stat_sum":{"num_bytes":30716859,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":1349,"num_read_kb":4424,"num_write":129,"num_write_kb":585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"56'114","reported_seq":697,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.665586+0000","last_change":"2024-09-13T07:01:50.078398+0000","last_active":"2024-09-13T07:01:55.665586+0000","last_peered":"2024-09-13T07:01:55.665586+0000","last_clean":"2024-09-13T07:01:55.665586+0000","last_became_active":"2024-09-13T07:01:50.078015+0000","last_became_peered":"2024-09-13T07:01:50.078015+0000","last_unstale":"2024-09-13T07:01:55.665586+0000","last_undegraded":"2024-09-13T07:01:55.665586+0000","last_fullsized":"2024-09-13T07:01:55.665586+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:52.203434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060210699999999995,"stat_sum":{"num_bytes":21541747,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":822,"num_read_kb":2478,"num_write":146,"num_write_kb":664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":60382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"56'132","reported_seq":619,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T07:01:55.563236+0000","last_change":"2024-09-13T07:01:21.313226+0000","last_active":"2024-09-13T07:01:55.563236+0000","last_peered":"2024-09-13T07:01:55.563236+0000","last_clean":"2024-09-13T07:01:55.563236+0000","last_became_active":"2024-09-13T07:01:19.279419+0000","last_became_peered":"2024-09-13T07:01:19.279419+0000","last_unstale":"2024-09-13T07:01:55.563236+0000","last_undegraded":"2024-09-13T07:01:55.563236+0000","last_fullsized":"2024-09-13T07:01:55.563236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:46:29.817757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001803165,"stat_sum":{"num_bytes":21694996,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":763,"num_read_kb":1376,"num_write":179,"num_write_kb":7545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":631585624,"num_objects":549,"num_object_clones":144,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":549,"num_whiteouts":185,"num_read":36208,"num_read_kb":110787,"num_write":4051,"num_write_kb":74546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":11549209,"num_keys_recovered":59,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62709760,"data_stored":70108848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3419,"ondisk_log_size":3419,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38880,"kb_used_data":7768,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94332960,"statfs":{"total":96636764160,"available":96596951040,"internally_reserved":0,"allocated":7954432,"data_stored":7908596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37024,"kb_used_data":10004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334816,"statfs":{"total":96636764160,"available":96598851584,"internally_reserved":0,"allocated":10244096,"data_stored":11329522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40424,"kb_used_data":9308,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94331416,"statfs":{"total":96636764160,"available":96595369984,"internally_reserved":0,"allocated":9531392,"data_stored":10206370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42736,"kb_used_data":11620,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94329104,"statfs":{"total":96636764160,"available":96593002496,"internally_reserved":0,"allocated":11898880,"data_stored":12542272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44932,"kb_used_data":9724,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94326908,"statfs":{"total":96636764160,"available":96590753792,"internally_reserved":0,"allocated":9957376,"data_stored":10842012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":2,"up_from":0,"seq":55834574862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39248,"kb_used_data":4040,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332592,"statfs":{"total":96636764160,"available":96596574208,"internally_reserved":0,"allocated":4136960,"data_stored":4763918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]}]},{"osd":0,"up_from":0,"seq":55834574862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9293824,"data_stored":10365051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4026368,"data_stored":4994214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2158592,"data_stored":2536533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10002432,"data_stored":11223787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11808768,"data_stored":12793510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9392128,"data_stored":10426407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":9080832,"data_stored":10517277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6946816,"data_stored":7252069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:01:57.766 INFO:tasks.thrashosds.thrasher:Setting 3.14 to [3, 5] 2024-09-13T07:01:57.767 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.14', '3', '5'] 2024-09-13T07:01:57.767 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.14 3 5 2024-09-13T07:01:57.821 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:57.821 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:57.821 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:57.821 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:57.821 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:57.831 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:01:57.851 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:57.850+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:57.943 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:57.943 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:57.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:57.943 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:57.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.943 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:57.944 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:57.944 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.882240466999999, 2024-09-13T07:01:57.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:01:57.944 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.944 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:57.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.944 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:01:57.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.944 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:01:57.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:01:57.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:57.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:01:57.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:01:57.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:01:57.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:01:57.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.949 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.949 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1534 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14365141] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:57.949 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.875412915999998, 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040478761000000002, 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1534 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:57.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065747+0000", 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075258+0000", 2024-09-13T07:01:57.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095108710000000006 2024-09-13T07:01:57.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075288+0000", 2024-09-13T07:01:57.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0085000000000001e-05 2024-09-13T07:01:57.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075305+0000", 2024-09-13T07:01:57.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7748e-05 2024-09-13T07:01:57.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075367+0000", 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1867999999999998e-05 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075520+0000", 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015298800000000001 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.076247+0000", 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072720499999999997 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105157+0000", 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028909642999999999 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106124+0000", 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096729400000000005 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106173+0000", 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8196999999999999e-05 2024-09-13T07:01:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106227+0000", 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4388999999999997e-05 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1557 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.800647246, 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012339057, 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.955 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1557 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140514+0000", 2024-09-13T07:01:57.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140502+0000", 2024-09-13T07:01:57.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999881 2024-09-13T07:01:57.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140526+0000", 2024-09-13T07:01:57.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4049999999999998e-05 2024-09-13T07:01:57.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140563+0000", 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6569000000000002e-05 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140580+0000", 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7725000000000001e-05 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140942+0000", 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036136300000000002 2024-09-13T07:01:57.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.141130+0000", 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018814999999999999 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.141433+0000", 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030329899999999998 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.147925+0000", 2024-09-13T07:01:57.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064915240000000003 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152757+0000", 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048328709999999999 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152802+0000", 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4314000000000003e-05 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152853+0000", 2024-09-13T07:01:57.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1149000000000001e-05 2024-09-13T07:01:57.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.961 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.961 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1558 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:57.961 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:57.961 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.800549508, 2024-09-13T07:01:57.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012353627000000001, 2024-09-13T07:01:57.961 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.961 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:57.961 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1558 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140612+0000", 2024-09-13T07:01:57.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140610+0000", 2024-09-13T07:01:57.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:57.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140619+0000", 2024-09-13T07:01:57.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8650000000000007e-06 2024-09-13T07:01:57.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140634+0000", 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4515000000000001e-05 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.140646+0000", 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1736000000000001e-05 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.144258+0000", 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036123499999999998 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.145034+0000", 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00077638900000000005 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.145395+0000", 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036117499999999998 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.149775+0000", 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043795639999999999 2024-09-13T07:01:57.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152906+0000", 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031306929999999999 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152941+0000", 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4919000000000002e-05 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.152965+0000", 2024-09-13T07:01:57.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4641e-05 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1676 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.600271304, 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067554429999999999, 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1676 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340890+0000", 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340889+0000", 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340896+0000", 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1420000000000004e-06 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340909+0000", 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3329000000000001e-05 2024-09-13T07:01:57.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.340920+0000", 2024-09-13T07:01:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1029999999999999e-05 2024-09-13T07:01:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341653+0000", 2024-09-13T07:01:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073243400000000001 2024-09-13T07:01:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.341825+0000", 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017243099999999999 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.347645+0000", 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058200639999999998 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1854 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [stat,zero 4194304~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.343151585000001, 2024-09-13T07:01:57.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070961979999999997, 2024-09-13T07:01:57.972 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.972 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.972 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1854 2024-09-13T07:01:57.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.972 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:57.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598010+0000", 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598007+0000", 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598022+0000", 2024-09-13T07:01:57.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598062+0000", 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9552999999999999e-05 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598081+0000", 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9247999999999999e-05 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598256+0000", 2024-09-13T07:01:57.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000174482 2024-09-13T07:01:57.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598365+0000", 2024-09-13T07:01:57.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109108 2024-09-13T07:01:57.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.605106+0000", 2024-09-13T07:01:57.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067409189999999997 2024-09-13T07:01:57.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.975 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1855 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [stat,zero 4194304~4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.343043304999998, 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074110900000000004, 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1855 2024-09-13T07:01:57.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.977 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:57.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598118+0000", 2024-09-13T07:01:57.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598117+0000", 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598125+0000", 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3110000000000006e-06 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598147+0000", 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2484999999999999e-05 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.598159+0000", 2024-09-13T07:01:57.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1939999999999999e-05 2024-09-13T07:01:57.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.599204+0000", 2024-09-13T07:01:57.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010449680000000001 2024-09-13T07:01:57.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.599409+0000", 2024-09-13T07:01:57.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020456700000000001 2024-09-13T07:01:57.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.605529+0000", 2024-09-13T07:01:57.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061199790000000002 2024-09-13T07:01:57.980 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.980 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.980 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.980 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7595 3.a 3:54f0339c:::rbd_data.1194c5936dae.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 564664~8776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:01:57.980 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.687839+0000", 2024-09-13T07:01:57.981 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.253322035, 2024-09-13T07:01:57.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013545, 2024-09-13T07:01:57.981 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.981 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:57.981 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.981 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.981 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.981 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7595 2024-09-13T07:01:57.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.981 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.687839+0000", 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.687839+0000", 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.687836+0000", 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690295+0000", 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024590800000000002 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690412+0000", 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116197 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690445+0000", 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2947999999999997e-05 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690548+0000", 2024-09-13T07:01:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103193 2024-09-13T07:01:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690733+0000", 2024-09-13T07:01:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018520799999999999 2024-09-13T07:01:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691055+0000", 2024-09-13T07:01:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032229699999999998 2024-09-13T07:01:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.695214+0000", 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041589050000000001 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701310+0000", 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060959370000000001 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701352+0000", 2024-09-13T07:01:57.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701384+0000", 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1612999999999997e-05 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7596 3.10 3:085a9411:::rbd_data.1194c5936dae.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 557056~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.690504+0000", 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.250657263000001, 2024-09-13T07:01:57.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011135499, 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7596 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690504+0000", 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690504+0000", 2024-09-13T07:01:57.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.690503+0000", 2024-09-13T07:01:57.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:57.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691953+0000", 2024-09-13T07:01:57.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014497920000000001 2024-09-13T07:01:57.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691978+0000", 2024-09-13T07:01:57.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5384999999999998e-05 2024-09-13T07:01:57.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.691991+0000", 2024-09-13T07:01:57.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.261e-05 2024-09-13T07:01:57.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.692271+0000", 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028019399999999999 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.692541+0000", 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026980400000000001 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.693006+0000", 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046576499999999998 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701109+0000", 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081030630000000006 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701549+0000", 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043989499999999999 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701606+0000", 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6292999999999998e-05 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.701639+0000", 2024-09-13T07:01:57.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3741000000000001e-05 2024-09-13T07:01:57.992 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.992 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.992 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.992 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:11218 3.10 3:0ba898fc:::rbd_data.1194a7cedc35.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:01:57.992 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:38.629190+0000", 2024-09-13T07:01:57.992 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.311970766000002, 2024-09-13T07:01:57.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070817470000000002, 2024-09-13T07:01:57.992 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11218 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.629190+0000", 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.629190+0000", 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.629188+0000", 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631056+0000", 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018678989999999999 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631082+0000", 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6156000000000002e-05 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631093+0000", 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0696999999999999e-05 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631216+0000", 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012328299999999999 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631435+0000", 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021897299999999999 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.631884+0000", 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044842900000000003 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636066+0000", 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041825059999999999 2024-09-13T07:01:57.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636133+0000", 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7000000000000002e-05 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636213+0000", 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9671000000000003e-05 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:38.636272+0000", 2024-09-13T07:01:57.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9126000000000003e-05 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:18674 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:head [stat,truncate 524288] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.276568+0000", 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.6645926869999998, 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069483679999999999, 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:57.998 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 18674 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276568+0000", 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276568+0000", 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:57.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276567+0000", 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276574+0000", 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2230000000000001e-06 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276586+0000", 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2109999999999999e-05 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276595+0000", 2024-09-13T07:01:58.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3960000000000006e-06 2024-09-13T07:01:58.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276651+0000", 2024-09-13T07:01:58.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6740999999999998e-05 2024-09-13T07:01:58.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276714+0000", 2024-09-13T07:01:58.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3015999999999994e-05 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.276825+0000", 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110841 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.280710+0000", 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038845809999999998 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.283439+0000", 2024-09-13T07:01:58.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027293700000000001 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.283474+0000", 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4885999999999998e-05 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.283517+0000", 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2867000000000001e-05 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20105 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667095+0000", 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2740665179999997, 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086114899999999994, 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20105 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667095+0000", 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667095+0000", 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667094+0000", 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:58.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667097+0000", 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4049999999999998e-06 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667102+0000", 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7450000000000001e-06 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667107+0000", 2024-09-13T07:01:58.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.669578+0000", 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024708180000000001 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.669641+0000", 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3089000000000005e-05 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675706+0000", 2024-09-13T07:01:58.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060652359999999999 2024-09-13T07:01:58.008 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.008 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.008 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.008 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20489 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:58.008 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667588+0000", 2024-09-13T07:01:58.008 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2735727360000002, 2024-09-13T07:01:58.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068857950000000001, 2024-09-13T07:01:58.008 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.008 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:58.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20489 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667588+0000", 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667588+0000", 2024-09-13T07:01:58.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667588+0000", 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667590+0000", 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2079999999999999e-06 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667596+0000", 2024-09-13T07:01:58.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.553e-06 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667608+0000", 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1703e-05 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674346+0000", 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067382320000000002 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674414+0000", 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8090999999999996e-05 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674474+0000", 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0337999999999997e-05 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20490 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667612+0000", 2024-09-13T07:01:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2735488440000005, 2024-09-13T07:01:58.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069912159999999998, 2024-09-13T07:01:58.013 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.013 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:58.013 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.013 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.013 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.013 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20490 2024-09-13T07:01:58.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.013 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667612+0000", 2024-09-13T07:01:58.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667612+0000", 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667612+0000", 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667614+0000", 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.486e-06 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667620+0000", 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.48e-06 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667629+0000", 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1249999999999999e-06 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674487+0000", 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068580710000000003 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674530+0000", 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3402999999999998e-05 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674604+0000", 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3028000000000006e-05 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.017 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20491 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:58.017 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667634+0000", 2024-09-13T07:01:58.017 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2735273530000004, 2024-09-13T07:01:58.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071287809999999998, 2024-09-13T07:01:58.017 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.017 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:58.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.017 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20491 2024-09-13T07:01:58.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667634+0000", 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667634+0000", 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667633+0000", 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667636+0000", 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4719999999999998e-06 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667642+0000", 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9159999999999998e-06 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667646+0000", 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4460000000000003e-06 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674624+0000", 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069781849999999996 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674691+0000", 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6686000000000005e-05 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674763+0000", 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1440999999999996e-05 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20492 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667650+0000", 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2735109849999997, 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072483499999999998, 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20492 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667650+0000", 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667650+0000", 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667650+0000", 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667652+0000", 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0020000000000001e-06 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667657+0000", 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3349999999999997e-06 2024-09-13T07:01:58.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667661+0000", 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3250000000000001e-06 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674775+0000", 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007113556 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674823+0000", 2024-09-13T07:01:58.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8214999999999998e-05 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674899+0000", 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5257000000000001e-05 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20493 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 2555904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667665+0000", 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2734958939999999, 2024-09-13T07:01:58.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073907210000000003, 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20493 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667665+0000", 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667665+0000", 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667665+0000", 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667672+0000", 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4490000000000002e-06 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667678+0000", 2024-09-13T07:01:58.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9290000000000003e-06 2024-09-13T07:01:58.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667683+0000", 2024-09-13T07:01:58.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-13T07:01:58.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674920+0000", 2024-09-13T07:01:58.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072375169999999997 2024-09-13T07:01:58.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.674988+0000", 2024-09-13T07:01:58.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7968999999999999e-05 2024-09-13T07:01:58.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675056+0000", 2024-09-13T07:01:58.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-13T07:01:58.029 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.029 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.030 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20494 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:58.030 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667687+0000", 2024-09-13T07:01:58.030 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.27347374, 2024-09-13T07:01:58.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074669300000000001, 2024-09-13T07:01:58.030 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.030 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:58.030 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.030 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.030 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.030 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20494 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667687+0000", 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667687+0000", 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667687+0000", 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667689+0000", 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667695+0000", 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6019999999999999e-06 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667699+0000", 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5430000000000004e-06 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675069+0000", 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007369942 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675109+0000", 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9869000000000001e-05 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675154+0000", 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5064000000000001e-05 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20495 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 3604480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667703+0000", 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2734579510000001, 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0075960799999999998, 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.034 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20495 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667703+0000", 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667703+0000", 2024-09-13T07:01:58.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667703+0000", 2024-09-13T07:01:58.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:58.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667705+0000", 2024-09-13T07:01:58.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2139999999999999e-06 2024-09-13T07:01:58.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667710+0000", 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667715+0000", 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3889999999999998e-06 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675167+0000", 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074520109999999997 2024-09-13T07:01:58.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675211+0000", 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4107999999999998e-05 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675299+0000", 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8617999999999998e-05 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:20496 3.10 3:0f107494:::rbd_data.1194d2359c4.000000000000000b:12 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.667725+0000", 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2734357110000003, 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077447180000000003, 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 20496 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667725+0000", 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667725+0000", 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667725+0000", 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667727+0000", 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6139999999999999e-06 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667733+0000", 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4049999999999999e-06 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.667737+0000", 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:01:58.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675320+0000", 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0075824869999999997 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675389+0000", 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9035000000000006e-05 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.675470+0000", 2024-09-13T07:01:58.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1291000000000005e-05 2024-09-13T07:01:58.043 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.043 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.043 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.043 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.043 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.043 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:58.044 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:01:58.045 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:58.041+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:58.145 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:58.142+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:58.167 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:58.167 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:58.167 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:01:58.167 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:58.177 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:01:58.246 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:58.242+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:58.339 INFO:teuthology.orchestra.run.smithi118.stdout:1369 write 0xa6724b4 thru 0xa681b23 (0xf670 bytes) 2024-09-13T07:01:58.339 INFO:teuthology.orchestra.run.smithi118.stdout:1370 punch from 0x9f3ec36 to 0x9f47db8, (0x9182 bytes) 2024-09-13T07:01:58.339 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:58.340 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:58.340 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:01:58.340 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:01:58.340 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:01:58.340 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:58.341 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:01:58.356 INFO:teuthology.orchestra.run.smithi118.stdout:1376 write 0xebfd4c2 thru 0xec06d40 (0x987f bytes) 2024-09-13T07:01:58.359 INFO:teuthology.orchestra.run.smithi118.stdout:1377 read 0xe29cce8 thru 0xe2aa1c0 (0xd4d9 bytes) 2024-09-13T07:01:58.374 INFO:teuthology.orchestra.run.smithi118.stdout:1378 trunc from 0xec06d41 to 0x7ebc4df 2024-09-13T07:01:58.388 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:58.386+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:58.389 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:58.390 INFO:teuthology.orchestra.run.smithi118.stdout:1381 writesame 0x6afdc52 thru 0x6b0c65e data_size 0xd(0xea0d bytes) 2024-09-13T07:01:58.395 INFO:teuthology.orchestra.run.smithi118.stdout:1382 read 0x546c5db thru 0x5471450 (0x4e76 bytes) 2024-09-13T07:01:58.397 INFO:teuthology.orchestra.run.smithi118.stdout:1383 punch from 0x778b3c4 to 0x77921a1, (0x6ddd bytes) 2024-09-13T07:01:58.397 INFO:teuthology.orchestra.run.smithi118.stdout:1385 punch from 0x2dc5ad3 to 0x2dcff82, (0xa4af bytes) 2024-09-13T07:01:58.397 INFO:teuthology.orchestra.run.smithi118.stdout:1386 write 0x56c571b thru 0x56c5e56 (0x73c bytes) 2024-09-13T07:01:58.399 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:58.399 INFO:teuthology.orchestra.run.smithi118.stdout:1388 read 0xeca8d thru 0xf2893 (0x5e07 bytes) 2024-09-13T07:01:58.401 INFO:teuthology.orchestra.run.smithi118.stdout:1389 write 0x323c1e2 thru 0x3247891 (0xb6b0 bytes) 2024-09-13T07:01:58.404 INFO:teuthology.orchestra.run.smithi118.stdout:1390 read 0x7865f18 thru 0x78734aa (0xd593 bytes) 2024-09-13T07:01:58.404 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:58.405 INFO:teuthology.orchestra.run.smithi118.stdout:1392 punch from 0x3536daa to 0x35458ec, (0xeb42 bytes) 2024-09-13T07:01:58.423 INFO:teuthology.orchestra.run.smithi118.stdout:1394 writesame 0xa6b1ba4 thru 0xa6be4b7 data_size 0x4(0xc914 bytes) 2024-09-13T07:01:58.426 INFO:teuthology.orchestra.run.smithi118.stdout:1395 read 0x68d0a2f thru 0x68dd911 (0xcee3 bytes) 2024-09-13T07:01:58.442 INFO:teuthology.orchestra.run.smithi118.stdout:1396 trunc from 0xa6be4b8 to 0x273b948 2024-09-13T07:01:58.448 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:58.446+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:58.464 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:01:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:01:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:01:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:01:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.948973619, 2024-09-13T07:01:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018207798000000001, 2024-09-13T07:01:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513665+0000", 2024-09-13T07:01:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513664+0000", 2024-09-13T07:01:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513674+0000", 2024-09-13T07:01:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8770000000000005e-06 2024-09-13T07:01:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513695+0000", 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1319e-05 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513714+0000", 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.914e-05 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.526929+0000", 2024-09-13T07:01:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013215146000000001 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.527080+0000", 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000150824 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.527312+0000", 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023218100000000001 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530679+0000", 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003366529 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531659+0000", 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0009803940000000001 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531784+0000", 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012519399999999999 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531873+0000", 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8307999999999995e-05 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.948911350000003, 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018374253, 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513727+0000", 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513726+0000", 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513933+0000", 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020722500000000001 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513984+0000", 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0719e-05 2024-09-13T07:01:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.513999+0000", 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4877e-05 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528581+0000", 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014581604999999999 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528674+0000", 2024-09-13T07:01:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3021000000000003e-05 2024-09-13T07:01:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.528803+0000", 2024-09-13T07:01:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012964199999999999 2024-09-13T07:01:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531891+0000", 2024-09-13T07:01:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00308764 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531990+0000", 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9407999999999994e-05 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532064+0000", 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3793000000000003e-05 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532101+0000", 2024-09-13T07:01:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7357000000000002e-05 2024-09-13T07:01:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.948622090999997, 2024-09-13T07:01:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018326366, 2024-09-13T07:01:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514016+0000", 2024-09-13T07:01:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514015+0000", 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514026+0000", 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1053000000000001e-05 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514047+0000", 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0687e-05 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514065+0000", 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8799000000000001e-05 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529495+0000", 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015429877 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529642+0000", 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014682399999999999 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.529871+0000", 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022936399999999999 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532183+0000", 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023112889999999998 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532199+0000", 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6082e-05 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532269+0000", 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9812000000000002e-05 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532343+0000", 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4141999999999996e-05 2024-09-13T07:01:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4166.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T07:01:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.948559854999999, 2024-09-13T07:01:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018420999, 2024-09-13T07:01:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4166", 2024-09-13T07:01:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/2504144367", 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514079+0000", 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514078+0000", 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514088+0000", 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0433e-05 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514104+0000", 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5616000000000001e-05 2024-09-13T07:01:58.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.514133+0000", 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9453000000000001e-05 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530751+0000", 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016617996999999999 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.530926+0000", 2024-09-13T07:01:58.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017493599999999999 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.531174+0000", 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024812100000000002 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532358+0000", 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00118396 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532421+0000", 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2749999999999994e-05 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532467+0000", 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6427000000000001e-05 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:02.532500+0000", 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2329000000000002e-05 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.488 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1174 3.13 3:cb2ed8dc:::rbd_data.11943a018f34.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 6419344~27893] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:58.488 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:01:58.488 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.578841879999999, 2024-09-13T07:01:58.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041813481999999999, 2024-09-13T07:01:58.488 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.488 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.488 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1174 2024-09-13T07:01:58.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883795+0000", 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894234+0000", 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010439133 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894276+0000", 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429000000000003e-05 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894296+0000", 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9279999999999998e-05 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894477+0000", 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018160600000000001 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894889+0000", 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041202399999999998 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.898212+0000", 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033228400000000001 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.922541+0000", 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024329052 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925458+0000", 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002916891 2024-09-13T07:01:58.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925531+0000", 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2781999999999995e-05 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925610+0000", 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9355000000000002e-05 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.408466079, 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:01:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:01:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:01:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:01:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:01:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:01:58.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:01:58.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1535 3.12 e27/22 3:4d1f9422:::rbd_object_map.11943a018f34:head v 27'27, mlcod=27'25)", 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.394509247999999, 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027873574000000002, 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068128+0000", 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068140+0000", 2024-09-13T07:01:58.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2279000000000001e-05 2024-09-13T07:01:58.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068175+0000", 2024-09-13T07:01:58.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4872000000000001e-05 2024-09-13T07:01:58.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068193+0000", 2024-09-13T07:01:58.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7540000000000001e-05 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068302+0000", 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010928 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068351+0000", 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036000000000003e-05 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.095950+0000", 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027598988000000001 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.096003+0000", 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2722000000000001e-05 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:12392 3.11 3:8b43190b:::rbd_data.11945fc173dd.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:40.270727+0000", 2024-09-13T07:01:58.502 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.191911936, 2024-09-13T07:01:58.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023367789, 2024-09-13T07:01:58.503 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.503 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.503 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12392 2024-09-13T07:01:58.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.503 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270727+0000", 2024-09-13T07:01:58.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270727+0000", 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270724+0000", 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270740+0000", 2024-09-13T07:01:58.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6467999999999999e-05 2024-09-13T07:01:58.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270778+0000", 2024-09-13T07:01:58.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8345e-05 2024-09-13T07:01:58.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270795+0000", 2024-09-13T07:01:58.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6171000000000002e-05 2024-09-13T07:01:58.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270852+0000", 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7365999999999999e-05 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.270951+0000", 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8908999999999998e-05 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.274009+0000", 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030581110000000001 2024-09-13T07:01:58.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.293838+0000", 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019829368 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.293936+0000", 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8066000000000006e-05 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.294022+0000", 2024-09-13T07:01:58.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5908000000000005e-05 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.294094+0000", 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2044000000000003e-05 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:12614 3.11 3:8b43190b:::rbd_data.11945fc173dd.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:40.730488+0000", 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.732151017, 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023878531000000001, 2024-09-13T07:01:58.508 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12614 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730488+0000", 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730488+0000", 2024-09-13T07:01:58.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730486+0000", 2024-09-13T07:01:58.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:58.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730498+0000", 2024-09-13T07:01:58.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1960999999999999e-05 2024-09-13T07:01:58.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730524+0000", 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6301000000000001e-05 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730544+0000", 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9760999999999999e-05 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730630+0000", 2024-09-13T07:01:58.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6768999999999995e-05 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.730723+0000", 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2071999999999995e-05 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.735004+0000", 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042816770000000002 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.754112+0000", 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019107902999999999 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.754210+0000", 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8191999999999998e-05 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.754295+0000", 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4401999999999996e-05 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:40.754366+0000", 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1320000000000002e-05 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16733 3.12 3:4dd7f56d:::rbd_data.1194b0f2ab1d.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:50.142664+0000", 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.3199749169999997, 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019628573, 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.514 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16733 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142664+0000", 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142664+0000", 2024-09-13T07:01:58.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142662+0000", 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142675+0000", 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3356000000000001e-05 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142709+0000", 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3767999999999999e-05 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142723+0000", 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4357e-05 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.142882+0000", 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015923599999999999 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.143106+0000", 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000223453 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.146873+0000", 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037672550000000002 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.162202+0000", 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01532932 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.162211+0000", 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7530000000000007e-06 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.162246+0000", 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4542999999999998e-05 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.162292+0000", 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6635000000000003e-05 2024-09-13T07:01:58.519 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.520 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.520 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.520 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16762 3.d 3:b3b4cf0b:::rbd_data.1194b0f2ab1d.0000000000000045:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-13T07:01:58.520 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:50.199245+0000", 2024-09-13T07:01:58.520 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.2633936349999999, 2024-09-13T07:01:58.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021081807000000001, 2024-09-13T07:01:58.520 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.520 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.520 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.520 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.520 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16762 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199245+0000", 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199245+0000", 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199242+0000", 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199256+0000", 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3754000000000001e-05 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199287+0000", 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0948999999999998e-05 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199302+0000", 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5291999999999999e-05 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199399+0000", 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6747000000000001e-05 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199741+0000", 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000341713 2024-09-13T07:01:58.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.205391+0000", 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056499189999999998 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220249+0000", 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014857977 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220257+0000", 2024-09-13T07:01:58.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220296+0000", 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9001000000000001e-05 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220327+0000", 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0667000000000003e-05 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16763 3.15 3:ad1c703d:::rbd_data.1194b0f2ab1d.0000000000000046:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:50.199333+0000", 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.2633059269999993, 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021534991999999999, 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16763 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199333+0000", 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199333+0000", 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199331+0000", 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199339+0000", 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0229999999999992e-06 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199351+0000", 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2320000000000001e-05 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199361+0000", 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3039999999999994e-06 2024-09-13T07:01:58.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199488+0000", 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000126993 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.199812+0000", 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032460699999999998 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.205326+0000", 2024-09-13T07:01:58.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055137900000000002 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220772+0000", 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015445667999999999 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220795+0000", 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3266999999999999e-05 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220840+0000", 2024-09-13T07:01:58.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4620000000000003e-05 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.220868+0000", 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7866000000000001e-05 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19457 3.f 3:f48ffbdf:::rbd_data.1194d2359c4.0000000000000012:12 [sparse-read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.591927+0000", 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8707119219999999, 2024-09-13T07:01:58.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017043903999999999, 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19457 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591927+0000", 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591927+0000", 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591926+0000", 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591930+0000", 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3840000000000001e-06 2024-09-13T07:01:58.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591938+0000", 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.14e-06 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.591945+0000", 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6260000000000003e-06 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608846+0000", 2024-09-13T07:01:58.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016901177 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608897+0000", 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0118999999999999e-05 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.608971+0000", 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3936000000000005e-05 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19516 3.11 3:8f71fce3:::rbd_header.119437d9d50d:head [watch reconnect cookie 140335576756256 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.657585+0000", 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8050531359999997, 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028756766999999999, 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19516 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657585+0000", 2024-09-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657585+0000", 2024-09-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657584+0000", 2024-09-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:58.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657592+0000", 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5599999999999996e-06 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657605+0000", 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2866e-05 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657614+0000", 2024-09-13T07:01:58.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1050000000000001e-06 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657870+0000", 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000256331 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657958+0000", 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8077000000000004e-05 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.686342+0000", 2024-09-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028384063000000001 2024-09-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:48357 3.9 e58/42 3:953ff28c:::rbd_data.1194fc792839.0000000000000000:head v 58'53, mlcod=57'51)", 2024-09-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.106275+0000", 2024-09-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.35636309100000002, 2024-09-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030129890999999999, 2024-09-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.106275+0000", 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.106275+0000", 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.106273+0000", 2024-09-13T07:01:58.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.113942+0000", 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0076687910000000003 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.113969+0000", 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7078999999999999e-05 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.113979+0000", 2024-09-13T07:01:58.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7319999999999993e-06 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.114033+0000", 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3692e-05 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.114062+0000", 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9094e-05 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.136272+0000", 2024-09-13T07:01:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022210189000000002 2024-09-13T07:01:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.136405+0000", 2024-09-13T07:01:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013353 2024-09-13T07:01:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.250584845, 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:01:58.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:01:58.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:01:58.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:01:58.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:01:58.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:01:58.550 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.551 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.551 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:58.551 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:01:58.551 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.224815134, 2024-09-13T07:01:58.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:01:58.551 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.551 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.552 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:01:58.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.552 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:01:58.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:01:58.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:01:58.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.557 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48373 3.12 3:4988147c:::rbd_object_map.1194fc792839:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:58.557 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258458+0000", 2024-09-13T07:01:58.557 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.204180684, 2024-09-13T07:01:58.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017947579000000002, 2024-09-13T07:01:58.557 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.557 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.557 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48373 2024-09-13T07:01:58.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258458+0000", 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258458+0000", 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258456+0000", 2024-09-13T07:01:58.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258463+0000", 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3640000000000001e-06 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258470+0000", 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1640000000000004e-06 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258475+0000", 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9259999999999999e-06 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258553+0000", 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8183999999999995e-05 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258695+0000", 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000142235 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:01:58.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259095+0000", 2024-09-13T07:01:58.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000399575 2024-09-13T07:01:58.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.261964+0000", 2024-09-13T07:01:58.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028686969999999999 2024-09-13T07:01:58.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.276263+0000", 2024-09-13T07:01:58.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01429894 2024-09-13T07:01:58.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.276343+0000", 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0550000000000006e-05 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.276405+0000", 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2206999999999996e-05 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.562 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.203663968, 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:01:58.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:01:58.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48377 3.d 3:b0571a54:::rbd_data.1194fc792839.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13654590~2926018] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.18176768400000001, 2024-09-13T07:01:58.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037131338, 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48377 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280868+0000", 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289338+0000", 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084696370000000003 2024-09-13T07:01:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289362+0000", 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4263000000000001e-05 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289374+0000", 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1209999999999999e-05 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289460+0000", 2024-09-13T07:01:58.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6883000000000001e-05 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289603+0000", 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014259299999999999 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.290036+0000", 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000432545 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317645+0000", 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027609584 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317836+0000", 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190645 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317925+0000", 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8732000000000004e-05 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:01:58.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.318002+0000", 2024-09-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7618000000000001e-05 2024-09-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:01:58.574 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:01:58.576 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:1398 writesame 0x733b06c thru 0x734191f data_size 0x2(0x68b4 bytes) 2024-09-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:1400 read 0x26c7c02 thru 0x26d038f (0x878e bytes) 2024-09-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:1401 read 0x177d9a8 thru 0x178d0d0 (0xf729 bytes) 2024-09-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:1402 punch from 0x6a52c0a to 0x6a624dd, (0xf8d3 bytes) 2024-09-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:1404 punch from 0x37d2690 to 0x37e1ab1, (0xf421 bytes) 2024-09-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:1405 write 0x29c1f10 thru 0x29cc645 (0xa736 bytes) 2024-09-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:1406 write 0xc6c951b thru 0xc6ceed7 (0x59bd bytes) 2024-09-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:1407 read 0x6ee460f thru 0x6ee4838 (0x22a bytes) 2024-09-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:1410 read 0x8b9a9c4 thru 0x8ba7805 (0xce42 bytes) 2024-09-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:1412 punch from 0x9c20e0b to 0x9c2c904, (0xbaf9 bytes) 2024-09-13T07:01:58.578 INFO:teuthology.orchestra.run.smithi118.stdout:1414 punch from 0x239d215 to 0x23aca2d, (0xf818 bytes) 2024-09-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi118.stdout:1415 writesame 0xd60015b thru 0xd607be1 data_size 0x1(0x7a87 bytes) 2024-09-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi118.stdout:1416 read 0x7a6cac thru 0x7a8cc3 (0x2018 bytes) 2024-09-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi118.stdout:1418 punch from 0x2468c51 to 0x24783ed, (0xf79c bytes) 2024-09-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi118.stdout:1421 read 0x3d49f9f thru 0x3d52edf (0x8f41 bytes) 2024-09-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi118.stdout:1422 read 0xbdc3ea9 thru 0xbdcb63d (0x7795 bytes) 2024-09-13T07:01:58.579 INFO:teuthology.orchestra.run.smithi118.stdout:1423 writesame 0x41ad00b thru 0x41b7499 data_size 0x1(0xa48f bytes) 2024-09-13T07:01:58.579 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:58.573+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:58.583 INFO:teuthology.orchestra.run.smithi118.stdout:1425 trunc from 0xd607be2 to 0x241bbd 2024-09-13T07:01:58.601 INFO:teuthology.orchestra.run.smithi118.stdout:1426 read 0x1652e1 thru 0x170989 (0xb6a9 bytes) 2024-09-13T07:01:58.618 INFO:teuthology.orchestra.run.smithi118.stdout:1428 writesame 0x493cba6 thru 0x494340d data_size 0x1a(0x6868 bytes) 2024-09-13T07:01:58.621 INFO:teuthology.orchestra.run.smithi118.stdout:1433 read 0x2f2ce65 thru 0x2f38258 (0xb3f4 bytes) 2024-09-13T07:01:58.640 INFO:teuthology.orchestra.run.smithi118.stdout:1434 writesame 0x81da416 thru 0x81e7812 data_size 0x1(0xd3fd bytes) 2024-09-13T07:01:58.658 INFO:teuthology.orchestra.run.smithi118.stdout:1435 write 0x91a94ed thru 0x91b2a45 (0x9559 bytes) 2024-09-13T07:01:58.662 INFO:teuthology.orchestra.run.smithi118.stdout:1436 write 0x11a5563 thru 0x11ac69b (0x7139 bytes) 2024-09-13T07:01:58.676 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:58.674+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:58.682 INFO:teuthology.orchestra.run.smithi118.stdout:1437 writesame 0xbc9193c thru 0xbc9386b data_size 0x8(0x1f30 bytes) 2024-09-13T07:01:58.684 INFO:teuthology.orchestra.run.smithi118.stdout:1438 punch from 0x9507875 to 0x9514d76, (0xd501 bytes) 2024-09-13T07:01:58.684 INFO:teuthology.orchestra.run.smithi118.stdout:1439 read 0x1ffe889 thru 0x200344f (0x4bc7 bytes) 2024-09-13T07:01:58.685 INFO:teuthology.orchestra.run.smithi118.stdout:1440 punch from 0xbafa13c to 0xbafa98e, (0x852 bytes) 2024-09-13T07:01:58.685 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:58.685 INFO:teuthology.orchestra.run.smithi118.stdout:1445 read 0x5a79487 thru 0x5a83842 (0xa3bc bytes) 2024-09-13T07:01:58.686 INFO:teuthology.orchestra.run.smithi118.stdout:1447 write 0x3d50135 thru 0x3d59db1 (0x9c7d bytes) 2024-09-13T07:01:58.689 INFO:teuthology.orchestra.run.smithi118.stdout:1448 write 0x93f293e thru 0x93f4a66 (0x2129 bytes) 2024-09-13T07:01:58.692 INFO:teuthology.orchestra.run.smithi118.stdout:1449 punch from 0x5a3ebeb to 0x5a44188, (0x559d bytes) 2024-09-13T07:01:58.692 INFO:teuthology.orchestra.run.smithi118.stdout:1450 punch from 0x71704d5 to 0x717c2bf, (0xbdea bytes) 2024-09-13T07:01:58.692 INFO:teuthology.orchestra.run.smithi118.stdout:1451 read 0x36b00d8 thru 0x36b4cc2 (0x4beb bytes) 2024-09-13T07:01:58.692 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:58.693 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:58.693 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:58.693 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:58.693 INFO:teuthology.orchestra.run.smithi118.stdout:1452 punch from 0x4741f20 to 0x474ddfd, (0xbedd bytes) 2024-09-13T07:01:58.693 INFO:teuthology.orchestra.run.smithi118.stdout:1454 read 0x39958b2 thru 0x39a2863 (0xcfb2 bytes) 2024-09-13T07:01:58.694 INFO:teuthology.orchestra.run.smithi118.stdout:1455 punch from 0xb823c0c to 0xb82a906, (0x6cfa bytes) 2024-09-13T07:01:58.694 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:01:58.694 INFO:teuthology.orchestra.run.smithi118.stdout:1457 punch from 0x5f5ddf0 to 0x5f5eb88, (0xd98 bytes) 2024-09-13T07:01:58.694 INFO:teuthology.orchestra.run.smithi118.stdout:1458 write 0x7455dbf thru 0x746303f (0xd281 bytes) 2024-09-13T07:01:58.697 INFO:teuthology.orchestra.run.smithi118.stdout:1460 clone 18 order 22 su 65536 sc 4 2024-09-13T07:01:58.708 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:01:58.777 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:58.777+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:58.862 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:58.862 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:58.862 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:58.863 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:58.863 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:58.872 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:01:58.878 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:58.874+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:58.977 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:58.977 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:58.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:58.977 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:58.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.977 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:58.977 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:58.978 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.470502385000003, 2024-09-13T07:01:58.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:01:58.978 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:58.978 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:58.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:58.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.978 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:01:58.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.978 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:58.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:58.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:58.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:01:58.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:01:58.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:58.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:01:58.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:01:58.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:01:58.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:01:58.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:01:58.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:58.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:58.983 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.445674674999999, 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:58.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:58.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:01:58.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:58.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:58.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:01:58.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:58.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:58.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:01:58.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:01:58.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:58.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:01:58.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:01:58.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.989 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:58.990 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:58.990 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.907002851000001, 2024-09-13T07:01:58.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:01:58.990 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:58.990 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.990 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:58.990 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:58.990 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.990 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:01:58.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.990 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:58.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:58.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:58.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:58.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:58.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:01:58.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:58.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:58.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:01:58.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:01:58.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:58.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:01:58.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:01:58.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:58.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:01:58.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:01:58.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:01:58.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:01:58.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:58.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:01:58.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:01:58.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:58.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:01:58.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:01:58.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:58.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:01:58.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:01:58.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:58.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:01:58.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:01:58.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:58.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:01:58.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:01:58.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:58.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:01:58.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:01:58.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.877377658, 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:58.996 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:58.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:01:58.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:58.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:58.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:01:58.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:58.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:58.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:01:58.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:01:58.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:58.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:59.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:01:59.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:01:59.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:59.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:01:59.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:01:59.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:01:59.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.344664027, 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.003 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:01:59.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.003 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:59.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:59.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:01:59.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:59.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:01:59.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:01:59.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:01:59.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:01:59.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:01:59.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:01:59.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:01:59.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:01:59.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:01:59.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:01:59.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:59.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:01:59.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:01:59.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:59.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:01:59.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:01:59.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:59.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:01:59.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:01:59.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:59.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:01:59.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:01:59.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:01:59.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:01:59.008 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.008 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.008 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.009 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:59.009 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:59.009 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.344581570999999, 2024-09-13T07:01:59.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:01:59.009 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.009 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:59.009 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.009 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.009 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.009 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:01:59.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.010 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:59.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:01:59.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:01:59.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:59.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:01:59.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:01:59.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:01:59.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:01:59.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:01:59.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:01:59.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:01:59.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:01:59.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:01:59.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:01:59.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:59.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:01:59.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:01:59.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:59.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:01:59.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:01:59.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:59.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:01:59.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:01:59.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:59.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:01:59.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:01:59.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:01:59.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:01:59.015 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.015 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.015 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.015 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.344549761000003, 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:59.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:01:59.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:01:59.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:01:59.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:01:59.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:01:59.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:01:59.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:01:59.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:01:59.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:01:59.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.344494603000001, 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:01:59.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:01:59.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:59.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.344449324000003, 2024-09-13T07:01:59.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:01:59.028 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.028 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:59.028 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.028 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.028 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.028 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:01:59.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.028 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:01:59.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:01:59.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.033 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.344410789999998, 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:01:59.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:59.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:01:59.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:01:59.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:59.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:01:59.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:01:59.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:59.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:01:59.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:01:59.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:59.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:01:59.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:01:59.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:01:59.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:01:59.039 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.039 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.039 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.344349170999998, 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.040 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:01:59.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:01:59.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:01:59.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:01:59.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:01:59.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:01:59.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:01:59.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:59.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:01:59.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:01:59.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:59.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:01:59.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:01:59.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.045 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.330126487999998, 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.046 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:01:59.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:01:59.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:59.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:59.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:01:59.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:01:59.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:01:59.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:01:59.051 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.051 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.051 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.318247968000001, 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.052 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:01:59.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.317690906999999, 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.056 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:01:59.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:01:59.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.317668751999999, 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:01:59.060 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:01:59.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:01:59.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:01:59.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:01:59.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:01:59.064 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.064 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.064 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.317649034999999, 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:01:59.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:01:59.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:01:59.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:01:59.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:01:59.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:01:59.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:01:59.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:01:59.069 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.069 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.069 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.069 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:59.069 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.317631705, 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:59.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:01:59.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:01:59.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:01:59.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:01:59.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:01:59.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:01:59.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:01:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.317336043000001, 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.074 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:01:59.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:01:59.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:01:59.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:01:59.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:01:59.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.756133223999999, 2024-09-13T07:01:59.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:01:59.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:01:59.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:01:59.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:01:59.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:01:59.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:01:59.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:01:59.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:01:59.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:01:59.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:01:59.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:01:59.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:01:59.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:01:59.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.724054321000001, 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:01:59.084 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:01:59.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:01:59.088 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.088 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.088 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.088 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.088 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.088 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:59.090 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:01:59.090 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:59.086+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:59.153 INFO:teuthology.orchestra.run.smithi118.stderr:set 3.14 pg_upmap mapping to [3,5] 2024-09-13T07:01:59.153 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:01:59.190 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:01:59.186+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:01:59.216 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:01:59.217 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:59.227 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:01:59.276 INFO:teuthology.orchestra.run.smithi118.stdout:flattening image image_client.0-clone17 2024-09-13T07:01:59.291 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:01:59.289+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:01:59.336 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:01:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:01:59.337 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:01:59.337 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:59.348 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:01:59.393 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:01:59.390+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:01:59.410 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:01:59.457 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:01:59.457 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:01:59.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:01:59.457 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:01:59.457 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.963174714000001, 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:01:59.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:01:59.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.462 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.463 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.463 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:01:59.463 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.47669115, 2024-09-13T07:01:59.463 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:01:59.463 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.463 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:59.463 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.463 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.463 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.463 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:01:59.463 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.463 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.463 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:01:59.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:01:59.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.476621221, 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.467 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.468 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:01:59.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.476605349, 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:01:59.471 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.475 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.4765348920000001, 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.476 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.477 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.473182854, 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.480 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:01:59.481 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.482 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.483 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.484 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.484 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.484 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.484 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:01:59.484 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.473166553, 2024-09-13T07:01:59.484 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:01:59.484 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.484 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.484 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.484 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.485 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:01:59.486 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:01:59.487 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:01:59.488 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.488 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.488 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.488 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.488 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.488 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.488 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:01:59.488 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.4731467679999999, 2024-09-13T07:01:59.488 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:01:59.488 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.488 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.488 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.488 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.489 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:01:59.490 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:01:59.491 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.4731304949999999, 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.492 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.493 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:01:59.494 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:01:59.495 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.4731126699999999, 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.496 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.497 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.498 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.499 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:01:59.499 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:01:59.499 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.499 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.499 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.499 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:01:59.499 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:01:59.499 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.499 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.499 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.499 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:01:59.499 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:01:59.499 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.473096577, 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.500 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.501 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.502 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.503 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.4730753910000001, 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:01:59.504 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.505 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.506 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:01:59.507 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:01:59.507 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.507 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.507 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.507 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:01:59.507 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:01:59.507 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.507 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.507 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.508 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:01:59.508 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:01:59.508 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.508 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.508 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.508 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.508 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.508 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.508 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:01:59.508 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.473059358, 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.509 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:01:59.510 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:01:59.511 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.473010119, 2024-09-13T07:01:59.512 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.513 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:01:59.514 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.515 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.515 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:01:59.515 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:01:59.515 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.515 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.515 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:01:59.515 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:01:59.515 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.515 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.515 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:01:59.516 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.4729920590000001, 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:01:59.518 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:01:59.519 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.4729755440000001, 2024-09-13T07:01:59.520 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.521 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.522 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:01:59.523 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.472957179, 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:01:59.524 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.525 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.526 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.527 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.4729410810000001, 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.528 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.529 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.529 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.529 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.529 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:01:59.529 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.529 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.529 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.529 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.529 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:01:59.530 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.530 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.530 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.530 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.530 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:01:59.530 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.530 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.530 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.530 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.530 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:01:59.530 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.531 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.531 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.531 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:01:59.531 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:01:59.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.531 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.531 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.531 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:01:59.531 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:01:59.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.531 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.531 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.532 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.4729251460000001, 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.533 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.534 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.534 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.534 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:01:59.534 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.534 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.534 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.534 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.534 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:01:59.534 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.534 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.534 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.534 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.534 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.535 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:01:59.536 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:01:59.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.536 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.536 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:01:59.536 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:01:59.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.536 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.536 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:01:59.536 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:01:59.536 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.536 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.4729086490000001, 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.537 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:01:59.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:01:59.539 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.540 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.541 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:01:59.541 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:01:59.541 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:01:59.542 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:01:59.542 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-13T07:01:59.543 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:01:59.541+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:01:59.574 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:01:59.643 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:59.641+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:59.657 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:01:59.685 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:01:59.716 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:59.716 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:59.717 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:01:59.730 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:01:59.743 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:01:59.744 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:01:59.742+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:01:59.763 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:01:59.790 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:01:59.816 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:01:59.818 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:01:59.818 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:01:59.818 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:01:59.818 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:01:59.819 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:01:59.819 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:01:59.819 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:01:59.819 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:01:59.839 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:59.839 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:59.844 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:01:59.841+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:01:59.848 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:01:59.945 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:01:59.942+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:01:59.954 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:01:59.954 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:01:59.964 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:00.046 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:00.042+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:00.068 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:00.068 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:00.077 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:00.147 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:00.145+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:00.159 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:02:00.183 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:00.183 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:00.193 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:00.248 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:00.246+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:00.347 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:00.347 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:00.349 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:00.345+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:00.357 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:00.450 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:00.449+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:00.464 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:00.464 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:00.474 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:00.551 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:00.546+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:00.581 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:00.581 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:00.591 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:00.652 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:00.649+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:00.694 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:00.695 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:00.704 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:00.753 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:00.749+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:00.807 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:00.807 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:00.817 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:00.854 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:00.853+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:00.926 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:00.926 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:00.936 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:00.955 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:00.950+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:01.046 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:01.046 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:01.055 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:01.056 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:01.053+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:01.158 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:01.154+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:01.164 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:01.164 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:01.178 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:01.236 INFO:teuthology.orchestra.run.smithi118.stdout:1463 read 0x704e72d thru 0x705dba9 (0xf47d bytes) 2024-09-13T07:02:01.238 INFO:teuthology.orchestra.run.smithi118.stdout:1464 read 0xa57a277 thru 0xa57be63 (0x1bed bytes) 2024-09-13T07:02:01.242 INFO:teuthology.orchestra.run.smithi118.stdout:1465 punch from 0x41b7092 to 0x41bca76, (0x59e4 bytes) 2024-09-13T07:02:01.247 INFO:teuthology.orchestra.run.smithi118.stdout:1466 write 0x9cf5ec0 thru 0x9d027f6 (0xc937 bytes) 2024-09-13T07:02:01.258 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:01.254+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:01.263 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:01.263 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:01.263 INFO:teuthology.orchestra.run.smithi118.stdout:1472 write 0x166190c thru 0x1669d71 (0x8466 bytes) 2024-09-13T07:02:01.272 INFO:teuthology.orchestra.run.smithi118.stdout:1473 read 0x610a65f thru 0x6118b84 (0xe526 bytes) 2024-09-13T07:02:01.273 INFO:teuthology.orchestra.run.smithi118.stdout:1474 read 0x99cc24d thru 0x99d0f63 (0x4d17 bytes) 2024-09-13T07:02:01.282 INFO:teuthology.orchestra.run.smithi118.stdout:1476 trunc from 0xbc9386c to 0x7c57d87 2024-09-13T07:02:01.326 INFO:teuthology.orchestra.run.smithi118.stdout:1477 trunc from 0x7c57d87 to 0xa25625b 2024-09-13T07:02:01.335 INFO:teuthology.orchestra.run.smithi118.stdout:1480 writesame 0xd2266c thru 0xd25c7b data_size 0x28(0x3610 bytes) 2024-09-13T07:02:01.346 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:01.346 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:01.348 INFO:teuthology.orchestra.run.smithi118.stdout:1481 read 0x60d7509 thru 0x60e217b (0xac73 bytes) 2024-09-13T07:02:01.349 INFO:teuthology.orchestra.run.smithi118.stdout:1483 flatten 2024-09-13T07:02:01.359 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:01.359 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:01.357+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:01.444 INFO:teuthology.orchestra.run.smithi118.stdout:1484 write 0xe6a09c2 thru 0xe6a14de (0xb1d bytes) 2024-09-13T07:02:01.449 INFO:teuthology.orchestra.run.smithi118.stdout:1485 write 0x902d9fb thru 0x903bf03 (0xe509 bytes) 2024-09-13T07:02:01.457 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:01.460 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:01.458+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:01.481 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:01.481 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:01.484 INFO:teuthology.orchestra.run.smithi118.stdout:1487 trunc from 0xe6a14df to 0x4a17fd 2024-09-13T07:02:01.491 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:01.530 INFO:teuthology.orchestra.run.smithi118.stdout:1488 write 0x5841973 thru 0x584a0ef (0x877d bytes) 2024-09-13T07:02:01.560 INFO:teuthology.orchestra.run.smithi118.stdout:1489 write 0xc342f7e thru 0xc34ffbd (0xd040 bytes) 2024-09-13T07:02:01.560 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:01.557+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:01.567 INFO:teuthology.orchestra.run.smithi118.stdout:1491 punch from 0xbb7826 to 0xbc43be, (0xcb98 bytes) 2024-09-13T07:02:01.567 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:01.567 INFO:teuthology.orchestra.run.smithi118.stdout:1493 punch from 0x7db272b to 0x7dbff44, (0xd819 bytes) 2024-09-13T07:02:01.567 INFO:teuthology.orchestra.run.smithi118.stdout:1494 punch from 0x67018eb to 0x670d1af, (0xb8c4 bytes) 2024-09-13T07:02:01.567 INFO:teuthology.orchestra.run.smithi118.stdout:1497 read 0xb4309d8 thru 0xb43b36a (0xa993 bytes) 2024-09-13T07:02:01.567 INFO:teuthology.orchestra.run.smithi118.stdout:1498 read 0x2ee5f95 thru 0x2ef2664 (0xc6d0 bytes) 2024-09-13T07:02:01.567 INFO:teuthology.orchestra.run.smithi118.stdout:1499 read 0x36e5d4b thru 0x36e9bee (0x3ea4 bytes) 2024-09-13T07:02:01.579 INFO:teuthology.orchestra.run.smithi118.stdout:1500 writesame 0xd89cb1a thru 0xd8a7016 data_size 0x39(0xa4fd bytes) 2024-09-13T07:02:01.579 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:01.580 INFO:teuthology.orchestra.run.smithi118.stdout:1503 read 0x9b384f1 thru 0x9b3ce66 (0x4976 bytes) 2024-09-13T07:02:01.580 INFO:teuthology.orchestra.run.smithi118.stdout:1505 read 0x52b6fe9 thru 0x52b93d0 (0x23e8 bytes) 2024-09-13T07:02:01.580 INFO:teuthology.orchestra.run.smithi118.stdout:1506 punch from 0xcb6c84c to 0xcb797c4, (0xcf78 bytes) 2024-09-13T07:02:01.580 INFO:teuthology.orchestra.run.smithi118.stdout:1507 read 0x1305b17 thru 0x1309d0d (0x41f7 bytes) 2024-09-13T07:02:01.580 INFO:teuthology.orchestra.run.smithi118.stdout:1508 write 0xb0f95a0 thru 0xb1062f8 (0xcd59 bytes) 2024-09-13T07:02:01.601 INFO:teuthology.orchestra.run.smithi118.stdout:1510 trunc from 0xd8a7017 to 0x7aae015 2024-09-13T07:02:01.605 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:01.606 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:01.616 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:01.620 INFO:teuthology.orchestra.run.smithi118.stdout:1513 write 0x12aa4de thru 0x12b3239 (0x8d5c bytes) 2024-09-13T07:02:01.627 INFO:teuthology.orchestra.run.smithi118.stdout:1515 trunc from 0x7aae015 to 0x820a4b1 2024-09-13T07:02:01.638 INFO:teuthology.orchestra.run.smithi118.stdout:1516 read 0x7c454df thru 0x7c52b96 (0xd6b8 bytes) 2024-09-13T07:02:01.638 INFO:teuthology.orchestra.run.smithi118.stdout:1518 read 0x38d363b thru 0x38dd128 (0x9aee bytes) 2024-09-13T07:02:01.638 INFO:teuthology.orchestra.run.smithi118.stdout:1519 punch from 0x5558a04 to 0x556595c, (0xcf58 bytes) 2024-09-13T07:02:01.639 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:01.653 INFO:teuthology.orchestra.run.smithi118.stdout:1521 write 0x96bfea7 thru 0x96cf33f (0xf499 bytes) 2024-09-13T07:02:01.661 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:01.661+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:01.666 INFO:teuthology.orchestra.run.smithi118.stdout:1523 trunc from 0x96cf340 to 0x7f12882 2024-09-13T07:02:01.687 INFO:teuthology.orchestra.run.smithi118.stdout:1524 writesame 0x1672268 thru 0x1681132 data_size 0x47(0xeecb bytes) 2024-09-13T07:02:01.694 INFO:teuthology.orchestra.run.smithi118.stdout:1525 read 0x1d899df thru 0x1d95009 (0xb62b bytes) 2024-09-13T07:02:01.714 INFO:teuthology.orchestra.run.smithi118.stdout:1526 write 0xad34212 thru 0xad41609 (0xd3f8 bytes) 2024-09-13T07:02:01.724 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:01.724 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:01.733 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:01.743 INFO:teuthology.orchestra.run.smithi118.stdout:1528 trunc from 0xad4160a to 0x18def1 2024-09-13T07:02:01.769 INFO:teuthology.orchestra.run.smithi118.stdout:1529 read 0xce6bd thru 0xdb5f2 (0xcf36 bytes) 2024-09-13T07:02:01.783 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:01.782+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:01.800 INFO:teuthology.orchestra.run.smithi118.stdout:1534 write 0xaae73c7 thru 0xaae758e (0x1c8 bytes) 2024-09-13T07:02:01.807 INFO:teuthology.orchestra.run.smithi118.stdout:1535 punch from 0x6d53a69 to 0x6d601b8, (0xc74f bytes) 2024-09-13T07:02:01.814 INFO:teuthology.orchestra.run.smithi118.stdout:1538 trunc from 0xaae758f to 0x731a1f8 2024-09-13T07:02:01.832 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:02:01.836 INFO:teuthology.orchestra.run.smithi118.stdout:1539 write 0x302a6f4 thru 0x3031384 (0x6c91 bytes) 2024-09-13T07:02:01.844 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:01.844 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:01.845 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:01.845 INFO:teuthology.orchestra.run.smithi118.stdout:1541 read 0x1f23181 thru 0x1f33079 (0xfef9 bytes) 2024-09-13T07:02:01.845 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:01.849 INFO:teuthology.orchestra.run.smithi118.stdout:1544 trunc from 0x731a1f8 to 0x8fc6cd9 2024-09-13T07:02:01.854 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:01.862 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:01.858+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:01.867 INFO:teuthology.orchestra.run.smithi118.stdout:1545 trunc from 0x8fc6cd9 to 0xaf3f232 2024-09-13T07:02:01.878 INFO:teuthology.orchestra.run.smithi118.stdout:1546 read 0x4144ea0 thru 0x414aa87 (0x5be8 bytes) 2024-09-13T07:02:01.878 INFO:teuthology.orchestra.run.smithi118.stdout:1548 read 0xa3a665f thru 0xa3b57fc (0xf19e bytes) 2024-09-13T07:02:01.888 INFO:teuthology.orchestra.run.smithi118.stdout:1549 writesame 0xaff7894 thru 0xaffb6b5 data_size 0x21(0x3e22 bytes) 2024-09-13T07:02:01.888 INFO:teuthology.orchestra.run.smithi118.stdout:1553 write 0x4ad900a thru 0x4ae3973 (0xa96a bytes) 2024-09-13T07:02:01.897 INFO:teuthology.orchestra.run.smithi118.stdout:1554 read 0x8deee2a thru 0x8dfb5de (0xc7b5 bytes) 2024-09-13T07:02:01.897 INFO:teuthology.orchestra.run.smithi118.stdout:1555 read 0x46d7d05 thru 0x46e0729 (0x8a25 bytes) 2024-09-13T07:02:01.898 INFO:teuthology.orchestra.run.smithi118.stdout:1557 read 0x9432cdb thru 0x9439c74 (0x6f9a bytes) 2024-09-13T07:02:01.898 INFO:teuthology.orchestra.run.smithi118.stdout:1558 read 0x883d846 thru 0x884a283 (0xca3e bytes) 2024-09-13T07:02:01.898 INFO:teuthology.orchestra.run.smithi118.stdout:1559 read 0x23551f9 thru 0x235e950 (0x9758 bytes) 2024-09-13T07:02:01.898 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:01.911 INFO:teuthology.orchestra.run.smithi118.stdout:1561 trunc from 0xaffb6b6 to 0x400406c 2024-09-13T07:02:01.931 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:01.954 INFO:teuthology.orchestra.run.smithi118.stdout:1564 writesame 0xb5ea611 thru 0xb5f0bdc data_size 0xa(0x65cc bytes) 2024-09-13T07:02:01.954 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:01.954 INFO:teuthology.orchestra.run.smithi118.stdout:1566 read 0xadc68d2 thru 0xadc6d96 (0x4c5 bytes) 2024-09-13T07:02:01.954 INFO:teuthology.orchestra.run.smithi118.stdout:1567 read 0x965f741 thru 0x96646b7 (0x4f77 bytes) 2024-09-13T07:02:01.954 INFO:teuthology.orchestra.run.smithi118.stdout:1568 read 0x10f5115 thru 0x10f8e94 (0x3d80 bytes) 2024-09-13T07:02:01.954 INFO:teuthology.orchestra.run.smithi118.stdout:1569 punch from 0x7d60954 to 0x7d656e2, (0x4d8e bytes) 2024-09-13T07:02:01.954 INFO:teuthology.orchestra.run.smithi118.stdout:1570 writesame 0xa8ea6bc thru 0xa8ecede data_size 0xf(0x2823 bytes) 2024-09-13T07:02:01.960 INFO:teuthology.orchestra.run.smithi118.stdout:1571 writesame 0x3369af8 thru 0x33770fd data_size 0x5(0xd606 bytes) 2024-09-13T07:02:01.963 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:01.958+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:01.968 INFO:teuthology.orchestra.run.smithi118.stdout:1573 writesame 0xaa80949 thru 0xaa8c7da data_size 0x8d(0xbe92 bytes) 2024-09-13T07:02:01.975 INFO:teuthology.orchestra.run.smithi118.stdout:1574 punch from 0x48f4f32 to 0x4900faf, (0xc07d bytes) 2024-09-13T07:02:01.975 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:01.976 INFO:teuthology.orchestra.run.smithi118.stdout:1578 punch from 0x3d0e5f0 to 0x3d1ce95, (0xe8a5 bytes) 2024-09-13T07:02:01.978 INFO:teuthology.orchestra.run.smithi118.stdout:1579 trunc from 0xb5f0bdd to 0xcb4c264 2024-09-13T07:02:01.979 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:01.979 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:01.983 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:01.988 INFO:teuthology.orchestra.run.smithi118.stdout:1580 write 0x569a446 thru 0x56a2f00 (0x8abb bytes) 2024-09-13T07:02:01.990 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:01.997 INFO:teuthology.orchestra.run.smithi118.stdout:1581 read 0x37d27e1 thru 0x37d7521 (0x4d41 bytes) 2024-09-13T07:02:02.003 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:02.010 INFO:teuthology.orchestra.run.smithi118.stdout:1583 trunc from 0xcb4c264 to 0x54d1ef8 2024-09-13T07:02:02.033 INFO:teuthology.orchestra.run.smithi118.stdout:1584 write 0x1fd5087 thru 0x1fe1e92 (0xce0c bytes) 2024-09-13T07:02:02.038 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:02.042 INFO:teuthology.orchestra.run.smithi118.stdout:1585 punch from 0x4f896ec to 0x4f90406, (0x6d1a bytes) 2024-09-13T07:02:02.056 INFO:teuthology.orchestra.run.smithi118.stdout:1586 writesame 0x6a2f8c5 thru 0x6a34e35 data_size 0x17(0x5571 bytes) 2024-09-13T07:02:02.064 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:02.061+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:02.064 INFO:teuthology.orchestra.run.smithi118.stdout:1588 trunc from 0x6a34e36 to 0x360a3dc 2024-09-13T07:02:02.071 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:02.088 INFO:teuthology.orchestra.run.smithi118.stdout:1590 read 0xc65d90 thru 0xc72809 (0xca7a bytes) 2024-09-13T07:02:02.093 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:02.113 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:02.113 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:02.113 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:02.113 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:02.113 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:02.114 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:02.119 INFO:teuthology.orchestra.run.smithi118.stdout:1591 write 0xd3c9dcd thru 0xd3d7d72 (0xdfa6 bytes) 2024-09-13T07:02:02.123 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:02.129 INFO:teuthology.orchestra.run.smithi118.stdout:1592 write 0x4eba6bd thru 0x4ec0950 (0x6294 bytes) 2024-09-13T07:02:02.135 INFO:teuthology.orchestra.run.smithi118.stdout:1593 write 0xe844fc thru 0xe8aa95 (0x659a bytes) 2024-09-13T07:02:02.136 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:02.140 INFO:teuthology.orchestra.run.smithi118.stdout:1594 read 0xcd07e41 thru 0xcd14291 (0xc451 bytes) 2024-09-13T07:02:02.140 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:02.140 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:02.141 INFO:teuthology.orchestra.run.smithi118.stdout:1599 punch from 0x406842 to 0x41165c, (0xae1a bytes) 2024-09-13T07:02:02.141 INFO:teuthology.orchestra.run.smithi118.stdout:1600 write 0x45cc013 thru 0x45d61f8 (0xa1e6 bytes) 2024-09-13T07:02:02.145 INFO:teuthology.orchestra.run.smithi118.stdout:1603 writesame 0x5ce2ab9 thru 0x5ce6740 data_size 0x34(0x3c88 bytes) 2024-09-13T07:02:02.150 INFO:teuthology.orchestra.run.smithi118.stdout:1604 punch from 0xbcd5756 to 0xbcd6eff, (0x17a9 bytes) 2024-09-13T07:02:02.150 INFO:teuthology.orchestra.run.smithi118.stdout:1605 punch from 0xa19e371 to 0xa1a3576, (0x5205 bytes) 2024-09-13T07:02:02.151 INFO:teuthology.orchestra.run.smithi118.stdout:1606 read 0x5cd5f42 thru 0x5cd7ee8 (0x1fa7 bytes) 2024-09-13T07:02:02.151 INFO:teuthology.orchestra.run.smithi118.stdout:1607 write 0xa3d5264 thru 0xa3e2ee8 (0xdc85 bytes) 2024-09-13T07:02:02.158 INFO:teuthology.orchestra.run.smithi118.stdout:1608 write 0xaed8e6f thru 0xaeda791 (0x1923 bytes) 2024-09-13T07:02:02.161 INFO:teuthology.orchestra.run.smithi118.stdout:1609 punch from 0xa070a4c to 0xa07b9b9, (0xaf6d bytes) 2024-09-13T07:02:02.161 INFO:teuthology.orchestra.run.smithi118.stdout:1611 read 0x25b2685 thru 0x25baee2 (0x885e bytes) 2024-09-13T07:02:02.161 INFO:teuthology.orchestra.run.smithi118.stdout:1612 read 0x26e0fa3 thru 0x26e3a83 (0x2ae1 bytes) 2024-09-13T07:02:02.161 INFO:teuthology.orchestra.run.smithi118.stdout:1613 write 0x3d0fbd6 thru 0x3d1beac (0xc2d7 bytes) 2024-09-13T07:02:02.163 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:02.165 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:02.165+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:02.166 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:02.166 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:02.166 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:02.166 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:02.166 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:02.166 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:02.167 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:02.167 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:02.173 INFO:teuthology.orchestra.run.smithi118.stdout:1614 read 0x4095e97 thru 0x409b4ac (0x5616 bytes) 2024-09-13T07:02:02.173 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:02:02.174 INFO:teuthology.orchestra.run.smithi118.stdout:1616 write 0x40a71e3 thru 0x40b3994 (0xc7b2 bytes) 2024-09-13T07:02:02.179 INFO:teuthology.orchestra.run.smithi118.stdout:1619 punch from 0x8eea7c5 to 0x8eeee9e, (0x46d9 bytes) 2024-09-13T07:02:02.179 INFO:teuthology.orchestra.run.smithi118.stdout:1621 punch from 0x181fdc1 to 0x182ea17, (0xec56 bytes) 2024-09-13T07:02:02.180 INFO:teuthology.orchestra.run.smithi118.stdout:1622 writesame 0x9d36341 thru 0x9d3a211 data_size 0x1(0x3ed1 bytes) 2024-09-13T07:02:02.194 INFO:teuthology.orchestra.run.smithi118.stdout:1623 punch from 0x2f435 to 0x33f64, (0x4b2f bytes) 2024-09-13T07:02:02.195 INFO:teuthology.orchestra.run.smithi118.stdout:1625 punch from 0x1e19dae to 0x1e1ad02, (0xf54 bytes) 2024-09-13T07:02:02.195 INFO:teuthology.orchestra.run.smithi118.stdout:1627 write 0x23515f1 thru 0x236104a (0xfa5a bytes) 2024-09-13T07:02:02.205 INFO:teuthology.orchestra.run.smithi118.stdout:1628 write 0x5afe4d thru 0x5b5909 (0x5abd bytes) 2024-09-13T07:02:02.218 INFO:teuthology.orchestra.run.smithi118.stdout:1629 write 0xdc45a56 thru 0xdc4643c (0x9e7 bytes) 2024-09-13T07:02:02.221 INFO:teuthology.orchestra.run.smithi118.stdout:1630 write 0x6e36ee4 thru 0x6e466b0 (0xf7cd bytes) 2024-09-13T07:02:02.230 INFO:teuthology.orchestra.run.smithi118.stdout:1632 clone 19 order 22 su 32768 sc 6 2024-09-13T07:02:02.237 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:02.238 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:02.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:02.238 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:02.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.238 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:02.238 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:02.238 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.730410012999997, 2024-09-13T07:02:02.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:02:02.238 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.238 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:02.238 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.239 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.239 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.239 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:02:02.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.239 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:02.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:02.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:02:02.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:02.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:02:02.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:02:02.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:02:02.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:02:02.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:02:02.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:02:02.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:02:02.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:02:02.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:02:02.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:02:02.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:02.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:02:02.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:02:02.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:02:02.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:02:02.245 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.245 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.245 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.245 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:02.245 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:02.245 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.705582303, 2024-09-13T07:02:02.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:02:02.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:02:02.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:02:02.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:02:02.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:02:02.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:02:02.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:02:02.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:02:02.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:02.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:02:02.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:02:02.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:02:02.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:02:02.251 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.251 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.251 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.251 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:02.252 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:02.252 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.166910479000002, 2024-09-13T07:02:02.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:02:02.252 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.252 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:02.252 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.252 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.252 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.252 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:02:02.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.253 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:02.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:02.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:02:02.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:02.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:02:02.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:02:02.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:02:02.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:02:02.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:02:02.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:02:02.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:02:02.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:02:02.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:02:02.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:02:02.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:02.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:02:02.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:02:02.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:02.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:02:02.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:02:02.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:02.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:02:02.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:02:02.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:02.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:02:02.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:02:02.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:02:02.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:02:02.261 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.261 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.261 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.261 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:02.261 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:02.262 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.137285286000001, 2024-09-13T07:02:02.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:02:02.262 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.262 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:02.262 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.262 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.262 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.262 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:02:02.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.263 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:02.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:02.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:02:02.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:02.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:02:02.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:02:02.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:02:02.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:02:02.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:02:02.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:02:02.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:02:02.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:02:02.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:02:02.268 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.268 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.268 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.268 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:02.268 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:02.268 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 35.604571655000001, 2024-09-13T07:02:02.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:02.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:02.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:02.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:02:02.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:02:02.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:02:02.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:02:02.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:02:02.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:02.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:02:02.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:02:02.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:02.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:02:02.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:02:02.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:02.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:02:02.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:02:02.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:02.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:02:02.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:02:02.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:02:02.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:02:02.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.274 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.274 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 35.604489199, 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:02.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:02.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:02:02.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:02.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:02:02.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:02:02.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:02:02.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:02:02.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:02.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:02:02.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:02:02.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:02:02.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:02:02.280 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.280 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.280 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 35.604457388999997, 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:02.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:02.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:02:02.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:02:02.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:02:02.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:02:02.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:02.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:02:02.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:02:02.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:02.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:02:02.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:02:02.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:02.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:02:02.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:02:02.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:02:02.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:02:02.286 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.286 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.286 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 35.604402231000002, 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:02.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:02.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:02:02.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:02.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:02:02.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:02:02.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:02:02.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:02:02.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:02:02.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:02:02.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:02:02.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:02:02.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:02:02.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:02:02.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:02.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:02:02.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:02:02.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:02:02.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:02:02.293 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.293 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.293 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.293 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:02.293 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:02.293 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 35.604356952000003, 2024-09-13T07:02:02.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:02:02.293 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:02.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:02:02.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:02.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:02:02.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:02:02.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:02:02.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:02.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 35.604318417999998, 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.299 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:02:02.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:02.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:02:02.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:02:02.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:02:02.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:02:02.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:02:02.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:02:02.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:02:02.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:02:02.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:02:02.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:02:02.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:02.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:02:02.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:02:02.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:02.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:02:02.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:02:02.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:02.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:02:02.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:02:02.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:02.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:02:02.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:02:02.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:02:02.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:02:02.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.309 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.310 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:02.310 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:02.310 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 35.604256798999998, 2024-09-13T07:02:02.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:02:02.311 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.311 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:02.311 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.311 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.311 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.311 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:02:02.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.312 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:02.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:02.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:02:02.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:02.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:02:02.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:02:02.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:02:02.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:02:02.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:02:02.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:02:02.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:02:02.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:02:02.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:02:02.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:02:02.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:02.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:02:02.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:02:02.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:02.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:02:02.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:02:02.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:02.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:02:02.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:02:02.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:02.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:02:02.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:02:02.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:02:02.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:02:02.319 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.319 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.319 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 35.590034115999998, 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:02.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:02.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:02:02.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:02.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:02:02.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:02:02.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:02:02.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:02:02.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:02:02.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:02:02.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:02:02.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:02:02.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:02:02.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:02:02.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:02.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:02:02.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:02:02.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:02.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:02:02.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:02:02.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:02.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:02:02.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:02:02.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:02.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:02:02.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:02:02.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:02:02.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:02:02.327 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.327 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.327 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.327 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:02.327 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:02.327 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.578155595999998, 2024-09-13T07:02:02.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:02:02.328 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.328 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:02.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.328 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:02:02.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.328 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:02.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:02.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:02:02.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:02:02.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:02:02.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:02:02.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:02:02.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:02:02.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:02:02.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.577598535, 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:02.332 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.333 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.333 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.333 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:02:02.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.333 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:02.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:02:02.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:02:02.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:02:02.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:02:02.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.336 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.337 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:02.337 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:02.337 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.57757638, 2024-09-13T07:02:02.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:02:02.337 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.337 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:02.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.337 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:02:02.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.337 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:02:02.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:02:02.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:02:02.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:02:02.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:02:02.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:02:02.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:02:02.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:02:02.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:02:02.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:02:02.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:02:02.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:02:02.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:02:02.341 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.341 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.341 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.341 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:02.341 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:02.341 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.577556662999999, 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:02.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:02.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:02:02.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:02:02.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:02:02.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:02:02.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:02:02.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:02:02.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:02:02.345 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.577539333000001, 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.346 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:02:02.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.347 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:02.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:02.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:02:02.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:02:02.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:02:02.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:02:02.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:02.350 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.577243671000002, 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:02.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:02.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:02:02.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:02:02.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:02:02.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:02:02.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:02:02.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:02:02.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.016040852, 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:02:02.355 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.356 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:02.356 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.356 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.356 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.356 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:02:02.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.356 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:02.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:02.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:02:02.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:02.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:02:02.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:02:02.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:02:02.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:02:02.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:02:02.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:02:02.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:02:02.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:02.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:02:02.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:02:02.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:02.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:02:02.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:02:02.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:02.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:02.361 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 21.983961949000001, 2024-09-13T07:02:02.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:02:02.362 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.362 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:02.362 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:02.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:02.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:02:02.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:02:02.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:02:02.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:02:02.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:02:02.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:02:02.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:02:02.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:02:02.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:02:02.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:02:02.365 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.365 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.365 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.365 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.365 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.365 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:02.367 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:02.367 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:02.365+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:02.468 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:02.465+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:02.517 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:02.518 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:02.518 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:02.518 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:02.527 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:02.569 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:02.566+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:02.632 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:02.632 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:02.633 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:02.633 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:02.633 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:02.642 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:02.670 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:02.669+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:02.745 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:02.746 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:02.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:02.746 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:02.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.746 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:02:02.746 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:02.746 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.252307489, 2024-09-13T07:02:02.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:02:02.746 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.746 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:02.746 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.746 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.746 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:02:02.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:02:02.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7658239250000003, 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.751 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:02.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:02:02.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.755 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.756 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.756 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:02.756 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7657539959999999, 2024-09-13T07:02:02.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:02:02.756 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.756 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:02.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.756 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:02:02.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.757 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:02.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:02.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:02:02.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:02:02.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:02:02.760 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.760 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.760 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.760 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.760 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:02.760 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7657381240000003, 2024-09-13T07:02:02.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:02:02.760 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:02:02.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7656676669999998, 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:02:02.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:02:02.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7623156289999997, 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.769 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:02.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:02:02.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:02:02.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:02:02.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:02:02.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:02:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7622993280000001, 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.773 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:02:02.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.774 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:02.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:02.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:02:02.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.762279543, 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.777 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.780 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.76226327, 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:02:02.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:02:02.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:02:02.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:02:02.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:02:02.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:02:02.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:02:02.784 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.785 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.785 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.785 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:02.785 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7622454449999996, 2024-09-13T07:02:02.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:02:02.785 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.785 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:02:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7622293520000003, 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:02:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:02:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:02:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:02:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:02:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:02:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7622081659999997, 2024-09-13T07:02:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:02:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:02:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:02:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:02:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:02:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:02:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:02:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7621921330000001, 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:02:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:02:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:02:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:02:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:02:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7621428940000001, 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:02:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7621248339999998, 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:02:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:02:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7621083190000002, 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:02:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7620899540000003, 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7620738559999998, 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:02:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:02:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7620579210000002, 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:02:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:02:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:02:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.7620414239999995, 2024-09-13T07:02:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:02:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:02:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:02:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:02:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:02:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:02.829 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:02.831 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:02.831 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:02.829+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:02.931 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:02.930+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:02.985 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:02.985 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:02.985 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:02.985 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:02.995 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:03.032 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:03.030+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:03.100 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:03.100 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:03.100 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:03.100 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:03.100 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:03.109 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:03.132 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:03.129+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:03.179 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:02:03.223 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:03.223 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:03.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:03.223 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:03.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.223 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:434 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'4, mlcod=26'2)", 2024-09-13T07:02:03.223 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:03.223 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 38.686587838000001, 2024-09-13T07:02:03.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001931945, 2024-09-13T07:02:03.223 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.224 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.224 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:03.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:03.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535695+0000", 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535709+0000", 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3577999999999999e-05 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535732+0000", 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3085e-05 2024-09-13T07:02:03.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535742+0000", 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.048e-05 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535840+0000", 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7159999999999998e-05 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535871+0000", 2024-09-13T07:02:03.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1510999999999997e-05 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537572+0000", 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017008279999999999 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537629+0000", 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6520000000000001e-05 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:435 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'5, mlcod=26'2)", 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 38.685456131999999, 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017126769999999999, 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:03.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:03.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:03.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536835+0000", 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4939999999999997e-06 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536854+0000", 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8655999999999999e-05 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536862+0000", 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6570000000000003e-06 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536926+0000", 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4037999999999999e-05 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536958+0000", 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1584000000000001e-05 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538493+0000", 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00153513 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538541+0000", 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7883999999999998e-05 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:483 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'24, mlcod=26'21)", 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 38.606977503000003, 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018371780000000001, 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.232 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615306+0000", 2024-09-13T07:02:03.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615322+0000", 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5519000000000002e-05 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615335+0000", 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3854e-05 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615345+0000", 2024-09-13T07:02:03.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4930000000000004e-06 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615431+0000", 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6346000000000003e-05 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615480+0000", 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8303999999999997e-05 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617102+0000", 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016225250000000001 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617144+0000", 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1936e-05 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:518 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'38, mlcod=26'36)", 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.897400765, 2024-09-13T07:02:03.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018201179999999999, 2024-09-13T07:02:03.237 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.237 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.237 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:03.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324883+0000", 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324891+0000", 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1499999999999999e-06 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324924+0000", 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2601999999999997e-05 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324942+0000", 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7796999999999999e-05 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325060+0000", 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011786900000000001 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325123+0000", 2024-09-13T07:02:03.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3474999999999998e-05 2024-09-13T07:02:03.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326620+0000", 2024-09-13T07:02:03.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014968830000000001 2024-09-13T07:02:03.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326704+0000", 2024-09-13T07:02:03.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3716999999999998e-05 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:597 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'39, mlcod=26'37)", 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.780573437000001, 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017959829999999999, 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441709+0000", 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441728+0000", 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9627999999999999e-05 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441765+0000", 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6559e-05 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441783+0000", 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7691000000000001e-05 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441904+0000", 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121184 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441966+0000", 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2428999999999995e-05 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443448+0000", 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00148197 2024-09-13T07:02:03.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443507+0000", 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8699000000000002e-05 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.165693142999999, 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:02:03.245 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.249 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1548 3.4 e27/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 27'52, mlcod=27'50)", 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.080071607000001, 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017825779999999999, 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142211+0000", 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142232+0000", 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0888000000000001e-05 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142268+0000", 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6261999999999999e-05 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142285+0000", 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7640000000000001e-05 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142381+0000", 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5171000000000001e-05 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142450+0000", 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9778999999999997e-05 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143953+0000", 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001503129 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143995+0000", 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1878000000000001e-05 2024-09-13T07:02:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.254 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.254 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1550 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'14, mlcod=27'12)", 2024-09-13T07:02:03.256 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:03.256 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.079423896000002, 2024-09-13T07:02:03.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002750452, 2024-09-13T07:02:03.256 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.256 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.256 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:03.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142859+0000", 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142874+0000", 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5149e-05 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142911+0000", 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6439000000000001e-05 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142926+0000", 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5079000000000001e-05 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143012+0000", 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6230000000000006e-05 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143063+0000", 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1369000000000002e-05 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145539+0000", 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024753420000000002 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145611+0000", 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2392999999999996e-05 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1668 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'17, mlcod=27'15)", 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.881008434000002, 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033976620000000001, 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.260 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341281+0000", 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8109999999999999e-06 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341298+0000", 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6575e-05 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341308+0000", 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0369e-05 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341434+0000", 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125454 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341453+0000", 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9159000000000001e-05 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344638+0000", 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031851750000000002 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344674+0000", 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5410999999999997e-05 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1666 3.4 e28/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 28'55, mlcod=27'53)", 2024-09-13T07:02:03.264 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.880954762000002, 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001805809, 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:03.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341329+0000", 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341349+0000", 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.999e-05 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341357+0000", 2024-09-13T07:02:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0620000000000003e-06 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341361+0000", 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341710+0000", 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034873400000000001 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341736+0000", 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5772e-05 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343097+0000", 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013615319999999999 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343135+0000", 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8111999999999997e-05 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.268 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1800 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'20, mlcod=28'18)", 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.678584295999997, 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001874398, 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543699+0000", 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543713+0000", 2024-09-13T07:02:03.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4786e-05 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543737+0000", 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3626e-05 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543752+0000", 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.489e-05 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543832+0000", 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0396999999999996e-05 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543994+0000", 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016129100000000001 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545471+0000", 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014771960000000001 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545575+0000", 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103732 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1882 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'23, mlcod=28'21)", 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.582951315000003, 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001767172, 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639332+0000", 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:03.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639343+0000", 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1436e-05 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639361+0000", 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7915e-05 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639370+0000", 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3189999999999995e-06 2024-09-13T07:02:03.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639425+0000", 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4917999999999998e-05 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639454+0000", 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9862999999999999e-05 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641059+0000", 2024-09-13T07:02:03.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016048950000000001 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641100+0000", 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0883999999999997e-05 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3069 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'33, mlcod=31'31)", 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.059811599, 2024-09-13T07:02:03.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018597010000000001, 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162471+0000", 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162520+0000", 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8466e-05 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162546+0000", 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6089999999999999e-05 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162562+0000", 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5962000000000001e-05 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162646+0000", 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3985000000000003e-05 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162691+0000", 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5785999999999998e-05 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164265+0000", 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015740890000000001 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164333+0000", 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7014999999999994e-05 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.281 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3121 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'36, mlcod=31'34)", 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.945865212000001, 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019657780000000001, 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276417+0000", 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276497+0000", 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9729999999999997e-05 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276537+0000", 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0102999999999999e-05 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276553+0000", 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6518999999999999e-05 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276662+0000", 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109252 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276729+0000", 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7056000000000003e-05 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278270+0000", 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015408819999999999 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278385+0000", 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114613 2024-09-13T07:02:03.285 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3390 3.0 e32/22 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:head v 32'45, mlcod=31'42)", 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.726776864999998, 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002246587, 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:03.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:03.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495507+0000", 2024-09-13T07:02:03.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495530+0000", 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2592999999999999e-05 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495550+0000", 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0152999999999998e-05 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495562+0000", 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2336e-05 2024-09-13T07:02:03.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495662+0000", 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100317 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495717+0000", 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4304e-05 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497707+0000", 2024-09-13T07:02:03.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019906849999999999 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497754+0000", 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6715999999999998e-05 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3399 3.4 e32/22 3:22863a52:::rbd_data.11945d91e99c.0000000000000013:head v 32'65, mlcod=31'62)", 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.722840193000003, 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028830779999999999, 2024-09-13T07:02:03.290 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499442+0000", 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499459+0000", 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7042e-05 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499493+0000", 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3788000000000002e-05 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499512+0000", 2024-09-13T07:02:03.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9055999999999998e-05 2024-09-13T07:02:03.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499608+0000", 2024-09-13T07:02:03.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6168000000000005e-05 2024-09-13T07:02:03.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499658+0000", 2024-09-13T07:02:03.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9469999999999999e-05 2024-09-13T07:02:03.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502277+0000", 2024-09-13T07:02:03.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026188639999999998 2024-09-13T07:02:03.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502327+0000", 2024-09-13T07:02:03.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0578999999999998e-05 2024-09-13T07:02:03.294 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3395 3.0 e32/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 32'47, mlcod=31'43)", 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.722742107999998, 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027505870000000001, 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:03.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:03.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:03.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499560+0000", 2024-09-13T07:02:03.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8416000000000001e-05 2024-09-13T07:02:03.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499576+0000", 2024-09-13T07:02:03.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5602000000000001e-05 2024-09-13T07:02:03.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499587+0000", 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1112e-05 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499654+0000", 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7164999999999998e-05 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499674+0000", 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9638000000000001e-05 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502210+0000", 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025367100000000002 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502293+0000", 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2607000000000004e-05 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:4534 3.0 e32/22 3:00392e9e:::rbd_data.11944cdc0656.000000000000000c:head v 32'51, mlcod=32'49)", 2024-09-13T07:02:03.299 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.311040921999997, 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017597330000000001, 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:03.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911242+0000", 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911306+0000", 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4480999999999995e-05 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911338+0000", 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1733000000000003e-05 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911351+0000", 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3241000000000001e-05 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911455+0000", 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103723 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911495+0000", 2024-09-13T07:02:03.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0683000000000003e-05 2024-09-13T07:02:03.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.912933+0000", 2024-09-13T07:02:03.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001437219 2024-09-13T07:02:03.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.913003+0000", 2024-09-13T07:02:03.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0629000000000004e-05 2024-09-13T07:02:03.303 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.303 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11205 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'50, mlcod=38'47)", 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 24.597700437, 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031049490000000001, 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:03.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624583+0000", 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625339+0000", 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00075563200000000001 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625358+0000", 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9916999999999999e-05 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625366+0000", 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8050000000000007e-06 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625415+0000", 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8231e-05 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625447+0000", 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2703999999999997e-05 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627652+0000", 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022050659999999999 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627689+0000", 2024-09-13T07:02:03.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6612000000000001e-05 2024-09-13T07:02:03.308 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.308 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.308 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.308 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11206 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'52, mlcod=38'47)", 2024-09-13T07:02:03.308 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:03.308 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 24.596429771, 2024-09-13T07:02:03.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022238470000000002, 2024-09-13T07:02:03.308 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.308 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.308 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625854+0000", 2024-09-13T07:02:03.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625863+0000", 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5380000000000008e-06 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625883+0000", 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9247000000000001e-05 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625893+0000", 2024-09-13T07:02:03.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0176e-05 2024-09-13T07:02:03.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626411+0000", 2024-09-13T07:02:03.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051847600000000003 2024-09-13T07:02:03.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626431+0000", 2024-09-13T07:02:03.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0199000000000001e-05 2024-09-13T07:02:03.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628028+0000", 2024-09-13T07:02:03.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015968110000000001 2024-09-13T07:02:03.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628078+0000", 2024-09-13T07:02:03.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0170000000000002e-05 2024-09-13T07:02:03.312 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.312 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.312 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.312 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.312 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.312 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:03.314 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:03.315 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:03.310+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:03.414 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:03.413+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:03.472 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:03.472 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:03.472 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:03.472 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:03.482 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:03.489 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-13T07:02:03.515 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:03.514+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:03.594 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:03.594 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:03.594 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:03.594 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:03.594 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:03.604 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:03.617 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:03.614+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:03.718 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:03.714+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:03.763 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:03.764 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:03.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:03.764 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:03.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.764 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1155 3.7 3:e095a6bd:::rbd_data.11943a018f34.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:03.764 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:02:03.764 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.929602281000001, 2024-09-13T07:02:03.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022798902999999999, 2024-09-13T07:02:03.764 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.764 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.764 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.765 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.765 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.765 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1155 2024-09-13T07:02:03.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.765 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:02:03.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:02:03.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832762+0000", 2024-09-13T07:02:03.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:03.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832776+0000", 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3828e-05 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832806+0000", 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0159e-05 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832820+0000", 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.384e-05 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832904+0000", 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4506999999999996e-05 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.833121+0000", 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000217223 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.839072+0000", 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0059509849999999998 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:03.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855489+0000", 2024-09-13T07:02:03.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016416907000000001 2024-09-13T07:02:03.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:03.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855503+0000", 2024-09-13T07:02:03.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3658999999999999e-05 2024-09-13T07:02:03.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855535+0000", 2024-09-13T07:02:03.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1696000000000003e-05 2024-09-13T07:02:03.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855563+0000", 2024-09-13T07:02:03.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8578999999999999e-05 2024-09-13T07:02:03.770 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.770 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.770 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1200 3.7 3:e095a6bd:::rbd_data.11943a018f34.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.783263331000001, 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018736263, 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1200 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:02:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:02:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979100+0000", 2024-09-13T07:02:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979115+0000", 2024-09-13T07:02:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4823e-05 2024-09-13T07:02:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979149+0000", 2024-09-13T07:02:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3772999999999997e-05 2024-09-13T07:02:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979163+0000", 2024-09-13T07:02:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3845999999999999e-05 2024-09-13T07:02:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979333+0000", 2024-09-13T07:02:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000169622 2024-09-13T07:02:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979603+0000", 2024-09-13T07:02:03.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00027078999999999997 2024-09-13T07:02:03.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:03.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.986017+0000", 2024-09-13T07:02:03.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064139590000000003 2024-09-13T07:02:03.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:03.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997476+0000", 2024-09-13T07:02:03.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011458582 2024-09-13T07:02:03.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:03.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997588+0000", 2024-09-13T07:02:03.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011161 2024-09-13T07:02:03.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997732+0000", 2024-09-13T07:02:03.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000144 2024-09-13T07:02:03.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997839+0000", 2024-09-13T07:02:03.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107892 2024-09-13T07:02:03.777 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.777 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.777 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.778 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1531 3.c e27/22 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head v 27'9, mlcod=27'7)", 2024-09-13T07:02:03.778 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:02:03.778 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.687835174, 2024-09-13T07:02:03.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039127038000000003, 2024-09-13T07:02:03.778 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.778 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.778 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:02:03.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:02:03.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074530+0000", 2024-09-13T07:02:03.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:03.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099434+0000", 2024-09-13T07:02:03.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.024903926999999999 2024-09-13T07:02:03.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099459+0000", 2024-09-13T07:02:03.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5384999999999998e-05 2024-09-13T07:02:03.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099469+0000", 2024-09-13T07:02:03.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.024e-05 2024-09-13T07:02:03.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099566+0000", 2024-09-13T07:02:03.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6533999999999994e-05 2024-09-13T07:02:03.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099600+0000", 2024-09-13T07:02:03.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4007000000000002e-05 2024-09-13T07:02:03.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113477+0000", 2024-09-13T07:02:03.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01387741 2024-09-13T07:02:03.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113658+0000", 2024-09-13T07:02:03.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000180953 2024-09-13T07:02:03.782 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.782 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.782 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.782 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4022 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:03.783 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:02:03.783 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.107942729999998, 2024-09-13T07:02:03.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014992557, 2024-09-13T07:02:03.783 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.783 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:03.783 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.783 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.783 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.783 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4022 2024-09-13T07:02:03.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.783 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:02:03.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654424+0000", 2024-09-13T07:02:03.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654423+0000", 2024-09-13T07:02:03.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654427+0000", 2024-09-13T07:02:03.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0960000000000003e-06 2024-09-13T07:02:03.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654437+0000", 2024-09-13T07:02:03.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7070000000000004e-06 2024-09-13T07:02:03.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654445+0000", 2024-09-13T07:02:03.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9049999999999997e-06 2024-09-13T07:02:03.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666423+0000", 2024-09-13T07:02:03.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011978223 2024-09-13T07:02:03.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666521+0000", 2024-09-13T07:02:03.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7832999999999997e-05 2024-09-13T07:02:03.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669416+0000", 2024-09-13T07:02:03.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002895393 2024-09-13T07:02:03.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.787 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.787 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4023 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:03.787 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:02:03.787 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.107913844000002, 2024-09-13T07:02:03.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015452399, 2024-09-13T07:02:03.788 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.788 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:03.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.788 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4023 2024-09-13T07:02:03.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.788 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:02:03.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654452+0000", 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654456+0000", 2024-09-13T07:02:03.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7570000000000002e-06 2024-09-13T07:02:03.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654465+0000", 2024-09-13T07:02:03.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7140000000000005e-06 2024-09-13T07:02:03.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654486+0000", 2024-09-13T07:02:03.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0290000000000001e-05 2024-09-13T07:02:03.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669455+0000", 2024-09-13T07:02:03.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014969651 2024-09-13T07:02:03.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669722+0000", 2024-09-13T07:02:03.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026677899999999997 2024-09-13T07:02:03.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669905+0000", 2024-09-13T07:02:03.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018285999999999999 2024-09-13T07:02:03.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.792 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.792 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4024 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:03.792 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:02:03.792 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.107873044000002, 2024-09-13T07:02:03.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015682979, 2024-09-13T07:02:03.792 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.792 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:03.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.793 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4024 2024-09-13T07:02:03.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.793 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:02:03.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:02:03.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:02:03.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654497+0000", 2024-09-13T07:02:03.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4800000000000003e-06 2024-09-13T07:02:03.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654507+0000", 2024-09-13T07:02:03.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8749999999999995e-06 2024-09-13T07:02:03.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654515+0000", 2024-09-13T07:02:03.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0350000000000001e-06 2024-09-13T07:02:03.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669936+0000", 2024-09-13T07:02:03.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015420916 2024-09-13T07:02:03.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670030+0000", 2024-09-13T07:02:03.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3712e-05 2024-09-13T07:02:03.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670176+0000", 2024-09-13T07:02:03.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000146613 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4025 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.107844036000003, 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016683559000000001, 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.798 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4025 2024-09-13T07:02:03.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.798 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:02:03.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:02:03.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654526+0000", 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654535+0000", 2024-09-13T07:02:03.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4199999999999996e-06 2024-09-13T07:02:03.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654543+0000", 2024-09-13T07:02:03.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:02:03.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670307+0000", 2024-09-13T07:02:03.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015763482999999998 2024-09-13T07:02:03.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670399+0000", 2024-09-13T07:02:03.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2068e-05 2024-09-13T07:02:03.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671206+0000", 2024-09-13T07:02:03.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000807352 2024-09-13T07:02:03.803 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.803 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.804 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.804 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4026 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 13238272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:03.804 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:02:03.804 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.107816307, 2024-09-13T07:02:03.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016876344000000001, 2024-09-13T07:02:03.805 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.805 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:03.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.805 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4026 2024-09-13T07:02:03.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.805 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:02:03.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:02:03.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654554+0000", 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1119999999999998e-06 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654564+0000", 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0327e-05 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654571+0000", 2024-09-13T07:02:03.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3440000000000003e-06 2024-09-13T07:02:03.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671239+0000", 2024-09-13T07:02:03.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016668039999999999 2024-09-13T07:02:03.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671327+0000", 2024-09-13T07:02:03.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7818999999999994e-05 2024-09-13T07:02:03.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671427+0000", 2024-09-13T07:02:03.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9324e-05 2024-09-13T07:02:03.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.809 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.809 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4027 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:03.809 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:02:03.809 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.107787545000001, 2024-09-13T07:02:03.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017835597000000002, 2024-09-13T07:02:03.809 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4027 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:02:03.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654578+0000", 2024-09-13T07:02:03.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654590+0000", 2024-09-13T07:02:03.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1595e-05 2024-09-13T07:02:03.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654601+0000", 2024-09-13T07:02:03.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0827e-05 2024-09-13T07:02:03.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654609+0000", 2024-09-13T07:02:03.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4919999999999993e-06 2024-09-13T07:02:03.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672157+0000", 2024-09-13T07:02:03.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017547770000000001 2024-09-13T07:02:03.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672234+0000", 2024-09-13T07:02:03.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7509999999999995e-05 2024-09-13T07:02:03.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672415+0000", 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018005500000000001 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11152 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608627+0000", 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.153739553000001, 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012436806, 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.815 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11152 2024-09-13T07:02:03.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.815 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608627+0000", 2024-09-13T07:02:03.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608627+0000", 2024-09-13T07:02:03.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608626+0000", 2024-09-13T07:02:03.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:03.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608631+0000", 2024-09-13T07:02:03.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4109999999999998e-06 2024-09-13T07:02:03.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608642+0000", 2024-09-13T07:02:03.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1158000000000001e-05 2024-09-13T07:02:03.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608650+0000", 2024-09-13T07:02:03.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5110000000000003e-06 2024-09-13T07:02:03.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620871+0000", 2024-09-13T07:02:03.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012221019 2024-09-13T07:02:03.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.620956+0000", 2024-09-13T07:02:03.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5223999999999996e-05 2024-09-13T07:02:03.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621064+0000", 2024-09-13T07:02:03.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010727599999999999 2024-09-13T07:02:03.818 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.819 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.819 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11153 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:02:03.819 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608658+0000", 2024-09-13T07:02:03.819 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.153708437999999, 2024-09-13T07:02:03.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012691595, 2024-09-13T07:02:03.819 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.819 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:03.819 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.820 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11153 2024-09-13T07:02:03.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.820 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608658+0000", 2024-09-13T07:02:03.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608658+0000", 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608657+0000", 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608661+0000", 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7629999999999998e-06 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608672+0000", 2024-09-13T07:02:03.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0732e-05 2024-09-13T07:02:03.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608680+0000", 2024-09-13T07:02:03.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8429999999999993e-06 2024-09-13T07:02:03.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621106+0000", 2024-09-13T07:02:03.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01242594 2024-09-13T07:02:03.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621190+0000", 2024-09-13T07:02:03.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4005000000000006e-05 2024-09-13T07:02:03.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621350+0000", 2024-09-13T07:02:03.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000699999999999 2024-09-13T07:02:03.823 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.824 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.824 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.824 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11173 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:02:03.824 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608785+0000", 2024-09-13T07:02:03.824 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.153581891999998, 2024-09-13T07:02:03.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012946599, 2024-09-13T07:02:03.824 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.824 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:03.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.825 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.825 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.825 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11173 2024-09-13T07:02:03.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.825 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608785+0000", 2024-09-13T07:02:03.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608785+0000", 2024-09-13T07:02:03.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608784+0000", 2024-09-13T07:02:03.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608788+0000", 2024-09-13T07:02:03.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5070000000000002e-06 2024-09-13T07:02:03.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608799+0000", 2024-09-13T07:02:03.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0431000000000001e-05 2024-09-13T07:02:03.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608807+0000", 2024-09-13T07:02:03.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5220000000000007e-06 2024-09-13T07:02:03.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621397+0000", 2024-09-13T07:02:03.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01258975 2024-09-13T07:02:03.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621478+0000", 2024-09-13T07:02:03.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0408000000000007e-05 2024-09-13T07:02:03.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621731+0000", 2024-09-13T07:02:03.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025365899999999999 2024-09-13T07:02:03.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.829 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.829 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11174 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:02:03.829 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:02:03.829 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.153550974000002, 2024-09-13T07:02:03.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01318603, 2024-09-13T07:02:03.830 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.830 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:03.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.830 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11174 2024-09-13T07:02:03.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.830 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:02:03.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:02:03.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608815+0000", 2024-09-13T07:02:03.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608819+0000", 2024-09-13T07:02:03.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9740000000000004e-06 2024-09-13T07:02:03.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608829+0000", 2024-09-13T07:02:03.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0383999999999999e-05 2024-09-13T07:02:03.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608838+0000", 2024-09-13T07:02:03.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6719999999999993e-06 2024-09-13T07:02:03.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621751+0000", 2024-09-13T07:02:03.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012913017000000001 2024-09-13T07:02:03.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.621809+0000", 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8505000000000003e-05 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622002+0000", 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000192141 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.834 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:11175 3.b 3:d28d1636:::rbd_data.1194897e7c0a.0000000000000002:c [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T07:02:03.835 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:02:03.835 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.153521114, 2024-09-13T07:02:03.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013984661000000001, 2024-09-13T07:02:03.835 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.835 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:03.835 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.836 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.836 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.836 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 11175 2024-09-13T07:02:03.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.836 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:02:03.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:02:03.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608845+0000", 2024-09-13T07:02:03.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:03.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608849+0000", 2024-09-13T07:02:03.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8449999999999996e-06 2024-09-13T07:02:03.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608859+0000", 2024-09-13T07:02:03.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0335e-05 2024-09-13T07:02:03.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.608868+0000", 2024-09-13T07:02:03.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6070000000000008e-06 2024-09-13T07:02:03.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622108+0000", 2024-09-13T07:02:03.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013240676 2024-09-13T07:02:03.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622223+0000", 2024-09-13T07:02:03.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115277 2024-09-13T07:02:03.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.622830+0000", 2024-09-13T07:02:03.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00060653900000000001 2024-09-13T07:02:03.842 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.842 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.843 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.843 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12393 3.b 3:d3c3e30e:::rbd_data.11945fc173dd.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:02:03.843 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:02:03.844 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.489902240999999, 2024-09-13T07:02:03.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027522214, 2024-09-13T07:02:03.844 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.844 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.844 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.845 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.845 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.845 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12393 2024-09-13T07:02:03.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.845 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:02:03.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:02:03.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272462+0000", 2024-09-13T07:02:03.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:03.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272471+0000", 2024-09-13T07:02:03.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8510000000000005e-06 2024-09-13T07:02:03.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272486+0000", 2024-09-13T07:02:03.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5409000000000001e-05 2024-09-13T07:02:03.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272494+0000", 2024-09-13T07:02:03.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0020000000000006e-06 2024-09-13T07:02:03.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272575+0000", 2024-09-13T07:02:03.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0467000000000001e-05 2024-09-13T07:02:03.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272710+0000", 2024-09-13T07:02:03.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135554 2024-09-13T07:02:03.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:02:03.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.279539+0000", 2024-09-13T07:02:03.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068285439999999998 2024-09-13T07:02:03.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:03.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299872+0000", 2024-09-13T07:02:03.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020332507 2024-09-13T07:02:03.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:03.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299920+0000", 2024-09-13T07:02:03.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8009000000000001e-05 2024-09-13T07:02:03.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299956+0000", 2024-09-13T07:02:03.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6554999999999998e-05 2024-09-13T07:02:03.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299986+0000", 2024-09-13T07:02:03.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0331000000000001e-05 2024-09-13T07:02:03.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.851 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.851 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:48366 3.2 3:4282dbac:::rbd_data.1194fc792839.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:03.851 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:58.185470+0000", 2024-09-13T07:02:03.851 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.5768964890000001, 2024-09-13T07:02:03.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.038773378999999997, 2024-09-13T07:02:03.851 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.852 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.852 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.852 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.852 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.852 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 48366 2024-09-13T07:02:03.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.852 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.185470+0000", 2024-09-13T07:02:03.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.185470+0000", 2024-09-13T07:02:03.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.185467+0000", 2024-09-13T07:02:03.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:03.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192092+0000", 2024-09-13T07:02:03.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066248230000000002 2024-09-13T07:02:03.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192114+0000", 2024-09-13T07:02:03.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1685999999999999e-05 2024-09-13T07:02:03.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192122+0000", 2024-09-13T07:02:03.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4989999999999994e-06 2024-09-13T07:02:03.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192208+0000", 2024-09-13T07:02:03.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5995000000000006e-05 2024-09-13T07:02:03.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192454+0000", 2024-09-13T07:02:03.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024575200000000001 2024-09-13T07:02:03.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:03.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.193274+0000", 2024-09-13T07:02:03.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081968299999999998 2024-09-13T07:02:03.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:03.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.212795+0000", 2024-09-13T07:02:03.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019520991000000001 2024-09-13T07:02:03.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:03.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.224086+0000", 2024-09-13T07:02:03.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011291539 2024-09-13T07:02:03.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.224184+0000", 2024-09-13T07:02:03.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7850000000000007e-05 2024-09-13T07:02:03.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.224243+0000", 2024-09-13T07:02:03.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9200999999999998e-05 2024-09-13T07:02:03.857 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.857 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.857 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:48374 3.19 e58/22 3:9e2350d8:::rbd_data.1194fc792839.000000000000000b:head v 58'169, mlcod=57'167)", 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:58.225355+0000", 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.5370114499999996, 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020769721000000001, 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.225355+0000", 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.225355+0000", 2024-09-13T07:02:03.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.225354+0000", 2024-09-13T07:02:03.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:03.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.230958+0000", 2024-09-13T07:02:03.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0056040789999999997 2024-09-13T07:02:03.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.230984+0000", 2024-09-13T07:02:03.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5440999999999999e-05 2024-09-13T07:02:03.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.230996+0000", 2024-09-13T07:02:03.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2773e-05 2024-09-13T07:02:03.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.231056+0000", 2024-09-13T07:02:03.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9574999999999998e-05 2024-09-13T07:02:03.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.231075+0000", 2024-09-13T07:02:03.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9038999999999998e-05 2024-09-13T07:02:03.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.246077+0000", 2024-09-13T07:02:03.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015001887 2024-09-13T07:02:03.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.246125+0000", 2024-09-13T07:02:03.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7854e-05 2024-09-13T07:02:03.866 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.866 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.866 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.867 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:48372 3.17 e58/51 3:ea07a22c:::rbd_data.1194fc792839.000000000000000a:head v 58'295, mlcod=58'293)", 2024-09-13T07:02:03.867 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:58.233475+0000", 2024-09-13T07:02:03.867 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.5288918740000002, 2024-09-13T07:02:03.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022474327999999998, 2024-09-13T07:02:03.867 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.867 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.867 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.233475+0000", 2024-09-13T07:02:03.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.233475+0000", 2024-09-13T07:02:03.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.233472+0000", 2024-09-13T07:02:03.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:03.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.242565+0000", 2024-09-13T07:02:03.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090921000000000005 2024-09-13T07:02:03.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.242585+0000", 2024-09-13T07:02:03.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0523e-05 2024-09-13T07:02:03.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.242594+0000", 2024-09-13T07:02:03.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9460000000000004e-06 2024-09-13T07:02:03.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.242662+0000", 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7553999999999997e-05 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.242693+0000", 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1027e-05 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.255867+0000", 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013174557999999999 2024-09-13T07:02:03.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.255949+0000", 2024-09-13T07:02:03.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1712999999999995e-05 2024-09-13T07:02:03.871 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.871 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.871 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.871 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:48376 3.c e58/22 3:350dc8a4:::rbd_data.1194fc792839.000000000000000d:head v 58'174, mlcod=57'172)", 2024-09-13T07:02:03.871 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:58.248429+0000", 2024-09-13T07:02:03.871 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.5139372919999996, 2024-09-13T07:02:03.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018893709000000002, 2024-09-13T07:02:03.872 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.872 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.872 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.248429+0000", 2024-09-13T07:02:03.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.248429+0000", 2024-09-13T07:02:03.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.248418+0000", 2024-09-13T07:02:03.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999886 2024-09-13T07:02:03.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.256140+0000", 2024-09-13T07:02:03.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0077221959999999997 2024-09-13T07:02:03.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.256165+0000", 2024-09-13T07:02:03.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5587000000000001e-05 2024-09-13T07:02:03.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.256183+0000", 2024-09-13T07:02:03.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7699000000000001e-05 2024-09-13T07:02:03.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.256244+0000", 2024-09-13T07:02:03.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0973000000000001e-05 2024-09-13T07:02:03.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.256270+0000", 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6152e-05 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.267217+0000", 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010946309 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.267323+0000", 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106351 2024-09-13T07:02:03.875 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.876 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.876 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.876 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:48516 3.7 3:e1968794:::rbd_data.1194fc792839.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:03.876 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:58.541728+0000", 2024-09-13T07:02:03.876 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.2206386440000001, 2024-09-13T07:02:03.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01917818, 2024-09-13T07:02:03.876 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:03.876 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:03.876 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:03.876 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 48516 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541728+0000", 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541728+0000", 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:03.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:03.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541725+0000", 2024-09-13T07:02:03.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999967 2024-09-13T07:02:03.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:03.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541738+0000", 2024-09-13T07:02:03.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.292e-05 2024-09-13T07:02:03.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:03.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541761+0000", 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.321e-05 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541780+0000", 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9363000000000001e-05 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541881+0000", 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010107600000000001 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:03.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.542000+0000", 2024-09-13T07:02:03.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000118776 2024-09-13T07:02:03.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:03.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.547292+0000", 2024-09-13T07:02:03.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052920279999999998 2024-09-13T07:02:03.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:03.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.560746+0000", 2024-09-13T07:02:03.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013453635 2024-09-13T07:02:03.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:03.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.560766+0000", 2024-09-13T07:02:03.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0163e-05 2024-09-13T07:02:03.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:03.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.560848+0000", 2024-09-13T07:02:03.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1731999999999996e-05 2024-09-13T07:02:03.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:03.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:03.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:03.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.560906+0000", 2024-09-13T07:02:03.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8464000000000001e-05 2024-09-13T07:02:03.882 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.882 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.882 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.882 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:03.882 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:03.882 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:03.884 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:03.885 INFO:teuthology.orchestra.run.smithi118.stdout:1633 write 0x19f3aac thru 0x19ffeb6 (0xc40b bytes) 2024-09-13T07:02:03.885 INFO:teuthology.orchestra.run.smithi118.stdout:1634 read 0xd80e919 thru 0xd8129d5 (0x40bd bytes) 2024-09-13T07:02:03.885 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:03.885 INFO:teuthology.orchestra.run.smithi118.stdout:1636 write 0x1ab6a4f thru 0x1ac2a61 (0xc013 bytes) 2024-09-13T07:02:03.885 INFO:teuthology.orchestra.run.smithi118.stdout:1637 read 0x7e254f1 thru 0x7e34f66 (0xfa76 bytes) 2024-09-13T07:02:03.885 INFO:teuthology.orchestra.run.smithi118.stdout:1638 read 0x8d1b43c thru 0x8d2b15c (0xfd21 bytes) 2024-09-13T07:02:03.885 INFO:teuthology.orchestra.run.smithi118.stdout:1640 punch from 0x1561043 to 0x1568190, (0x714d bytes) 2024-09-13T07:02:03.886 INFO:teuthology.orchestra.run.smithi118.stdout:1641 read 0xd3dfef0 thru 0xd3e4dda (0x4eeb bytes) 2024-09-13T07:02:03.886 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:03.886 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:03.886 INFO:teuthology.orchestra.run.smithi118.stdout:1645 writesame 0x8c1431c thru 0x8c1a9e9 data_size 0x1(0x66ce bytes) 2024-09-13T07:02:03.886 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:03.881+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:03.897 INFO:teuthology.orchestra.run.smithi118.stdout:1646 writesame 0x3d224f4 thru 0x3d27ade data_size 0x35(0x55eb bytes) 2024-09-13T07:02:03.928 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:03.928 INFO:teuthology.orchestra.run.smithi118.stdout:1648 writesame 0x7bc1fd7 thru 0x7bced15 data_size 0x1(0xcd3f bytes) 2024-09-13T07:02:03.947 INFO:teuthology.orchestra.run.smithi118.stdout:1649 read 0xc997710 thru 0xc9a1c4b (0xa53c bytes) 2024-09-13T07:02:03.947 INFO:teuthology.orchestra.run.smithi118.stdout:1650 read 0x29d8f63 thru 0x29e36fc (0xa79a bytes) 2024-09-13T07:02:03.948 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:03.948 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:03.948 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:03.948 INFO:teuthology.orchestra.run.smithi118.stdout:1655 write 0x64f4c02 thru 0x6501188 (0xc587 bytes) 2024-09-13T07:02:03.976 INFO:teuthology.orchestra.run.smithi118.stdout:1657 read 0x326a42e thru 0x3276e1c (0xc9ef bytes) 2024-09-13T07:02:03.978 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:03.978 INFO:teuthology.orchestra.run.smithi118.stdout:1659 writesame 0xb62d16f thru 0xb633336 data_size 0x54(0x61c8 bytes) 2024-09-13T07:02:03.984 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:03.981+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:03.991 INFO:teuthology.orchestra.run.smithi118.stdout:1660 punch from 0x38ad73 to 0x399a7a, (0xed07 bytes) 2024-09-13T07:02:04.001 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:04.001 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:04.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.001 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:58656 3.1a 3:596b9532:::rbd_header.1194987e4741:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T07:02:04.001 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:04.001699+0000", 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00028332299999999998, 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031905799999999999, 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 58656 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:04.001699+0000", 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:04.001699+0000", 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:04.001698+0000", 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:04.001705+0000", 2024-09-13T07:02:04.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3690000000000001e-06 2024-09-13T07:02:04.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:04.001724+0000", 2024-09-13T07:02:04.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8669999999999999e-05 2024-09-13T07:02:04.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:04.001733+0000", 2024-09-13T07:02:04.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0960000000000007e-06 2024-09-13T07:02:04.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:04.001789+0000", 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5872000000000003e-05 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:04.001913+0000", 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377199999999999 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-13T07:02:04.005 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:04.006 INFO:teuthology.orchestra.run.smithi118.stdout:1661 write 0xe0baf8b thru 0xe0c5279 (0xa2ef bytes) 2024-09-13T07:02:04.012 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:04.013 INFO:teuthology.orchestra.run.smithi118.stdout:1662 writesame 0x13de55f thru 0x13de7a0 data_size 0x2(0x242 bytes) 2024-09-13T07:02:04.037 INFO:teuthology.orchestra.run.smithi118.stdout:1665 write 0x47191c7 thru 0x471a568 (0x13a2 bytes) 2024-09-13T07:02:04.062 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.062 INFO:teuthology.orchestra.run.smithi118.stdout:1667 write 0xb1e039f thru 0xb1ed398 (0xcffa bytes) 2024-09-13T07:02:04.085 INFO:teuthology.orchestra.run.smithi118.stdout:1668 trunc from 0xe0c527a to 0x79988b6 2024-09-13T07:02:04.085 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:04.082+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:04.118 INFO:teuthology.orchestra.run.smithi118.stdout:1669 trunc from 0x79988b6 to 0xa538588 2024-09-13T07:02:04.119 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:04.120 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:04.129 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:04.141 INFO:teuthology.orchestra.run.smithi118.stdout:1670 write 0xa820125 thru 0xa82a755 (0xa631 bytes) 2024-09-13T07:02:04.148 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.148 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.148 INFO:teuthology.orchestra.run.smithi118.stdout:1675 read 0x89b2d50 thru 0x89bf04f (0xc300 bytes) 2024-09-13T07:02:04.149 INFO:teuthology.orchestra.run.smithi118.stdout:1676 punch from 0x8d0e6fb to 0x8d16f98, (0x889d bytes) 2024-09-13T07:02:04.149 INFO:teuthology.orchestra.run.smithi118.stdout:1677 read 0x124b61b thru 0x1259a4a (0xe430 bytes) 2024-09-13T07:02:04.161 INFO:teuthology.orchestra.run.smithi118.stdout:1679 trunc from 0xa82a756 to 0xe1c1031 2024-09-13T07:02:04.171 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:02:04.171 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:02:04.171 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T07:02:04.171 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T07:02:04.172 INFO:teuthology.orchestra.run.smithi118.stdout:1681 writesame 0xd1f7ba0 thru 0xd1f84d8 data_size 0x1(0x939 bytes) 2024-09-13T07:02:04.179 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:02:04.181 INFO:teuthology.orchestra.run.smithi118.stdout:1682 read 0x4429bf9 thru 0x44367ed (0xcbf5 bytes) 2024-09-13T07:02:04.186 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:04.182+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:04.190 INFO:teuthology.orchestra.run.smithi118.stdout:1684 trunc from 0xe1c1031 to 0x9bceb9d 2024-09-13T07:02:04.208 INFO:teuthology.orchestra.run.smithi118.stdout:1688 write 0x69cee2f thru 0x69d211b (0x32ed bytes) 2024-09-13T07:02:04.221 INFO:teuthology.orchestra.run.smithi118.stdout:1689 read 0x14570ef thru 0x1465054 (0xdf66 bytes) 2024-09-13T07:02:04.222 INFO:teuthology.orchestra.run.smithi118.stdout:1690 read 0x483d252 thru 0x4846680 (0x942f bytes) 2024-09-13T07:02:04.222 INFO:teuthology.orchestra.run.smithi118.stdout:1691 read 0x70c89a5 thru 0x70ceaee (0x614a bytes) 2024-09-13T07:02:04.235 INFO:teuthology.orchestra.run.smithi118.stdout:1692 trunc from 0x9bceb9d to 0x28b863f 2024-09-13T07:02:04.239 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:04.239 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:04.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:04.240 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:04.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.240 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1524 3.14 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:04.240 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:04.240 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.191719511000002, 2024-09-13T07:02:04.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061190880000000003, 2024-09-13T07:02:04.240 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.240 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:04.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.240 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1524 2024-09-13T07:02:04.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.241 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:04.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:04.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044767+0000", 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052670+0000", 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079022450000000004 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052682+0000", 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2336e-05 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052696+0000", 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052755+0000", 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9298e-05 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052947+0000", 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019153900000000001 2024-09-13T07:02:04.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.053694+0000", 2024-09-13T07:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074663099999999999 2024-09-13T07:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.102148+0000", 2024-09-13T07:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048454329999999997 2024-09-13T07:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105816+0000", 2024-09-13T07:02:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036675950000000001 2024-09-13T07:02:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105921+0000", 2024-09-13T07:02:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105874 2024-09-13T07:02:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105959+0000", 2024-09-13T07:02:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7991000000000003e-05 2024-09-13T07:02:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19083 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3817472~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787253701000001, 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049218062999999999, 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.246 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19083 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:04.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:04.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449236+0000", 2024-09-13T07:02:04.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.502e-06 2024-09-13T07:02:04.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449238+0000", 2024-09-13T07:02:04.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.689e-06 2024-09-13T07:02:04.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449241+0000", 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5119999999999998e-06 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498359+0000", 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049117630000000002 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498399+0000", 2024-09-13T07:02:04.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0726000000000003e-05 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498452+0000", 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3220999999999997e-05 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19084 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3465216~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787244991, 2024-09-13T07:02:04.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049328261999999998, 2024-09-13T07:02:04.251 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.251 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.251 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19084 2024-09-13T07:02:04.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.251 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:04.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:04.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:04.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:04.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449244+0000", 2024-09-13T07:02:04.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.454e-06 2024-09-13T07:02:04.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449247+0000", 2024-09-13T07:02:04.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.717e-06 2024-09-13T07:02:04.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449249+0000", 2024-09-13T07:02:04.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.424e-06 2024-09-13T07:02:04.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498477+0000", 2024-09-13T07:02:04.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049227235000000001 2024-09-13T07:02:04.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498518+0000", 2024-09-13T07:02:04.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0846000000000002e-05 2024-09-13T07:02:04.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498571+0000", 2024-09-13T07:02:04.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3810000000000001e-05 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19085 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3842048~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449252+0000", 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787236377999999, 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049435828000000001, 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19085 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449252+0000", 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449252+0000", 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449251+0000", 2024-09-13T07:02:04.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:04.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449253+0000", 2024-09-13T07:02:04.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4389999999999999e-06 2024-09-13T07:02:04.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449256+0000", 2024-09-13T07:02:04.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449258+0000", 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.435e-06 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498594+0000", 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049335639000000001 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498635+0000", 2024-09-13T07:02:04.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1229000000000001e-05 2024-09-13T07:02:04.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498688+0000", 2024-09-13T07:02:04.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2555000000000001e-05 2024-09-13T07:02:04.259 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.259 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.259 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.259 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19086 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3489792~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.259 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:04.259 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787227722000001, 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049540534999999997, 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19086 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449269+0000", 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6459999999999999e-06 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449273+0000", 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9469999999999996e-06 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449276+0000", 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498707+0000", 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049431286999999997 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498748+0000", 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0976999999999997e-05 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498801+0000", 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3090000000000002e-05 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.263 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.264 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19087 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3866624~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.264 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:04.264 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787209891, 2024-09-13T07:02:04.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049644714999999999, 2024-09-13T07:02:04.264 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.264 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.264 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.264 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.264 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.264 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19087 2024-09-13T07:02:04.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449280+0000", 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.655e-06 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449282+0000", 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8600000000000001e-06 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449285+0000", 2024-09-13T07:02:04.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4600000000000002e-06 2024-09-13T07:02:04.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498819+0000", 2024-09-13T07:02:04.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049534340000000003 2024-09-13T07:02:04.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498871+0000", 2024-09-13T07:02:04.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1412000000000002e-05 2024-09-13T07:02:04.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498923+0000", 2024-09-13T07:02:04.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2252e-05 2024-09-13T07:02:04.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.268 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.268 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19088 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3514368~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.268 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:04.268 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787200981, 2024-09-13T07:02:04.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049750612999999999, 2024-09-13T07:02:04.269 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.269 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.269 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19088 2024-09-13T07:02:04.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449288+0000", 2024-09-13T07:02:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.469e-06 2024-09-13T07:02:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449291+0000", 2024-09-13T07:02:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:02:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449294+0000", 2024-09-13T07:02:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5069999999999999e-06 2024-09-13T07:02:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498941+0000", 2024-09-13T07:02:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049647494 2024-09-13T07:02:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498983+0000", 2024-09-13T07:02:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1771999999999999e-05 2024-09-13T07:02:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499038+0000", 2024-09-13T07:02:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4888e-05 2024-09-13T07:02:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19089 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3891200~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787192247, 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049857261, 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19089 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449297+0000", 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4330000000000001e-06 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449301+0000", 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5260000000000002e-06 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449303+0000", 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5110000000000002e-06 2024-09-13T07:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499050+0000", 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049747121999999998 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499091+0000", 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0580000000000001e-05 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499153+0000", 2024-09-13T07:02:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2305999999999996e-05 2024-09-13T07:02:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19090 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3538944~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787182764000001, 2024-09-13T07:02:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049959569000000002, 2024-09-13T07:02:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19090 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449307+0000", 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4279999999999999e-06 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449309+0000", 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.706e-06 2024-09-13T07:02:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449312+0000", 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4930000000000001e-06 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499169+0000", 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049857745000000002 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499210+0000", 2024-09-13T07:02:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0706e-05 2024-09-13T07:02:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499265+0000", 2024-09-13T07:02:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4719000000000002e-05 2024-09-13T07:02:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19091 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3915776~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787174122, 2024-09-13T07:02:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050065959, 2024-09-13T07:02:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19091 2024-09-13T07:02:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449315+0000", 2024-09-13T07:02:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.485e-06 2024-09-13T07:02:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449318+0000", 2024-09-13T07:02:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6740000000000001e-06 2024-09-13T07:02:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449320+0000", 2024-09-13T07:02:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.413e-06 2024-09-13T07:02:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499283+0000", 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049962196 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499323+0000", 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0799e-05 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499380+0000", 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6610999999999997e-05 2024-09-13T07:02:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19092 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3563520~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449323+0000", 2024-09-13T07:02:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787165431, 2024-09-13T07:02:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050176212999999997, 2024-09-13T07:02:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19092 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449323+0000", 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449323+0000", 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449322+0000", 2024-09-13T07:02:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449324+0000", 2024-09-13T07:02:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.424e-06 2024-09-13T07:02:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449327+0000", 2024-09-13T07:02:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-13T07:02:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449329+0000", 2024-09-13T07:02:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4820000000000001e-06 2024-09-13T07:02:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499397+0000", 2024-09-13T07:02:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050068362999999998 2024-09-13T07:02:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499441+0000", 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3668999999999998e-05 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499499+0000", 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7778999999999997e-05 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19093 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3940352~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787156648, 2024-09-13T07:02:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050278704, 2024-09-13T07:02:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19093 2024-09-13T07:02:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449333+0000", 2024-09-13T07:02:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5570000000000001e-06 2024-09-13T07:02:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449335+0000", 2024-09-13T07:02:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-13T07:02:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449338+0000", 2024-09-13T07:02:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.491e-06 2024-09-13T07:02:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499513+0000", 2024-09-13T07:02:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050175143999999998 2024-09-13T07:02:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499554+0000", 2024-09-13T07:02:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0729000000000003e-05 2024-09-13T07:02:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499610+0000", 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6307000000000001e-05 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19094 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3588096~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787147745, 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050385554999999999, 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19094 2024-09-13T07:02:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449342+0000", 2024-09-13T07:02:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.652e-06 2024-09-13T07:02:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449345+0000", 2024-09-13T07:02:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:02:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449348+0000", 2024-09-13T07:02:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5849999999999998e-06 2024-09-13T07:02:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499626+0000", 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050278728000000002 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499669+0000", 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2400000000000001e-05 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499726+0000", 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7191999999999997e-05 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19095 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3964928~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787138034, 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050492171000000002, 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19095 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449351+0000", 2024-09-13T07:02:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.449e-06 2024-09-13T07:02:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449354+0000", 2024-09-13T07:02:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.836e-06 2024-09-13T07:02:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449357+0000", 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5459999999999998e-06 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499744+0000", 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050387339000000003 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499786+0000", 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2293999999999999e-05 2024-09-13T07:02:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499842+0000", 2024-09-13T07:02:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5927000000000002e-05 2024-09-13T07:02:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19096 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3612672~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.787128965000001, 2024-09-13T07:02:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050599058000000002, 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19096 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449360+0000", 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4529999999999999e-06 2024-09-13T07:02:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449363+0000", 2024-09-13T07:02:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6680000000000001e-06 2024-09-13T07:02:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449366+0000", 2024-09-13T07:02:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6359999999999998e-06 2024-09-13T07:02:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499860+0000", 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050494820000000003 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499902+0000", 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1111e-05 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499958+0000", 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6594000000000001e-05 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19110 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 4186112~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.784005615, 2024-09-13T07:02:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049302258000000002, 2024-09-13T07:02:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19110 2024-09-13T07:02:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452487+0000", 2024-09-13T07:02:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9980000000000003e-06 2024-09-13T07:02:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452490+0000", 2024-09-13T07:02:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1410000000000001e-06 2024-09-13T07:02:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452493+0000", 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6010000000000002e-06 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501587+0000", 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049094282000000003 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501680+0000", 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2675000000000003e-05 2024-09-13T07:02:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501785+0000", 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000104993 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19111 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3760128~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.783992953, 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049401467999999997, 2024-09-13T07:02:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19111 2024-09-13T07:02:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452497+0000", 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7099999999999999e-06 2024-09-13T07:02:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452499+0000", 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6929999999999999e-06 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452502+0000", 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5900000000000002e-06 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501801+0000", 2024-09-13T07:02:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049299005 2024-09-13T07:02:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501838+0000", 2024-09-13T07:02:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7282e-05 2024-09-13T07:02:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501897+0000", 2024-09-13T07:02:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8405000000000001e-05 2024-09-13T07:02:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19112 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3784704~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.783983813999999, 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049618640999999998, 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19112 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452505+0000", 2024-09-13T07:02:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.364e-06 2024-09-13T07:02:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452508+0000", 2024-09-13T07:02:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7470000000000001e-06 2024-09-13T07:02:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452511+0000", 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501932+0000", 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049421395999999999 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502019+0000", 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7247e-05 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502123+0000", 2024-09-13T07:02:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103589 2024-09-13T07:02:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19113 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3809280~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.783974789, 2024-09-13T07:02:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049709729000000001, 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19113 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452515+0000", 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659999999999999e-06 2024-09-13T07:02:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452517+0000", 2024-09-13T07:02:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6189999999999998e-06 2024-09-13T07:02:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452520+0000", 2024-09-13T07:02:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7760000000000002e-06 2024-09-13T07:02:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502134+0000", 2024-09-13T07:02:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049614232000000001 2024-09-13T07:02:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502173+0000", 2024-09-13T07:02:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8486999999999999e-05 2024-09-13T07:02:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502223+0000", 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0392999999999997e-05 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:48369 3.f e58/22 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head v 58'78, mlcod=57'75)", 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.996021646, 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.057439072000000001, 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240464+0000", 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245716+0000", 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005252329 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245745+0000", 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8844e-05 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245758+0000", 2024-09-13T07:02:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2242000000000001e-05 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245849+0000", 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1187999999999995e-05 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245898+0000", 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8881000000000002e-05 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297837+0000", 2024-09-13T07:02:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051938947999999999 2024-09-13T07:02:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297906+0000", 2024-09-13T07:02:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8955999999999995e-05 2024-09-13T07:02:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:04.335 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:04.337 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:04.338 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.338 INFO:teuthology.orchestra.run.smithi118.stdout:1694 punch from 0x7ffc04 to 0x80d4b6, (0xd8b2 bytes) 2024-09-13T07:02:04.338 INFO:teuthology.orchestra.run.smithi118.stdout:1695 punch from 0x3ca87c to 0x3d68de, (0xc062 bytes) 2024-09-13T07:02:04.338 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.338 INFO:teuthology.orchestra.run.smithi118.stdout:1697 read 0xf9f5f5 thru 0xfaee57 (0xf863 bytes) 2024-09-13T07:02:04.338 INFO:teuthology.orchestra.run.smithi118.stdout:1698 read 0xed5398 thru 0xee0090 (0xacf9 bytes) 2024-09-13T07:02:04.339 INFO:teuthology.orchestra.run.smithi118.stdout:1699 read 0x149e1b4 thru 0x14a1d86 (0x3bd3 bytes) 2024-09-13T07:02:04.339 INFO:teuthology.orchestra.run.smithi118.stdout:1700 writesame 0xd471cc8 thru 0xd473adc data_size 0x11(0x1e15 bytes) 2024-09-13T07:02:04.339 INFO:teuthology.orchestra.run.smithi118.stdout:1701 trunc from 0xd473add to 0xd5b1b12 2024-09-13T07:02:04.339 INFO:teuthology.orchestra.run.smithi118.stdout:1704 read 0x47fa0 thru 0x57c0e (0xfc6f bytes) 2024-09-13T07:02:04.339 INFO:teuthology.orchestra.run.smithi118.stdout:1705 write 0x2682441 thru 0x268e3a5 (0xbf65 bytes) 2024-09-13T07:02:04.339 INFO:teuthology.orchestra.run.smithi118.stdout:1707 trunc from 0xd5b1b12 to 0xa05e52a 2024-09-13T07:02:04.339 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:04.339 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:04.333+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:04.349 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:04.353 INFO:teuthology.orchestra.run.smithi118.stdout:1708 write 0xd2a286a thru 0xd2a3d53 (0x14ea bytes) 2024-09-13T07:02:04.359 INFO:teuthology.orchestra.run.smithi118.stdout:1709 write 0x29e1e1f thru 0x29f1079 (0xf25b bytes) 2024-09-13T07:02:04.363 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.363 INFO:teuthology.orchestra.run.smithi118.stdout:1711 read 0x5a627e7 thru 0x5a63df0 (0x160a bytes) 2024-09-13T07:02:04.363 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.363 INFO:teuthology.orchestra.run.smithi118.stdout:1713 read 0x165896b thru 0x16678b1 (0xef47 bytes) 2024-09-13T07:02:04.363 INFO:teuthology.orchestra.run.smithi118.stdout:1714 punch from 0xb0769b4 to 0xb083ea4, (0xd4f0 bytes) 2024-09-13T07:02:04.369 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:04.375 INFO:teuthology.orchestra.run.smithi118.stdout:1716 trunc from 0xd2a3d54 to 0x768326f 2024-09-13T07:02:04.392 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.392 INFO:teuthology.orchestra.run.smithi118.stdout:1719 read 0x1624750 thru 0x16325e7 (0xde98 bytes) 2024-09-13T07:02:04.393 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:04.393 INFO:teuthology.orchestra.run.smithi118.stdout:1721 writesame 0x23d8b23 thru 0x23dcf5b data_size 0x7(0x4439 bytes) 2024-09-13T07:02:04.396 INFO:teuthology.orchestra.run.smithi118.stdout:1722 writesame 0x4fa53f2 thru 0x4faa0bb data_size 0x1(0x4cca bytes) 2024-09-13T07:02:04.413 INFO:teuthology.orchestra.run.smithi118.stdout:1724 punch from 0x42fb4f5 to 0x4305b4a, (0xa655 bytes) 2024-09-13T07:02:04.413 INFO:teuthology.orchestra.run.smithi118.stdout:1727 read 0x209b516 thru 0x20a013d (0x4c28 bytes) 2024-09-13T07:02:04.423 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:04.425 INFO:teuthology.orchestra.run.smithi118.stdout:1728 write 0x7ce594d thru 0x7ceeabf (0x9173 bytes) 2024-09-13T07:02:04.435 INFO:teuthology.orchestra.run.smithi118.stdout:1730 read 0x5802c0 thru 0x584fd5 (0x4d16 bytes) 2024-09-13T07:02:04.435 INFO:teuthology.orchestra.run.smithi118.stdout:1732 read 0x49c90f3 thru 0x49cbfbf (0x2ecd bytes) 2024-09-13T07:02:04.435 INFO:teuthology.orchestra.run.smithi118.stdout:1733 write 0x39663e8 thru 0x396c671 (0x628a bytes) 2024-09-13T07:02:04.437 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:04.434+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:04.442 INFO:teuthology.orchestra.run.smithi118.stdout:1734 writesame 0x3f50351 thru 0x3f55d54 data_size 0xe(0x5a04 bytes) 2024-09-13T07:02:04.446 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:04.449 INFO:teuthology.orchestra.run.smithi118.stdout:1735 write 0x52597e0 thru 0x5264607 (0xae28 bytes) 2024-09-13T07:02:04.458 INFO:teuthology.orchestra.run.smithi118.stdout:1736 trunc from 0x7ceeac0 to 0x933355b 2024-09-13T07:02:04.458 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:04.458 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:04.467 INFO:teuthology.orchestra.run.smithi118.stdout:1737 flatten 2024-09-13T07:02:04.469 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:04.481 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:04.481 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:02:04.481 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:04.482 INFO:teuthology.orchestra.run.smithi118.stdout: "epoch": 63, 2024-09-13T07:02:04.482 INFO:teuthology.orchestra.run.smithi118.stdout: "fsid": "25bb30d1-9541-4e40-8659-18d83bb1bd8a", 2024-09-13T07:02:04.482 INFO:teuthology.orchestra.run.smithi118.stdout: "created": "2024-09-13T07:00:49.323883+0000", 2024-09-13T07:02:04.482 INFO:teuthology.orchestra.run.smithi118.stdout: "modified": "2024-09-13T07:02:03.164068+0000", 2024-09-13T07:02:04.482 INFO:teuthology.orchestra.run.smithi118.stdout: "last_up_change": "2024-09-13T07:01:00.146832+0000", 2024-09-13T07:02:04.482 INFO:teuthology.orchestra.run.smithi118.stdout: "last_in_change": "2024-09-13T07:01:46.465676+0000", 2024-09-13T07:02:04.482 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T07:02:04.482 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_num": 5799936, 2024-09-13T07:02:04.482 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_set": [ 2024-09-13T07:02:04.482 INFO:teuthology.orchestra.run.smithi118.stdout: "pglog_hardlimit", 2024-09-13T07:02:04.482 INFO:teuthology.orchestra.run.smithi118.stdout: "purged_snapdirs", 2024-09-13T07:02:04.482 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_deletes", 2024-09-13T07:02:04.482 INFO:teuthology.orchestra.run.smithi118.stdout: "sortbitwise" 2024-09-13T07:02:04.483 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:04.483 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_version": 4, 2024-09-13T07:02:04.483 INFO:teuthology.orchestra.run.smithi118.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T07:02:04.483 INFO:teuthology.orchestra.run.smithi118.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T07:02:04.483 INFO:teuthology.orchestra.run.smithi118.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T07:02:04.483 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_snapshot": "", 2024-09-13T07:02:04.483 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_max": 3, 2024-09-13T07:02:04.483 INFO:teuthology.orchestra.run.smithi118.stdout: "max_osd": 8, 2024-09-13T07:02:04.483 INFO:teuthology.orchestra.run.smithi118.stdout: "require_min_compat_client": "luminous", 2024-09-13T07:02:04.483 INFO:teuthology.orchestra.run.smithi118.stdout: "min_compat_client": "luminous", 2024-09-13T07:02:04.483 INFO:teuthology.orchestra.run.smithi118.stdout: "require_osd_release": "reef", 2024-09-13T07:02:04.483 INFO:teuthology.orchestra.run.smithi118.stdout: "allow_crimson": false, 2024-09-13T07:02:04.483 INFO:teuthology.orchestra.run.smithi118.stdout: "pools": [ 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 1, 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": ".mgr", 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:00.675365+0000", 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 1, 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool", 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:02:04.484 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:02:04.485 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:02:04.485 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:02:04.485 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 1, 2024-09-13T07:02:04.485 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 1, 2024-09-13T07:02:04.485 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 1, 2024-09-13T07:02:04.485 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 1, 2024-09-13T07:02:04.485 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 1, 2024-09-13T07:02:04.485 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:02:04.485 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:02:04.485 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:02:04.485 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:02:04.485 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:02:04.485 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "16", 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 0, 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 0, 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:02:04.486 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:02:04.487 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:02:04.488 INFO:teuthology.orchestra.run.smithi118.stdout: "options": { 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_max": 32, 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_min": 1 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": {} 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 8, 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 8, 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.25, 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 2, 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 2, 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 1, 2024-09-13T07:02:04.489 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 1, 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 1 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 2, 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": "rbd", 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:07.404263+0000", 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 8193, 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:02:04.490 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:02:04.491 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:02:04.491 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:02:04.491 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:02:04.491 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:02:04.491 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:02:04.491 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 8, 2024-09-13T07:02:04.491 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 8, 2024-09-13T07:02:04.491 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 8, 2024-09-13T07:02:04.491 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 8, 2024-09-13T07:02:04.491 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 8, 2024-09-13T07:02:04.491 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:02:04.491 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:02:04.491 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "20", 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 3, 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 20, 2024-09-13T07:02:04.492 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:02:04.493 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:02:04.493 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:02:04.493 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:02:04.493 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:02:04.493 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:02:04.493 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:02:04.493 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:02:04.493 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:02:04.493 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:02:04.493 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:02:04.493 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:02:04.493 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:02:04.493 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:02:04.494 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:02:04.495 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:02:04.495 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:02:04.495 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:02:04.495 INFO:teuthology.orchestra.run.smithi118.stdout: "options": {}, 2024-09-13T07:02:04.495 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:02:04.495 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd": {} 2024-09-13T07:02:04.495 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.495 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:02:04.495 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 2, 2024-09-13T07:02:04.495 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 2, 2024-09-13T07:02:04.495 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.75, 2024-09-13T07:02:04.495 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 1.5, 2024-09-13T07:02:04.496 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 1.5, 2024-09-13T07:02:04.496 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 1, 2024-09-13T07:02:04.496 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 1, 2024-09-13T07:02:04.496 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 1 2024-09-13T07:02:04.496 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.496 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.496 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.496 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 3, 2024-09-13T07:02:04.496 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": "pool_client.0", 2024-09-13T07:02:04.496 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:17.321011+0000", 2024-09-13T07:02:04.496 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 8193, 2024-09-13T07:02:04.496 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:02:04.496 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:02:04.497 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:02:04.497 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:02:04.497 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:02:04.497 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:02:04.497 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:02:04.497 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:02:04.497 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:02:04.497 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:02:04.497 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:02:04.497 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 32, 2024-09-13T07:02:04.497 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 32, 2024-09-13T07:02:04.497 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 32, 2024-09-13T07:02:04.497 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 32, 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 32, 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "63", 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:02:04.498 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 22, 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 63, 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:02:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:02:04.500 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:02:04.501 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:02:04.501 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:02:04.501 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:02:04.501 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:02:04.501 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:02:04.501 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:02:04.501 INFO:teuthology.orchestra.run.smithi118.stdout: "options": {}, 2024-09-13T07:02:04.501 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:02:04.501 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd": {} 2024-09-13T07:02:04.501 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.501 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:02:04.501 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 1.75, 2024-09-13T07:02:04.501 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 1.75, 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.75, 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 1.309999942779541, 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 1.309999942779541, 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 1, 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 1, 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 1 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: "osds": [ 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 0, 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "21736eee-00cd-4eef-9725-eca7776957b4", 2024-09-13T07:02:04.502 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 0, 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0, 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 22, 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.503 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6817", 2024-09-13T07:02:04.504 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:04.504 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.504 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.504 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.504 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6819", 2024-09-13T07:02:04.504 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:04.504 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.504 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.504 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.504 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:04.504 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.504 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.504 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6821", 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6823", 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.505 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6829", 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6831", 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.506 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6825", 2024-09-13T07:02:04.507 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:04.507 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.507 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.507 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.507 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6827", 2024-09-13T07:02:04.507 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:04.507 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.507 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.507 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.507 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6819/3731386737", 2024-09-13T07:02:04.507 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6823/3731386737", 2024-09-13T07:02:04.507 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6831/3731386737", 2024-09-13T07:02:04.507 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6827/3731386737", 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 1, 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "05818db2-841d-406d-be17-8fbdab4fff69", 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:04.508 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 48, 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6816", 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.509 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6818", 2024-09-13T07:02:04.510 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:04.510 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.510 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.510 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.510 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:04.510 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.510 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.510 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.510 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6820", 2024-09-13T07:02:04.510 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:04.510 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.510 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.510 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.511 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6822", 2024-09-13T07:02:04.511 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:04.511 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.511 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.511 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:04.511 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.511 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.511 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.511 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6828", 2024-09-13T07:02:04.511 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:04.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.511 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.512 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.512 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6830", 2024-09-13T07:02:04.512 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:04.512 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.512 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.512 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.512 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:04.512 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.512 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.512 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.512 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6824", 2024-09-13T07:02:04.512 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:04.512 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6826", 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6818/876310814", 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6822/876310814", 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6830/876310814", 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6826/876310814", 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:04.513 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 2, 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "2387761c-42d2-481b-b5b5-b16d3a943b22", 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 0, 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0, 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 17, 2024-09-13T07:02:04.514 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6808", 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6809", 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.515 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6810", 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6811", 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.516 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.517 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:04.517 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.517 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.517 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.517 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6814", 2024-09-13T07:02:04.517 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:04.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.517 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.517 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.517 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6815", 2024-09-13T07:02:04.517 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:04.517 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.517 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6812", 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6813", 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.518 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6809/690188762", 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6811/690188762", 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6815/690188762", 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6813/690188762", 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 3, 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "56500e90-f76b-47c8-a649-893400359c53", 2024-09-13T07:02:04.519 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 62, 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.520 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6800", 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6801", 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.521 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6802", 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6803", 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.522 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6806", 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6807", 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.523 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6804", 2024-09-13T07:02:04.524 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:04.524 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.524 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.524 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.524 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6805", 2024-09-13T07:02:04.524 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:04.524 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.524 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.524 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.524 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6801/2198175481", 2024-09-13T07:02:04.524 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6803/2198175481", 2024-09-13T07:02:04.524 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6807/2198175481", 2024-09-13T07:02:04.524 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6805/2198175481", 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 4, 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "f594a612-aa07-4565-a09e-08368cf397b6", 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:04.525 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 61, 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6810", 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.526 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6813", 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6817", 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6819", 2024-09-13T07:02:04.527 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:04.528 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.528 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.528 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.528 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:04.528 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.528 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.528 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.528 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6827", 2024-09-13T07:02:04.528 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:04.528 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.528 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.528 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.528 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6830", 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6822", 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.529 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6825", 2024-09-13T07:02:04.530 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:04.530 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.530 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.530 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.530 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6813/1835601651", 2024-09-13T07:02:04.530 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6819/1835601651", 2024-09-13T07:02:04.530 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6830/1835601651", 2024-09-13T07:02:04.530 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6825/1835601651", 2024-09-13T07:02:04.530 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:04.530 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:04.530 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:04.530 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.531 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.531 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 5, 2024-09-13T07:02:04.531 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "1013b81f-e6b4-430b-938d-88759d43b881", 2024-09-13T07:02:04.531 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:04.531 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:04.531 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:02:04.531 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:04.531 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:04.531 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:04.531 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:04.531 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 51, 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6800", 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6801", 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:04.532 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6802", 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6803", 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.533 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6806", 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6807", 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.534 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.535 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:04.535 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.535 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.535 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6804", 2024-09-13T07:02:04.535 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:04.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.535 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.535 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6805", 2024-09-13T07:02:04.535 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:04.535 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6801/2853696331", 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6803/2853696331", 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6807/2853696331", 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6805/2853696331", 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 6, 2024-09-13T07:02:04.536 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "367f5f5b-5919-4534-b88c-229f327e4950", 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 48, 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.537 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6808", 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6812", 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.538 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6815", 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6818", 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.539 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6828", 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6831", 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.540 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6821", 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6824", 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6812/131309660", 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6818/131309660", 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6831/131309660", 2024-09-13T07:02:04.541 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6824/131309660", 2024-09-13T07:02:04.542 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:04.542 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:04.542 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:04.542 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.542 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 7, 2024-09-13T07:02:04.542 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "90d9e486-e1ae-4305-80a6-6f483e534643", 2024-09-13T07:02:04.542 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:04.542 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:04.542 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:02:04.542 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:04.542 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:04.543 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:04.543 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:04.543 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 51, 2024-09-13T07:02:04.543 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:04.543 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:04.543 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:04.543 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.543 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.543 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6809", 2024-09-13T07:02:04.543 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:04.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.544 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.544 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6811", 2024-09-13T07:02:04.544 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:04.544 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.544 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.544 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:04.544 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.544 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.544 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6814", 2024-09-13T07:02:04.544 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:04.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6816", 2024-09-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6826", 2024-09-13T07:02:04.545 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6829", 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6820", 2024-09-13T07:02:04.546 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6823", 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6811/652173699", 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6816/652173699", 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6829/652173699", 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6823/652173699", 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:04.547 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_xinfo": [ 2024-09-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 0, 2024-09-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 54166, 2024-09-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.059233+0000", 2024-09-13T07:02:04.548 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 1, 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.083130+0000", 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 2, 2024-09-13T07:02:04.549 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:04.550 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:04.550 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:04.550 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:04.550 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 65536, 2024-09-13T07:02:04.550 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.848896+0000", 2024-09-13T07:02:04.550 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:04.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.550 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 3, 2024-09-13T07:02:04.550 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:04.550 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:04.550 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:04.551 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:04.551 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:04.551 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.847456+0000", 2024-09-13T07:02:04.551 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:04.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.551 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 4, 2024-09-13T07:02:04.551 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:04.551 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:04.551 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:04.551 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:04.551 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:04.551 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.914135+0000", 2024-09-13T07:02:04.552 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:04.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.552 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 5, 2024-09-13T07:02:04.552 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:04.552 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:04.552 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:04.552 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:04.552 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:04.552 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.834284+0000", 2024-09-13T07:02:04.552 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:04.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 6, 2024-09-13T07:02:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.003715+0000", 2024-09-13T07:02:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 7, 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.033514+0000", 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap": [ 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: "pgid": "3.14", 2024-09-13T07:02:04.554 INFO:teuthology.orchestra.run.smithi118.stdout: "osds": [ 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: 3, 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: 5 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap_items": [], 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap_primaries": [], 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_temp": [], 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_temp": [], 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: "blocklist": {}, 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: "range_blocklist": {}, 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profiles": { 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: "default": { 2024-09-13T07:02:04.555 INFO:teuthology.orchestra.run.smithi118.stdout: "crush-failure-domain": "osd", 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "k": "2", 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "m": "1", 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "plugin": "jerasure", 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "technique": "reed_sol_van" 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps_queue": [], 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "new_removed_snaps": [], 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "new_purged_snaps": [], 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_node_flags": {}, 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "device_class_flags": {}, 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode": { 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode_enabled": false, 2024-09-13T07:02:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_bucket_count": 0, 2024-09-13T07:02:04.557 INFO:teuthology.orchestra.run.smithi118.stdout: "degraded_stretch_mode": 0, 2024-09-13T07:02:04.557 INFO:teuthology.orchestra.run.smithi118.stdout: "recovering_stretch_mode": 0, 2024-09-13T07:02:04.557 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode_bucket": 0 2024-09-13T07:02:04.557 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:04.557 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:04.559 INFO:tasks.thrashosds.thrasher:j is {'epoch': 63, 'fsid': '25bb30d1-9541-4e40-8659-18d83bb1bd8a', 'created': '2024-09-13T07:00:49.323883+0000', 'modified': '2024-09-13T07:02:03.164068+0000', 'last_up_change': '2024-09-13T07:01:00.146832+0000', 'last_in_change': '2024-09-13T07:01:46.465676+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T07:01:00.675365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T07:01:07.404263+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T07:01:17.321011+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 22, 'snap_epoch': 63, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 0.75, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '21736eee-00cd-4eef-9725-eca7776957b4', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6817', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6819', 'nonce': 3731386737}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6821', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6823', 'nonce': 3731386737}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6829', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6831', 'nonce': 3731386737}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6825', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6827', 'nonce': 3731386737}]}, 'public_addr': '172.21.15.118:6819/3731386737', 'cluster_addr': '172.21.15.118:6823/3731386737', 'heartbeat_back_addr': '172.21.15.118:6831/3731386737', 'heartbeat_front_addr': '172.21.15.118:6827/3731386737', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '05818db2-841d-406d-be17-8fbdab4fff69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6816', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6818', 'nonce': 876310814}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6820', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6822', 'nonce': 876310814}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6828', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6830', 'nonce': 876310814}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6824', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6826', 'nonce': 876310814}]}, 'public_addr': '172.21.15.118:6818/876310814', 'cluster_addr': '172.21.15.118:6822/876310814', 'heartbeat_back_addr': '172.21.15.118:6830/876310814', 'heartbeat_front_addr': '172.21.15.118:6826/876310814', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2387761c-42d2-481b-b5b5-b16d3a943b22', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6808', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6809', 'nonce': 690188762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6810', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6811', 'nonce': 690188762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6814', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6815', 'nonce': 690188762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6812', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6813', 'nonce': 690188762}]}, 'public_addr': '172.21.15.118:6809/690188762', 'cluster_addr': '172.21.15.118:6811/690188762', 'heartbeat_back_addr': '172.21.15.118:6815/690188762', 'heartbeat_front_addr': '172.21.15.118:6813/690188762', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '56500e90-f76b-47c8-a649-893400359c53', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6800', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6801', 'nonce': 2198175481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6802', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6803', 'nonce': 2198175481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6806', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6807', 'nonce': 2198175481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6804', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6805', 'nonce': 2198175481}]}, 'public_addr': '172.21.15.118:6801/2198175481', 'cluster_addr': '172.21.15.118:6803/2198175481', 'heartbeat_back_addr': '172.21.15.118:6807/2198175481', 'heartbeat_front_addr': '172.21.15.118:6805/2198175481', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f594a612-aa07-4565-a09e-08368cf397b6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 1835601651}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6817', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 1835601651}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6827', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6830', 'nonce': 1835601651}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 1835601651}]}, 'public_addr': '172.21.15.164:6813/1835601651', 'cluster_addr': '172.21.15.164:6819/1835601651', 'heartbeat_back_addr': '172.21.15.164:6830/1835601651', 'heartbeat_front_addr': '172.21.15.164:6825/1835601651', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1013b81f-e6b4-430b-938d-88759d43b881', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 2853696331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 2853696331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 2853696331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 2853696331}]}, 'public_addr': '172.21.15.164:6801/2853696331', 'cluster_addr': '172.21.15.164:6803/2853696331', 'heartbeat_back_addr': '172.21.15.164:6807/2853696331', 'heartbeat_front_addr': '172.21.15.164:6805/2853696331', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '367f5f5b-5919-4534-b88c-229f327e4950', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6812', 'nonce': 131309660}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6815', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6818', 'nonce': 131309660}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 131309660}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6821', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6824', 'nonce': 131309660}]}, 'public_addr': '172.21.15.164:6812/131309660', 'cluster_addr': '172.21.15.164:6818/131309660', 'heartbeat_back_addr': '172.21.15.164:6831/131309660', 'heartbeat_front_addr': '172.21.15.164:6824/131309660', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '90d9e486-e1ae-4305-80a6-6f483e534643', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6809', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 652173699}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6816', 'nonce': 652173699}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 652173699}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 652173699}]}, 'public_addr': '172.21.15.164:6811/652173699', 'cluster_addr': '172.21.15.164:6816/652173699', 'heartbeat_back_addr': '172.21.15.164:6829/652173699', 'heartbeat_front_addr': '172.21.15.164:6823/652173699', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 54166, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.059233+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.083130+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.848896+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.847456+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.914135+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.834284+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.003715+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.033514+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.14', 'osds': [3, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T07:02:04.559 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.14 2024-09-13T07:02:04.559 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.14 2024-09-13T07:02:04.559 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:04.560 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:04.560 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:04.560 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:04.560 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:04.560 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:04.561 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:04.561 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:04.561 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:04.561 INFO:teuthology.orchestra.run.smithi118.stdout:1739 write 0x9c8e622 thru 0x9c8f89d (0x127c bytes) 2024-09-13T07:02:04.561 INFO:teuthology.orchestra.run.smithi118.stdout:1740 read 0x27b0349 thru 0x27b413e (0x3df6 bytes) 2024-09-13T07:02:04.562 INFO:teuthology.orchestra.run.smithi118.stdout:1741 write 0x9230ee3 thru 0x923f24d (0xe36b bytes) 2024-09-13T07:02:04.562 INFO:teuthology.orchestra.run.smithi118.stdout:1742 writesame 0x40c530c thru 0x40c7b1b data_size 0x8(0x2810 bytes) 2024-09-13T07:02:04.562 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.562 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:04.557+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:04.569 INFO:teuthology.orchestra.run.smithi118.stdout:1745 writesame 0xaacefd3 thru 0xaad9e3c data_size 0x32(0xae6a bytes) 2024-09-13T07:02:04.569 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.584 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:04.584 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:04.587 INFO:teuthology.orchestra.run.smithi118.stdout:1747 writesame 0xe220892 thru 0xe227376 data_size 0xd(0x6ae5 bytes) 2024-09-13T07:02:04.587 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.587 INFO:teuthology.orchestra.run.smithi118.stdout:1750 write 0x2370bd1 thru 0x23734fc (0x292c bytes) 2024-09-13T07:02:04.595 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:04.607 INFO:teuthology.orchestra.run.smithi118.stdout:1751 trunc from 0xe227377 to 0x636c642 2024-09-13T07:02:04.625 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.625 INFO:teuthology.orchestra.run.smithi118.stdout:1754 read 0xd2fa20 thru 0xd35640 (0x5c21 bytes) 2024-09-13T07:02:04.638 INFO:teuthology.orchestra.run.smithi118.stdout:1755 trunc from 0x636c642 to 0xbedda6d 2024-09-13T07:02:04.648 INFO:teuthology.orchestra.run.smithi118.stdout:1757 read 0x5346ff5 thru 0x534b978 (0x4984 bytes) 2024-09-13T07:02:04.649 INFO:teuthology.orchestra.run.smithi118.stdout:1758 write 0x7d05d90 thru 0x7d14525 (0xe796 bytes) 2024-09-13T07:02:04.658 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:04.654+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:04.661 INFO:teuthology.orchestra.run.smithi118.stdout:1759 write 0x63aec87 thru 0x63b77c1 (0x8b3b bytes) 2024-09-13T07:02:04.667 INFO:teuthology.orchestra.run.smithi118.stdout:1760 write 0x1b53217 thru 0x1b61f62 (0xed4c bytes) 2024-09-13T07:02:04.670 INFO:teuthology.orchestra.run.smithi118.stdout:1761 write 0x6451d52 thru 0x645b860 (0x9b0f bytes) 2024-09-13T07:02:04.675 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.675 INFO:teuthology.orchestra.run.smithi118.stdout:1763 writesame 0x4fb5782 thru 0x4fbde29 data_size 0x8(0x86a8 bytes) 2024-09-13T07:02:04.681 INFO:teuthology.orchestra.run.smithi118.stdout:1765 write 0x831e5cf thru 0x8320a84 (0x24b6 bytes) 2024-09-13T07:02:04.687 INFO:teuthology.orchestra.run.smithi118.stdout:1766 read 0x795b984 thru 0x796152a (0x5ba7 bytes) 2024-09-13T07:02:04.688 INFO:teuthology.orchestra.run.smithi118.stdout:1767 read 0x2a55d4e thru 0x2a59932 (0x3be5 bytes) 2024-09-13T07:02:04.688 INFO:teuthology.orchestra.run.smithi118.stdout:1768 writesame 0xa68cc32 thru 0xa6919de data_size 0x29(0x4dad bytes) 2024-09-13T07:02:04.693 INFO:teuthology.orchestra.run.smithi118.stdout:1769 write 0x12b47bb thru 0x12bb992 (0x71d8 bytes) 2024-09-13T07:02:04.700 INFO:teuthology.orchestra.run.smithi118.stdout:1770 read 0x3739471 thru 0x373d0b0 (0x3c40 bytes) 2024-09-13T07:02:04.700 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.700 INFO:teuthology.orchestra.run.smithi118.stdout:1772 read 0x5b64ff8 thru 0x5b6d3a2 (0x83ab bytes) 2024-09-13T07:02:04.705 INFO:teuthology.orchestra.run.smithi118.stdout:1774 trunc from 0xbedda6d to 0x944e20f 2024-09-13T07:02:04.711 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:04.711 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:04.717 INFO:teuthology.orchestra.run.smithi118.stdout:1775 punch from 0x6318ea3 to 0x63205a1, (0x76fe bytes) 2024-09-13T07:02:04.717 INFO:teuthology.orchestra.run.smithi118.stdout:1776 read 0x565cc30 thru 0x566932b (0xc6fc bytes) 2024-09-13T07:02:04.717 INFO:teuthology.orchestra.run.smithi118.stdout:1777 read 0x2088fb6 thru 0x208a6d7 (0x1722 bytes) 2024-09-13T07:02:04.722 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:04.737 INFO:teuthology.orchestra.run.smithi118.stdout:1779 write 0xe594f89 thru 0xe597eab (0x2f23 bytes) 2024-09-13T07:02:04.743 INFO:teuthology.orchestra.run.smithi118.stdout:1780 write 0x7e5e9c8 thru 0x7e61189 (0x27c2 bytes) 2024-09-13T07:02:04.755 INFO:teuthology.orchestra.run.smithi118.stdout:1781 trunc from 0xe597eac to 0x9102949 2024-09-13T07:02:04.760 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:04.757+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:04.778 INFO:teuthology.orchestra.run.smithi118.stdout:1782 read 0x615dc19 thru 0x6169c75 (0xc05d bytes) 2024-09-13T07:02:04.779 INFO:teuthology.orchestra.run.smithi118.stdout:1783 punch from 0x1e7ec89 to 0x1e85545, (0x68bc bytes) 2024-09-13T07:02:04.779 INFO:teuthology.orchestra.run.smithi118.stdout:1785 read 0x27322cd thru 0x273c568 (0xa29c bytes) 2024-09-13T07:02:04.779 INFO:teuthology.orchestra.run.smithi118.stdout:1786 read 0x3a5f2e thru 0x3b0a3f (0xab12 bytes) 2024-09-13T07:02:04.785 INFO:teuthology.orchestra.run.smithi118.stdout:1789 trunc from 0x9102949 to 0xac56c70 2024-09-13T07:02:04.796 INFO:teuthology.orchestra.run.smithi118.stdout:1790 write 0x9c53e68 thru 0x9c5528a (0x1423 bytes) 2024-09-13T07:02:04.802 INFO:teuthology.orchestra.run.smithi118.stdout:1791 write 0x3836c76 thru 0x383bd26 (0x50b1 bytes) 2024-09-13T07:02:04.808 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:04.808 INFO:teuthology.orchestra.run.smithi118.stdout:1795 read 0x78603c thru 0x78e221 (0x81e6 bytes) 2024-09-13T07:02:04.809 INFO:teuthology.orchestra.run.smithi118.stdout:1796 clone 20 order 18 su 1024 sc 4 2024-09-13T07:02:04.861 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:04.858+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:04.874 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:04.874 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:04.885 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:04.962 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:04.961+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:04.990 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:04.990 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:05.000 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:05.064 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:05.062+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:05.107 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:05.107 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:05.117 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:05.164 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:05.162+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:05.190 INFO:teuthology.orchestra.run.smithi118.stderr:clear 3.14 pg_upmap mapping 2024-09-13T07:02:05.192 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:02:05.265 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:05.262+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:05.278 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:05.278 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:05.288 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:05.366 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:05.362+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:05.441 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-13T07:02:05.441 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:05.442 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:05.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:05.466 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:05.465+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:05.560 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:05.560 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:05.568 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:05.565+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:05.570 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:05.668 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:05.666+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:05.676 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:05.677 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:05.687 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:05.769 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:05.766+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:05.790 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:05.791 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:05.800 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:05.871 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:05.870+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:05.904 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:05.904 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:05.913 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:05.972 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:05.969+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:06.022 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:06.022 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:06.031 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:06.077 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:06.077+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:06.174 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:06.170+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:06.186 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:06.186 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:06.196 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:02:06.198 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:06.275 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:06.274+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:06.297 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:06.297 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:06.307 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:06.376 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:06.374+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:06.413 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:06.413 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:06.422 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:06.477 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:06.477+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:06.532 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:06.533 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:06.546 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:06.562 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:02:06.609 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:06.609+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:06.649 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:06.650 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:06.659 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:06.679 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:06.674+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:06.701 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:06.722 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:06.741 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:06.756 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:06.756 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:06.762 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:06.766 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:06.780 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:06.777+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:06.785 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:06.807 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:06.835 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:06.856 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:06.858 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:06.858 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:06.858 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:06.859 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:06.859 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:06.859 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:06.859 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:06.859 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:06.866 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:06.867 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:06.867 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:06.867 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:06.867 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:06.877 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:06.921 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:06.921+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:06.982 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:06.981+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:07.030 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:07.030 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:07.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:07.030 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:07.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.030 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1155 3.7 3:e095a6bd:::rbd_data.11943a018f34.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:07.030 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:02:07.031 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.196189398999998, 2024-09-13T07:02:07.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022798902999999999, 2024-09-13T07:02:07.031 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.031 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:07.031 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.031 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.031 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.031 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1155 2024-09-13T07:02:07.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.031 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:02:07.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832764+0000", 2024-09-13T07:02:07.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832762+0000", 2024-09-13T07:02:07.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:07.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832776+0000", 2024-09-13T07:02:07.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3828e-05 2024-09-13T07:02:07.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832806+0000", 2024-09-13T07:02:07.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0159e-05 2024-09-13T07:02:07.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832820+0000", 2024-09-13T07:02:07.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.384e-05 2024-09-13T07:02:07.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.832904+0000", 2024-09-13T07:02:07.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4506999999999996e-05 2024-09-13T07:02:07.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.833121+0000", 2024-09-13T07:02:07.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000217223 2024-09-13T07:02:07.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:07.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.839072+0000", 2024-09-13T07:02:07.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0059509849999999998 2024-09-13T07:02:07.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:07.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855489+0000", 2024-09-13T07:02:07.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016416907000000001 2024-09-13T07:02:07.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:07.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855503+0000", 2024-09-13T07:02:07.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3658999999999999e-05 2024-09-13T07:02:07.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:07.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855535+0000", 2024-09-13T07:02:07.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1696000000000003e-05 2024-09-13T07:02:07.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.855563+0000", 2024-09-13T07:02:07.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8578999999999999e-05 2024-09-13T07:02:07.038 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.038 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.038 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.038 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1200 3.7 3:e095a6bd:::rbd_data.11943a018f34.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:07.039 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:02:07.039 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.049850448999997, 2024-09-13T07:02:07.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018736263, 2024-09-13T07:02:07.039 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.039 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:07.039 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.039 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.039 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.039 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1200 2024-09-13T07:02:07.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.039 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:02:07.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979103+0000", 2024-09-13T07:02:07.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979100+0000", 2024-09-13T07:02:07.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:07.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979115+0000", 2024-09-13T07:02:07.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4823e-05 2024-09-13T07:02:07.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979149+0000", 2024-09-13T07:02:07.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3772999999999997e-05 2024-09-13T07:02:07.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979163+0000", 2024-09-13T07:02:07.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3845999999999999e-05 2024-09-13T07:02:07.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979333+0000", 2024-09-13T07:02:07.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000169622 2024-09-13T07:02:07.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.979603+0000", 2024-09-13T07:02:07.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00027078999999999997 2024-09-13T07:02:07.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:07.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.986017+0000", 2024-09-13T07:02:07.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0064139590000000003 2024-09-13T07:02:07.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:07.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997476+0000", 2024-09-13T07:02:07.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011458582 2024-09-13T07:02:07.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:07.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997588+0000", 2024-09-13T07:02:07.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011161 2024-09-13T07:02:07.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:07.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997732+0000", 2024-09-13T07:02:07.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000144 2024-09-13T07:02:07.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.997839+0000", 2024-09-13T07:02:07.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000107892 2024-09-13T07:02:07.046 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.047 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.047 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.047 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1531 3.c e27/22 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head v 27'9, mlcod=27'7)", 2024-09-13T07:02:07.047 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:02:07.047 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 40.954422291999997, 2024-09-13T07:02:07.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039127038000000003, 2024-09-13T07:02:07.047 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.047 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:07.047 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074530+0000", 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:07.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099434+0000", 2024-09-13T07:02:07.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.024903926999999999 2024-09-13T07:02:07.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099459+0000", 2024-09-13T07:02:07.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5384999999999998e-05 2024-09-13T07:02:07.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099469+0000", 2024-09-13T07:02:07.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.024e-05 2024-09-13T07:02:07.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099566+0000", 2024-09-13T07:02:07.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6533999999999994e-05 2024-09-13T07:02:07.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099600+0000", 2024-09-13T07:02:07.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4007000000000002e-05 2024-09-13T07:02:07.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113477+0000", 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01387741 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113658+0000", 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000180953 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.051 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4023 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:07.052 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:02:07.052 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.374500961999999, 2024-09-13T07:02:07.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015452399, 2024-09-13T07:02:07.052 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.052 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:07.052 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.052 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.052 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.052 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4023 2024-09-13T07:02:07.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.052 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:02:07.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654453+0000", 2024-09-13T07:02:07.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654452+0000", 2024-09-13T07:02:07.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:07.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654456+0000", 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7570000000000002e-06 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654465+0000", 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7140000000000005e-06 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654486+0000", 2024-09-13T07:02:07.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0290000000000001e-05 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669455+0000", 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014969651 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669722+0000", 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026677899999999997 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669905+0000", 2024-09-13T07:02:07.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018285999999999999 2024-09-13T07:02:07.056 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.056 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.056 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.056 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4024 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:07.056 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:02:07.056 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.374460161999998, 2024-09-13T07:02:07.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015682979, 2024-09-13T07:02:07.056 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.056 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:07.056 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.057 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4024 2024-09-13T07:02:07.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.057 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:02:07.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:02:07.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654493+0000", 2024-09-13T07:02:07.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:07.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654497+0000", 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4800000000000003e-06 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654507+0000", 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8749999999999995e-06 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654515+0000", 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0350000000000001e-06 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.669936+0000", 2024-09-13T07:02:07.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015420916 2024-09-13T07:02:07.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670030+0000", 2024-09-13T07:02:07.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3712e-05 2024-09-13T07:02:07.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670176+0000", 2024-09-13T07:02:07.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000146613 2024-09-13T07:02:07.061 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.061 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.061 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.061 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4025 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:07.061 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:02:07.061 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.374431154, 2024-09-13T07:02:07.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016683559000000001, 2024-09-13T07:02:07.061 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.061 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:07.061 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.061 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4025 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654522+0000", 2024-09-13T07:02:07.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:07.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654526+0000", 2024-09-13T07:02:07.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8070000000000001e-06 2024-09-13T07:02:07.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654535+0000", 2024-09-13T07:02:07.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4199999999999996e-06 2024-09-13T07:02:07.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654543+0000", 2024-09-13T07:02:07.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:02:07.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670307+0000", 2024-09-13T07:02:07.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015763482999999998 2024-09-13T07:02:07.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.670399+0000", 2024-09-13T07:02:07.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2068e-05 2024-09-13T07:02:07.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671206+0000", 2024-09-13T07:02:07.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000807352 2024-09-13T07:02:07.065 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.065 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.065 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.066 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4026 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 13238272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:07.066 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:02:07.066 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.374403424999997, 2024-09-13T07:02:07.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016876344000000001, 2024-09-13T07:02:07.066 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.066 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:07.066 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.066 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.067 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.067 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4026 2024-09-13T07:02:07.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.067 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:02:07.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:02:07.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654550+0000", 2024-09-13T07:02:07.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:07.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654554+0000", 2024-09-13T07:02:07.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1119999999999998e-06 2024-09-13T07:02:07.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654564+0000", 2024-09-13T07:02:07.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0327e-05 2024-09-13T07:02:07.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654571+0000", 2024-09-13T07:02:07.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3440000000000003e-06 2024-09-13T07:02:07.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671239+0000", 2024-09-13T07:02:07.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016668039999999999 2024-09-13T07:02:07.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671327+0000", 2024-09-13T07:02:07.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7818999999999994e-05 2024-09-13T07:02:07.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.671427+0000", 2024-09-13T07:02:07.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9324e-05 2024-09-13T07:02:07.071 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.071 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.071 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.072 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4027 3.1e 3:79aaab1c:::rbd_data.11945d91e99c.0000000000000009:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:07.072 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:02:07.072 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.374374662999998, 2024-09-13T07:02:07.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017835597000000002, 2024-09-13T07:02:07.072 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.072 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:07.072 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.072 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.072 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.073 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4027 2024-09-13T07:02:07.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.073 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:02:07.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654579+0000", 2024-09-13T07:02:07.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654578+0000", 2024-09-13T07:02:07.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:07.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654590+0000", 2024-09-13T07:02:07.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1595e-05 2024-09-13T07:02:07.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654601+0000", 2024-09-13T07:02:07.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0827e-05 2024-09-13T07:02:07.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.654609+0000", 2024-09-13T07:02:07.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4919999999999993e-06 2024-09-13T07:02:07.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672157+0000", 2024-09-13T07:02:07.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.017547770000000001 2024-09-13T07:02:07.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672234+0000", 2024-09-13T07:02:07.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7509999999999995e-05 2024-09-13T07:02:07.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.672415+0000", 2024-09-13T07:02:07.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00018005500000000001 2024-09-13T07:02:07.077 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.077 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.077 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.078 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12393 3.b 3:d3c3e30e:::rbd_data.11945fc173dd.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:02:07.078 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:02:07.078 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.756489359, 2024-09-13T07:02:07.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027522214, 2024-09-13T07:02:07.078 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.078 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:07.078 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.078 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.078 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.079 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12393 2024-09-13T07:02:07.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.079 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:02:07.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:02:07.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272462+0000", 2024-09-13T07:02:07.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:07.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272471+0000", 2024-09-13T07:02:07.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8510000000000005e-06 2024-09-13T07:02:07.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272486+0000", 2024-09-13T07:02:07.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5409000000000001e-05 2024-09-13T07:02:07.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272494+0000", 2024-09-13T07:02:07.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0020000000000006e-06 2024-09-13T07:02:07.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272575+0000", 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0467000000000001e-05 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272710+0000", 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135554 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.279539+0000", 2024-09-13T07:02:07.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068285439999999998 2024-09-13T07:02:07.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:07.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299872+0000", 2024-09-13T07:02:07.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020332507 2024-09-13T07:02:07.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:07.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299920+0000", 2024-09-13T07:02:07.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8009000000000001e-05 2024-09-13T07:02:07.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:07.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299956+0000", 2024-09-13T07:02:07.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6554999999999998e-05 2024-09-13T07:02:07.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299986+0000", 2024-09-13T07:02:07.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0331000000000001e-05 2024-09-13T07:02:07.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.084 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.084 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:48366 3.2 3:4282dbac:::rbd_data.1194fc792839.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:58.185470+0000", 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.8434836069999996, 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.038773378999999997, 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 48366 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.185470+0000", 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.185470+0000", 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.185467+0000", 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192092+0000", 2024-09-13T07:02:07.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066248230000000002 2024-09-13T07:02:07.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192114+0000", 2024-09-13T07:02:07.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1685999999999999e-05 2024-09-13T07:02:07.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192122+0000", 2024-09-13T07:02:07.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4989999999999994e-06 2024-09-13T07:02:07.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192208+0000", 2024-09-13T07:02:07.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5995000000000006e-05 2024-09-13T07:02:07.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192454+0000", 2024-09-13T07:02:07.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024575200000000001 2024-09-13T07:02:07.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:07.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.193274+0000", 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081968299999999998 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.212795+0000", 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019520991000000001 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.224086+0000", 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011291539 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.224184+0000", 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7850000000000007e-05 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.224243+0000", 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9200999999999998e-05 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:48374 3.19 e58/22 3:9e2350d8:::rbd_data.1194fc792839.000000000000000b:head v 58'169, mlcod=57'167)", 2024-09-13T07:02:07.090 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:58.225355+0000", 2024-09-13T07:02:07.091 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.803598568, 2024-09-13T07:02:07.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020769721000000001, 2024-09-13T07:02:07.091 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.091 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:07.091 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.225355+0000", 2024-09-13T07:02:07.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.225355+0000", 2024-09-13T07:02:07.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.225354+0000", 2024-09-13T07:02:07.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:07.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.230958+0000", 2024-09-13T07:02:07.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0056040789999999997 2024-09-13T07:02:07.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.230984+0000", 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5440999999999999e-05 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.230996+0000", 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2773e-05 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.231056+0000", 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9574999999999998e-05 2024-09-13T07:02:07.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.231075+0000", 2024-09-13T07:02:07.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9038999999999998e-05 2024-09-13T07:02:07.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:07.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.246077+0000", 2024-09-13T07:02:07.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015001887 2024-09-13T07:02:07.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.246125+0000", 2024-09-13T07:02:07.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7854e-05 2024-09-13T07:02:07.095 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.095 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.095 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.095 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:48372 3.17 e58/51 3:ea07a22c:::rbd_data.1194fc792839.000000000000000a:head v 58'295, mlcod=58'293)", 2024-09-13T07:02:07.096 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:58.233475+0000", 2024-09-13T07:02:07.096 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.7954789919999996, 2024-09-13T07:02:07.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022474327999999998, 2024-09-13T07:02:07.096 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.096 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:07.096 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.233475+0000", 2024-09-13T07:02:07.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.233475+0000", 2024-09-13T07:02:07.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.233472+0000", 2024-09-13T07:02:07.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:07.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.242565+0000", 2024-09-13T07:02:07.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090921000000000005 2024-09-13T07:02:07.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.242585+0000", 2024-09-13T07:02:07.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0523e-05 2024-09-13T07:02:07.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.242594+0000", 2024-09-13T07:02:07.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9460000000000004e-06 2024-09-13T07:02:07.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.242662+0000", 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7553999999999997e-05 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.242693+0000", 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1027e-05 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.255867+0000", 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013174557999999999 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.255949+0000", 2024-09-13T07:02:07.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1712999999999995e-05 2024-09-13T07:02:07.100 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.100 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.100 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.100 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:48376 3.c e58/22 3:350dc8a4:::rbd_data.1194fc792839.000000000000000d:head v 58'174, mlcod=57'172)", 2024-09-13T07:02:07.100 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:58.248429+0000", 2024-09-13T07:02:07.100 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.7805244099999999, 2024-09-13T07:02:07.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018893709000000002, 2024-09-13T07:02:07.101 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.101 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:07.101 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.248429+0000", 2024-09-13T07:02:07.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.248429+0000", 2024-09-13T07:02:07.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.248418+0000", 2024-09-13T07:02:07.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999886 2024-09-13T07:02:07.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.256140+0000", 2024-09-13T07:02:07.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0077221959999999997 2024-09-13T07:02:07.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.256165+0000", 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5587000000000001e-05 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.256183+0000", 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7699000000000001e-05 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.256244+0000", 2024-09-13T07:02:07.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0973000000000001e-05 2024-09-13T07:02:07.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.256270+0000", 2024-09-13T07:02:07.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6152e-05 2024-09-13T07:02:07.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:07.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.267217+0000", 2024-09-13T07:02:07.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010946309 2024-09-13T07:02:07.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.267323+0000", 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106351 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:48516 3.7 3:e1968794:::rbd_data.1194fc792839.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:58.541728+0000", 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.4872257619999996, 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01917818, 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:07.105 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.106 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.106 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.106 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 48516 2024-09-13T07:02:07.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.106 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541728+0000", 2024-09-13T07:02:07.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541728+0000", 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541725+0000", 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999967 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541738+0000", 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.292e-05 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541761+0000", 2024-09-13T07:02:07.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.321e-05 2024-09-13T07:02:07.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541780+0000", 2024-09-13T07:02:07.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9363000000000001e-05 2024-09-13T07:02:07.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.541881+0000", 2024-09-13T07:02:07.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010107600000000001 2024-09-13T07:02:07.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.542000+0000", 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000118776 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.547292+0000", 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0052920279999999998 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.560746+0000", 2024-09-13T07:02:07.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013453635 2024-09-13T07:02:07.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:07.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.560766+0000", 2024-09-13T07:02:07.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0163e-05 2024-09-13T07:02:07.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:07.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.560848+0000", 2024-09-13T07:02:07.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1731999999999996e-05 2024-09-13T07:02:07.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.560906+0000", 2024-09-13T07:02:07.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8464000000000001e-05 2024-09-13T07:02:07.111 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.111 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.111 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.112 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:58568 3.2 3:441e3a7e:::rbd_data.119475582a6.0000000000000018:16 [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T07:02:07.112 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:03.954024+0000", 2024-09-13T07:02:07.112 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.074929493, 2024-09-13T07:02:07.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015195106, 2024-09-13T07:02:07.112 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.112 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:07.112 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.112 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.112 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.113 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 58568 2024-09-13T07:02:07.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.113 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954024+0000", 2024-09-13T07:02:07.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954024+0000", 2024-09-13T07:02:07.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954024+0000", 2024-09-13T07:02:07.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:07.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954028+0000", 2024-09-13T07:02:07.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0199999999999996e-06 2024-09-13T07:02:07.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954042+0000", 2024-09-13T07:02:07.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4208e-05 2024-09-13T07:02:07.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954054+0000", 2024-09-13T07:02:07.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1831000000000001e-05 2024-09-13T07:02:07.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969108+0000", 2024-09-13T07:02:07.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015054589 2024-09-13T07:02:07.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969146+0000", 2024-09-13T07:02:07.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8021000000000001e-05 2024-09-13T07:02:07.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969219+0000", 2024-09-13T07:02:07.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2991000000000006e-05 2024-09-13T07:02:07.118 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.118 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.118 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.118 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:58569 3.2 3:441e3a7e:::rbd_data.119475582a6.0000000000000018:16 [read 3276800~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T07:02:07.118 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:03.954065+0000", 2024-09-13T07:02:07.118 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0748888129999998, 2024-09-13T07:02:07.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015289670999999999, 2024-09-13T07:02:07.119 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.119 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:07.119 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.119 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.119 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.119 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 58569 2024-09-13T07:02:07.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.119 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954065+0000", 2024-09-13T07:02:07.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954065+0000", 2024-09-13T07:02:07.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954063+0000", 2024-09-13T07:02:07.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:07.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954069+0000", 2024-09-13T07:02:07.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7649999999999999e-06 2024-09-13T07:02:07.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954083+0000", 2024-09-13T07:02:07.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3797e-05 2024-09-13T07:02:07.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954094+0000", 2024-09-13T07:02:07.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.092e-05 2024-09-13T07:02:07.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969246+0000", 2024-09-13T07:02:07.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015151839 2024-09-13T07:02:07.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969287+0000", 2024-09-13T07:02:07.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1356999999999997e-05 2024-09-13T07:02:07.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969354+0000", 2024-09-13T07:02:07.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7355999999999997e-05 2024-09-13T07:02:07.123 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.123 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:58570 3.2 3:441e3a7e:::rbd_data.119475582a6.0000000000000018:16 [read 3473408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:03.954105+0000", 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0748485419999998, 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015385731999999999, 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 58570 2024-09-13T07:02:07.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.125 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954105+0000", 2024-09-13T07:02:07.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954105+0000", 2024-09-13T07:02:07.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954104+0000", 2024-09-13T07:02:07.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:07.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954110+0000", 2024-09-13T07:02:07.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1619999999999996e-06 2024-09-13T07:02:07.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954125+0000", 2024-09-13T07:02:07.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5512e-05 2024-09-13T07:02:07.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954137+0000", 2024-09-13T07:02:07.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1445e-05 2024-09-13T07:02:07.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969383+0000", 2024-09-13T07:02:07.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015245747 2024-09-13T07:02:07.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969421+0000", 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8930999999999997e-05 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969491+0000", 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9275000000000003e-05 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:58571 3.2 3:441e3a7e:::rbd_data.119475582a6.0000000000000018:16 [read 3670016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T07:02:07.128 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:03.954175+0000", 2024-09-13T07:02:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0747781490000001, 2024-09-13T07:02:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015467025000000001, 2024-09-13T07:02:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 58571 2024-09-13T07:02:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954175+0000", 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954175+0000", 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954175+0000", 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954181+0000", 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0599999999999996e-06 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954195+0000", 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4817000000000001e-05 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954207+0000", 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1226000000000001e-05 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969521+0000", 2024-09-13T07:02:07.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015314627000000001 2024-09-13T07:02:07.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969567+0000", 2024-09-13T07:02:07.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5268000000000001e-05 2024-09-13T07:02:07.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969642+0000", 2024-09-13T07:02:07.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5907000000000004e-05 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:58572 3.2 3:441e3a7e:::rbd_data.119475582a6.0000000000000018:16 [read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:03.954217+0000", 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0747363029999999, 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015531130000000001, 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.133 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.134 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 58572 2024-09-13T07:02:07.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.134 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954217+0000", 2024-09-13T07:02:07.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954217+0000", 2024-09-13T07:02:07.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954216+0000", 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954222+0000", 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9000000000000003e-06 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954235+0000", 2024-09-13T07:02:07.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.328e-05 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954252+0000", 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6663e-05 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969667+0000", 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015415306 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969707+0000", 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0074999999999998e-05 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969748+0000", 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1115000000000002e-05 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:58573 3.2 3:441e3a7e:::rbd_data.119475582a6.0000000000000018:16 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:03.954262+0000", 2024-09-13T07:02:07.137 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.0746913519999999, 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015601395000000001, 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 58573 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954262+0000", 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954262+0000", 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954261+0000", 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954267+0000", 2024-09-13T07:02:07.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:02:07.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:07.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954281+0000", 2024-09-13T07:02:07.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.434e-05 2024-09-13T07:02:07.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.954292+0000", 2024-09-13T07:02:07.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1124999999999999e-05 2024-09-13T07:02:07.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:07.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969781+0000", 2024-09-13T07:02:07.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015488327 2024-09-13T07:02:07.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:07.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969818+0000", 2024-09-13T07:02:07.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7537999999999999e-05 2024-09-13T07:02:07.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:07.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:07.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:07.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:03.969864+0000", 2024-09-13T07:02:07.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5513999999999998e-05 2024-09-13T07:02:07.142 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.142 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.142 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.142 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:07.142 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:07.142 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:07.144 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:07.144 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:07.142+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:07.244 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:07.242+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:07.262 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:07.263 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:07.263 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:07.263 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:07.274 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:07.348 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:07.341+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:07.374 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:07.375 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:07.384 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:07.446 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:07.442+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:07.502 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:07.502 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:07.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:07.502 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:07.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.502 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1524 3.14 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:07.502 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:07.502 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.455754833, 2024-09-13T07:02:07.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061190880000000003, 2024-09-13T07:02:07.502 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.503 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:07.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.503 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1524 2024-09-13T07:02:07.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.503 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:07.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:07.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044767+0000", 2024-09-13T07:02:07.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:07.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052670+0000", 2024-09-13T07:02:07.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079022450000000004 2024-09-13T07:02:07.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052682+0000", 2024-09-13T07:02:07.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2336e-05 2024-09-13T07:02:07.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052696+0000", 2024-09-13T07:02:07.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-13T07:02:07.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052755+0000", 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9298e-05 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052947+0000", 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019153900000000001 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.053694+0000", 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074663099999999999 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:07.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.102148+0000", 2024-09-13T07:02:07.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048454329999999997 2024-09-13T07:02:07.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:07.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105816+0000", 2024-09-13T07:02:07.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036675950000000001 2024-09-13T07:02:07.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:07.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105921+0000", 2024-09-13T07:02:07.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105874 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105959+0000", 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7991000000000003e-05 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19083 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3817472~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.051289023000001, 2024-09-13T07:02:07.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049218062999999999, 2024-09-13T07:02:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19083 2024-09-13T07:02:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:07.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:07.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:07.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449236+0000", 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.502e-06 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449238+0000", 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.689e-06 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449241+0000", 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5119999999999998e-06 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498359+0000", 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049117630000000002 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498399+0000", 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0726000000000003e-05 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498452+0000", 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3220999999999997e-05 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.512 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.513 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.513 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19084 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3465216~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.513 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:07.513 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.051280312999999, 2024-09-13T07:02:07.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049328261999999998, 2024-09-13T07:02:07.513 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.513 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.513 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19084 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:07.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:07.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449244+0000", 2024-09-13T07:02:07.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.454e-06 2024-09-13T07:02:07.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449247+0000", 2024-09-13T07:02:07.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.717e-06 2024-09-13T07:02:07.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449249+0000", 2024-09-13T07:02:07.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.424e-06 2024-09-13T07:02:07.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498477+0000", 2024-09-13T07:02:07.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049227235000000001 2024-09-13T07:02:07.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498518+0000", 2024-09-13T07:02:07.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0846000000000002e-05 2024-09-13T07:02:07.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498571+0000", 2024-09-13T07:02:07.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3810000000000001e-05 2024-09-13T07:02:07.517 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.517 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.517 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19085 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3842048~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449252+0000", 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.051271699999999, 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049435828000000001, 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19085 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449252+0000", 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449252+0000", 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449251+0000", 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449253+0000", 2024-09-13T07:02:07.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4389999999999999e-06 2024-09-13T07:02:07.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449256+0000", 2024-09-13T07:02:07.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:02:07.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449258+0000", 2024-09-13T07:02:07.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.435e-06 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498594+0000", 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049335639000000001 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498635+0000", 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1229000000000001e-05 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498688+0000", 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2555000000000001e-05 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19086 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3489792~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.051263044000001, 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049540534999999997, 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19086 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:07.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:07.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449269+0000", 2024-09-13T07:02:07.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6459999999999999e-06 2024-09-13T07:02:07.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449273+0000", 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9469999999999996e-06 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449276+0000", 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498707+0000", 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049431286999999997 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498748+0000", 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0976999999999997e-05 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498801+0000", 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3090000000000002e-05 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19087 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3866624~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.526 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:07.527 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.051245213, 2024-09-13T07:02:07.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049644714999999999, 2024-09-13T07:02:07.527 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.527 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.527 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.527 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.527 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.527 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19087 2024-09-13T07:02:07.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.527 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:07.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:07.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:07.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:07.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449280+0000", 2024-09-13T07:02:07.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.655e-06 2024-09-13T07:02:07.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449282+0000", 2024-09-13T07:02:07.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8600000000000001e-06 2024-09-13T07:02:07.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449285+0000", 2024-09-13T07:02:07.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4600000000000002e-06 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498819+0000", 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049534340000000003 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498871+0000", 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1412000000000002e-05 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498923+0000", 2024-09-13T07:02:07.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2252e-05 2024-09-13T07:02:07.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.531 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.531 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19088 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3514368~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.531 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:07.531 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.051236303, 2024-09-13T07:02:07.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049750612999999999, 2024-09-13T07:02:07.531 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.531 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.532 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19088 2024-09-13T07:02:07.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.532 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:07.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:07.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:07.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:07.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449288+0000", 2024-09-13T07:02:07.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.469e-06 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449291+0000", 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449294+0000", 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5069999999999999e-06 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498941+0000", 2024-09-13T07:02:07.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049647494 2024-09-13T07:02:07.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498983+0000", 2024-09-13T07:02:07.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1771999999999999e-05 2024-09-13T07:02:07.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499038+0000", 2024-09-13T07:02:07.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4888e-05 2024-09-13T07:02:07.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.535 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19089 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3891200~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.051227569, 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049857261, 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19089 2024-09-13T07:02:07.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.537 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:07.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:07.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449297+0000", 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4330000000000001e-06 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449301+0000", 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5260000000000002e-06 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449303+0000", 2024-09-13T07:02:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5110000000000002e-06 2024-09-13T07:02:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499050+0000", 2024-09-13T07:02:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049747121999999998 2024-09-13T07:02:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499091+0000", 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0580000000000001e-05 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499153+0000", 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2305999999999996e-05 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19090 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3538944~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:07.540 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.051218086, 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049959569000000002, 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19090 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449307+0000", 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4279999999999999e-06 2024-09-13T07:02:07.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449309+0000", 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.706e-06 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449312+0000", 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4930000000000001e-06 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499169+0000", 2024-09-13T07:02:07.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049857745000000002 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499210+0000", 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0706e-05 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499265+0000", 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4719000000000002e-05 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.545 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19091 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3915776~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.545 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:07.545 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.051209443999999, 2024-09-13T07:02:07.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050065959, 2024-09-13T07:02:07.545 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.545 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.545 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19091 2024-09-13T07:02:07.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.545 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:07.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:07.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449315+0000", 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.485e-06 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449318+0000", 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6740000000000001e-06 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449320+0000", 2024-09-13T07:02:07.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.413e-06 2024-09-13T07:02:07.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499283+0000", 2024-09-13T07:02:07.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049962196 2024-09-13T07:02:07.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499323+0000", 2024-09-13T07:02:07.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0799e-05 2024-09-13T07:02:07.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499380+0000", 2024-09-13T07:02:07.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6610999999999997e-05 2024-09-13T07:02:07.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.549 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.549 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19092 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3563520~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.549 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449323+0000", 2024-09-13T07:02:07.549 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.051200753, 2024-09-13T07:02:07.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050176212999999997, 2024-09-13T07:02:07.550 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.550 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.550 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19092 2024-09-13T07:02:07.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.550 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449323+0000", 2024-09-13T07:02:07.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449323+0000", 2024-09-13T07:02:07.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449322+0000", 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449324+0000", 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.424e-06 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449327+0000", 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-13T07:02:07.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449329+0000", 2024-09-13T07:02:07.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4820000000000001e-06 2024-09-13T07:02:07.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499397+0000", 2024-09-13T07:02:07.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050068362999999998 2024-09-13T07:02:07.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499441+0000", 2024-09-13T07:02:07.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3668999999999998e-05 2024-09-13T07:02:07.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499499+0000", 2024-09-13T07:02:07.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7778999999999997e-05 2024-09-13T07:02:07.555 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.555 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.555 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.556 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19093 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3940352~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.556 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:07.556 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.05119197, 2024-09-13T07:02:07.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050278704, 2024-09-13T07:02:07.556 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.556 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.556 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.556 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.556 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.556 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19093 2024-09-13T07:02:07.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.556 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:07.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:07.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449333+0000", 2024-09-13T07:02:07.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5570000000000001e-06 2024-09-13T07:02:07.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449335+0000", 2024-09-13T07:02:07.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-13T07:02:07.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449338+0000", 2024-09-13T07:02:07.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.491e-06 2024-09-13T07:02:07.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499513+0000", 2024-09-13T07:02:07.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050175143999999998 2024-09-13T07:02:07.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499554+0000", 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0729000000000003e-05 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499610+0000", 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6307000000000001e-05 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19094 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3588096~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.560 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:07.561 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.051183067, 2024-09-13T07:02:07.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050385554999999999, 2024-09-13T07:02:07.561 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.561 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.561 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19094 2024-09-13T07:02:07.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.561 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449342+0000", 2024-09-13T07:02:07.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.652e-06 2024-09-13T07:02:07.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449345+0000", 2024-09-13T07:02:07.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:02:07.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449348+0000", 2024-09-13T07:02:07.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5849999999999998e-06 2024-09-13T07:02:07.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499626+0000", 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050278728000000002 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499669+0000", 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2400000000000001e-05 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499726+0000", 2024-09-13T07:02:07.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7191999999999997e-05 2024-09-13T07:02:07.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.565 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.565 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19095 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3964928~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.565 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:07.565 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.051173356, 2024-09-13T07:02:07.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050492171000000002, 2024-09-13T07:02:07.565 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.565 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.566 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19095 2024-09-13T07:02:07.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.566 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:07.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449351+0000", 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.449e-06 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449354+0000", 2024-09-13T07:02:07.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.836e-06 2024-09-13T07:02:07.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449357+0000", 2024-09-13T07:02:07.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5459999999999998e-06 2024-09-13T07:02:07.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499744+0000", 2024-09-13T07:02:07.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050387339000000003 2024-09-13T07:02:07.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499786+0000", 2024-09-13T07:02:07.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2293999999999999e-05 2024-09-13T07:02:07.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499842+0000", 2024-09-13T07:02:07.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5927000000000002e-05 2024-09-13T07:02:07.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.569 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19096 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3612672~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.051164287000001, 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050599058000000002, 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19096 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:07.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:07.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:07.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:07.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449360+0000", 2024-09-13T07:02:07.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4529999999999999e-06 2024-09-13T07:02:07.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449363+0000", 2024-09-13T07:02:07.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6680000000000001e-06 2024-09-13T07:02:07.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449366+0000", 2024-09-13T07:02:07.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6359999999999998e-06 2024-09-13T07:02:07.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499860+0000", 2024-09-13T07:02:07.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050494820000000003 2024-09-13T07:02:07.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499902+0000", 2024-09-13T07:02:07.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1111e-05 2024-09-13T07:02:07.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499958+0000", 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6594000000000001e-05 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19110 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 4186112~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.048040937, 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049302258000000002, 2024-09-13T07:02:07.574 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.575 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.575 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.575 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.575 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.575 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19110 2024-09-13T07:02:07.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.575 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:07.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:07.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:07.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:07.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452487+0000", 2024-09-13T07:02:07.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9980000000000003e-06 2024-09-13T07:02:07.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452490+0000", 2024-09-13T07:02:07.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1410000000000001e-06 2024-09-13T07:02:07.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452493+0000", 2024-09-13T07:02:07.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6010000000000002e-06 2024-09-13T07:02:07.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501587+0000", 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049094282000000003 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501680+0000", 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2675000000000003e-05 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501785+0000", 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000104993 2024-09-13T07:02:07.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19111 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3760128~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.048028275, 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049401467999999997, 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.579 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19111 2024-09-13T07:02:07.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.580 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:07.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:07.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452497+0000", 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7099999999999999e-06 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452499+0000", 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6929999999999999e-06 2024-09-13T07:02:07.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452502+0000", 2024-09-13T07:02:07.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5900000000000002e-06 2024-09-13T07:02:07.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501801+0000", 2024-09-13T07:02:07.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049299005 2024-09-13T07:02:07.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501838+0000", 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7282e-05 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501897+0000", 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8405000000000001e-05 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19112 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3784704~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:07.583 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.048019136000001, 2024-09-13T07:02:07.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049618640999999998, 2024-09-13T07:02:07.584 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.584 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.584 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19112 2024-09-13T07:02:07.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.584 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:07.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452505+0000", 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.364e-06 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452508+0000", 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7470000000000001e-06 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452511+0000", 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501932+0000", 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049421395999999999 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502019+0000", 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7247e-05 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502123+0000", 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103589 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.587 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19113 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3809280~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.048010111, 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049709729000000001, 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19113 2024-09-13T07:02:07.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.589 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:07.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:07.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:07.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:07.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452515+0000", 2024-09-13T07:02:07.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659999999999999e-06 2024-09-13T07:02:07.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452517+0000", 2024-09-13T07:02:07.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6189999999999998e-06 2024-09-13T07:02:07.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452520+0000", 2024-09-13T07:02:07.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7760000000000002e-06 2024-09-13T07:02:07.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502134+0000", 2024-09-13T07:02:07.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049614232000000001 2024-09-13T07:02:07.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502173+0000", 2024-09-13T07:02:07.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8486999999999999e-05 2024-09-13T07:02:07.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502223+0000", 2024-09-13T07:02:07.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0392999999999997e-05 2024-09-13T07:02:07.592 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.592 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.593 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.593 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:48369 3.f e58/22 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head v 58'78, mlcod=57'75)", 2024-09-13T07:02:07.593 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:07.593 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2600569680000007, 2024-09-13T07:02:07.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.057439072000000001, 2024-09-13T07:02:07.593 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:07.593 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:07.593 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:07.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:07.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:07.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:07.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:07.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:07.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:07.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240464+0000", 2024-09-13T07:02:07.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245716+0000", 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005252329 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245745+0000", 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8844e-05 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:07.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245758+0000", 2024-09-13T07:02:07.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2242000000000001e-05 2024-09-13T07:02:07.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:07.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245849+0000", 2024-09-13T07:02:07.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1187999999999995e-05 2024-09-13T07:02:07.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:07.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245898+0000", 2024-09-13T07:02:07.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8881000000000002e-05 2024-09-13T07:02:07.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:07.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297837+0000", 2024-09-13T07:02:07.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051938947999999999 2024-09-13T07:02:07.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:07.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:07.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:07.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297906+0000", 2024-09-13T07:02:07.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8955999999999995e-05 2024-09-13T07:02:07.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.597 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.598 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.598 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:07.598 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:07.598 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:07.599 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:07.600 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:07.598+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:07.700 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:07.698+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:07.757 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:07.757 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:07.757 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:07.757 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:07.767 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:07.780 INFO:teuthology.orchestra.run.smithi118.stdout:1797 trunc from 0xac56c70 to 0xade6725 2024-09-13T07:02:07.806 INFO:teuthology.orchestra.run.smithi118.stdout:1798 read 0x76b9ec5 thru 0x76c4b71 (0xacad bytes) 2024-09-13T07:02:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:1799 punch from 0x5ec6c44 to 0x5ec9304, (0x26c0 bytes) 2024-09-13T07:02:07.811 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:07.815 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:07.814+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:07.817 INFO:teuthology.orchestra.run.smithi118.stdout:1801 trunc from 0xade6725 to 0xd791c56 2024-09-13T07:02:07.829 INFO:teuthology.orchestra.run.smithi118.stdout:1802 read 0x73b45a4 thru 0x73bfcc1 (0xb71e bytes) 2024-09-13T07:02:07.861 INFO:teuthology.orchestra.run.smithi118.stdout:1803 trunc from 0xd791c56 to 0x2ce2387 2024-09-13T07:02:07.890 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:07.890 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:07.890 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:07.890 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:07.890 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:07.901 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:07.901+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:07.903 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:07.949 INFO:teuthology.orchestra.run.smithi118.stdout:1804 write 0x9b4656f thru 0x9b5173d (0xb1cf bytes) 2024-09-13T07:02:07.961 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:07.970 INFO:teuthology.orchestra.run.smithi118.stdout:1806 trunc from 0x9b5173e to 0xd5fdf24 2024-09-13T07:02:07.982 INFO:teuthology.orchestra.run.smithi118.stdout:1807 write 0x137bbd6 thru 0x13847c2 (0x8bed bytes) 2024-09-13T07:02:08.002 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:08.001+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:08.013 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:08.013 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:08.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:08.014 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:08.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.014 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1531 3.c 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:08.014 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:08.014 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.962497935000002, 2024-09-13T07:02:08.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063168486999999995, 2024-09-13T07:02:08.014 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.014 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.014 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.014 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.014 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1531 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049511+0000", 2024-09-13T07:02:08.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:08.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069234+0000", 2024-09-13T07:02:08.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019723920999999998 2024-09-13T07:02:08.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069269+0000", 2024-09-13T07:02:08.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4446999999999997e-05 2024-09-13T07:02:08.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069286+0000", 2024-09-13T07:02:08.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7405999999999999e-05 2024-09-13T07:02:08.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069370+0000", 2024-09-13T07:02:08.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-05 2024-09-13T07:02:08.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069511+0000", 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014134299999999999 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070463+0000", 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095167899999999998 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105076+0000", 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034613139000000001 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112562+0000", 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074861429999999998 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112631+0000", 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112680+0000", 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9607000000000001e-05 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.019 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7628 3.1f 3:fdf2e50b:::rbd_data.1194c5936dae.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1032192~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.249571778000004, 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.630407088, 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7628 2024-09-13T07:02:08.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762435+0000", 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762543+0000", 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010850500000000001 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762597+0000", 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4262000000000003e-05 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762626+0000", 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9176e-05 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762761+0000", 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013405599999999999 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762776+0000", 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5679999999999999e-05 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388492+0000", 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.62571606300000004 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for missing object", 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388614+0000", 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012126 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390837+0000", 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002223806 2024-09-13T07:02:08.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390963+0000", 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012550300000000001 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391127+0000", 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163995 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391989+0000", 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086230499999999995 2024-09-13T07:02:08.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:08.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392662+0000", 2024-09-13T07:02:08.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067260799999999999 2024-09-13T07:02:08.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:08.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392728+0000", 2024-09-13T07:02:08.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6501000000000006e-05 2024-09-13T07:02:08.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392845+0000", 2024-09-13T07:02:08.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116665 2024-09-13T07:02:08.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.027 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.027 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15862 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3440640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.027 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.772590+0000", 2024-09-13T07:02:08.027 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.239419841, 2024-09-13T07:02:08.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048639120000000001, 2024-09-13T07:02:08.027 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15862 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772590+0000", 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772590+0000", 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772589+0000", 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772592+0000", 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772598+0000", 2024-09-13T07:02:08.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.463e-06 2024-09-13T07:02:08.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772603+0000", 2024-09-13T07:02:08.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5450000000000003e-06 2024-09-13T07:02:08.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821064+0000", 2024-09-13T07:02:08.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048461046000000001 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821137+0000", 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2507000000000002e-05 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821229+0000", 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2082000000000004e-05 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.032 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15863 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3465216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.032 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.772608+0000", 2024-09-13T07:02:08.032 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.239401567000002, 2024-09-13T07:02:08.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048769738, 2024-09-13T07:02:08.032 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.032 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.032 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.032 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.032 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.032 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15863 2024-09-13T07:02:08.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.032 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772608+0000", 2024-09-13T07:02:08.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772608+0000", 2024-09-13T07:02:08.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772607+0000", 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772611+0000", 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3759999999999999e-06 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772616+0000", 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3970000000000002e-06 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772621+0000", 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8459999999999999e-06 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821253+0000", 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048632149 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821308+0000", 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4639999999999999e-05 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821378+0000", 2024-09-13T07:02:08.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9987999999999995e-05 2024-09-13T07:02:08.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.036 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.036 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15870 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3637248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.036 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:02:08.036 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.238113164000001, 2024-09-13T07:02:08.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048771508999999998, 2024-09-13T07:02:08.036 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.036 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15870 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773899+0000", 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1949999999999999e-06 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773904+0000", 2024-09-13T07:02:08.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.356e-06 2024-09-13T07:02:08.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773909+0000", 2024-09-13T07:02:08.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0849999999999996e-06 2024-09-13T07:02:08.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822505+0000", 2024-09-13T07:02:08.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048595933000000001 2024-09-13T07:02:08.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822579+0000", 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4276999999999994e-05 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822668+0000", 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8522000000000004e-05 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15871 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.238096194000001, 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048913976999999997, 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15871 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:02:08.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:02:08.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:02:08.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773916+0000", 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0369999999999999e-06 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773921+0000", 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.383e-06 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773927+0000", 2024-09-13T07:02:08.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.147e-06 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822692+0000", 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048765583000000001 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822753+0000", 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0637999999999998e-05 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822827+0000", 2024-09-13T07:02:08.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4630999999999997e-05 2024-09-13T07:02:08.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.045 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.045 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15884 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.045 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.776551+0000", 2024-09-13T07:02:08.045 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.235458935, 2024-09-13T07:02:08.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048644055999999998, 2024-09-13T07:02:08.045 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.045 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15884 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776551+0000", 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776551+0000", 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776550+0000", 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776553+0000", 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1520000000000001e-06 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776559+0000", 2024-09-13T07:02:08.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4890000000000003e-06 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776564+0000", 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0390000000000004e-06 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825037+0000", 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048472701 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825109+0000", 2024-09-13T07:02:08.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1884000000000005e-05 2024-09-13T07:02:08.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825195+0000", 2024-09-13T07:02:08.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6187000000000007e-05 2024-09-13T07:02:08.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.049 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.049 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15885 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 4005888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.049 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.776569+0000", 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.235440930999999, 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048792239000000001, 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15885 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776569+0000", 2024-09-13T07:02:08.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776569+0000", 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776568+0000", 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776571+0000", 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0419999999999998e-06 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776577+0000", 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5709999999999997e-06 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776582+0000", 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9540000000000003e-06 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825231+0000", 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048649031000000002 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825288+0000", 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7148999999999998e-05 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825361+0000", 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3133999999999995e-05 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.053 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15907 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.231413521, 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048713259000000002, 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15907 2024-09-13T07:02:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:02:08.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780598+0000", 2024-09-13T07:02:08.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7e-06 2024-09-13T07:02:08.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780604+0000", 2024-09-13T07:02:08.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-06 2024-09-13T07:02:08.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780609+0000", 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6750000000000001e-06 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829162+0000", 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048553600000000002 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829226+0000", 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3547000000000007e-05 2024-09-13T07:02:08.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829309+0000", 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3547000000000005e-05 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15908 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.780613+0000", 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.231396608000001, 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048901932000000002, 2024-09-13T07:02:08.058 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15908 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780613+0000", 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780613+0000", 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780612+0000", 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780615+0000", 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0120000000000001e-06 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780621+0000", 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8320000000000002e-06 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780626+0000", 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829346+0000", 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048720439999999997 2024-09-13T07:02:08.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829419+0000", 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2347000000000004e-05 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829515+0000", 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6181000000000006e-05 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15928 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.227365364000001, 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048610352000000003, 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15928 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784646+0000", 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3479999999999998e-06 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784652+0000", 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.485e-06 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784657+0000", 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8189999999999999e-06 2024-09-13T07:02:08.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833039+0000", 2024-09-13T07:02:08.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048382890999999997 2024-09-13T07:02:08.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833116+0000", 2024-09-13T07:02:08.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6371000000000005e-05 2024-09-13T07:02:08.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833255+0000", 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138784 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15929 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784661+0000", 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.227348803999998, 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048815364, 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.067 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.068 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15929 2024-09-13T07:02:08.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.068 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784661+0000", 2024-09-13T07:02:08.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784661+0000", 2024-09-13T07:02:08.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784660+0000", 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784663+0000", 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5639999999999999e-06 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784668+0000", 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4079999999999997e-06 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784673+0000", 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5120000000000002e-06 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833284+0000", 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048611047999999997 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833382+0000", 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7571999999999993e-05 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833476+0000", 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4653000000000003e-05 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15930 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784677+0000", 2024-09-13T07:02:08.071 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.227332377, 2024-09-13T07:02:08.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048947388000000001, 2024-09-13T07:02:08.072 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.072 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.072 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15930 2024-09-13T07:02:08.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.072 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784677+0000", 2024-09-13T07:02:08.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784677+0000", 2024-09-13T07:02:08.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784676+0000", 2024-09-13T07:02:08.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:08.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784680+0000", 2024-09-13T07:02:08.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1930000000000002e-06 2024-09-13T07:02:08.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784685+0000", 2024-09-13T07:02:08.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:02:08.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784690+0000", 2024-09-13T07:02:08.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6949999999999999e-06 2024-09-13T07:02:08.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833496+0000", 2024-09-13T07:02:08.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048806041000000001 2024-09-13T07:02:08.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833541+0000", 2024-09-13T07:02:08.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4886999999999999e-05 2024-09-13T07:02:08.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833625+0000", 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4073000000000006e-05 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15931 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784704+0000", 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.227305922999999, 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049199247000000002, 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.077 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15931 2024-09-13T07:02:08.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.077 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784704+0000", 2024-09-13T07:02:08.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784704+0000", 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784703+0000", 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784706+0000", 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1e-06 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784712+0000", 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8730000000000004e-06 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784718+0000", 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3850000000000001e-06 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833671+0000", 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048952993 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833787+0000", 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011663300000000001 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833903+0000", 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115747 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15951 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.080 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788591+0000", 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.223419104000001, 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048616683000000001, 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15951 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788591+0000", 2024-09-13T07:02:08.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788591+0000", 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788590+0000", 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788593+0000", 2024-09-13T07:02:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5720000000000001e-06 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788598+0000", 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4840000000000003e-06 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788603+0000", 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8659999999999997e-06 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837089+0000", 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048485815000000002 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837143+0000", 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3794e-05 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837207+0000", 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4566999999999994e-05 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.084 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15952 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788607+0000", 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.223402551, 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048772226000000002, 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15952 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.085 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788607+0000", 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788607+0000", 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788606+0000", 2024-09-13T07:02:08.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788609+0000", 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7640000000000001e-06 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788615+0000", 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8490000000000002e-06 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788620+0000", 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3859999999999998e-06 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837240+0000", 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048619860000000001 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837301+0000", 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0304999999999999e-05 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837379+0000", 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8689999999999994e-05 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15953 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.223373165000002, 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04888609, 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15953 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:02:08.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788639+0000", 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4740000000000001e-06 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788646+0000", 2024-09-13T07:02:08.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788652+0000", 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7289999999999997e-06 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837400+0000", 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048747946 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837455+0000", 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5621000000000002e-05 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837523+0000", 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7158999999999999e-05 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15954 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1507328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:02:08.093 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.223353388, 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049039567999999999, 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15954 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788659+0000", 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1889999999999999e-06 2024-09-13T07:02:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788664+0000", 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788670+0000", 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6999999999999996e-06 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837552+0000", 2024-09-13T07:02:08.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048882402999999998 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837615+0000", 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2354000000000006e-05 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837696+0000", 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1286999999999996e-05 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.893421469, 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:08.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:08.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:02:08.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:02:08.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:02:08.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:02:08.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:02:08.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:08.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:02:08.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:02:08.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:02:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:02:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:02:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:02:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:08.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:08.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:02:08.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:02:08.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:02:08.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:02:08.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.104 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.104 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48376 3.c 3:350dc8a4:::rbd_data.1194fc792839.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:08.105 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:08.105 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.7828006260000002, 2024-09-13T07:02:08.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.068448320000000007, 2024-09-13T07:02:08.105 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.105 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.105 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48376 2024-09-13T07:02:08.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.105 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229207+0000", 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.241745+0000", 2024-09-13T07:02:08.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012537726000000001 2024-09-13T07:02:08.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244174+0000", 2024-09-13T07:02:08.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024284760000000002 2024-09-13T07:02:08.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244198+0000", 2024-09-13T07:02:08.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4629000000000001e-05 2024-09-13T07:02:08.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244312+0000", 2024-09-13T07:02:08.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114282 2024-09-13T07:02:08.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244679+0000", 2024-09-13T07:02:08.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000366399 2024-09-13T07:02:08.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:08.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.246046+0000", 2024-09-13T07:02:08.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013672479999999999 2024-09-13T07:02:08.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:08.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.266272+0000", 2024-09-13T07:02:08.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020226381000000002 2024-09-13T07:02:08.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:08.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297510+0000", 2024-09-13T07:02:08.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031237576999999999 2024-09-13T07:02:08.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:08.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297595+0000", 2024-09-13T07:02:08.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5413000000000004e-05 2024-09-13T07:02:08.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297657+0000", 2024-09-13T07:02:08.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1860999999999996e-05 2024-09-13T07:02:08.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.110 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.110 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.110 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:08.112 INFO:teuthology.orchestra.run.smithi118.stdout:1808 writesame 0x51e215a thru 0x51ec60c data_size 0x1(0xa4b3 bytes) 2024-09-13T07:02:08.112 INFO:teuthology.orchestra.run.smithi118.stdout:1809 trunc from 0xd5fdf24 to 0xe356248 2024-09-13T07:02:08.112 INFO:teuthology.orchestra.run.smithi118.stdout:1810 trunc from 0xe356248 to 0x2526a60 2024-09-13T07:02:08.112 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:08.109+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:08.113 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:08.207 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:02:08.210 INFO:teuthology.orchestra.run.smithi118.stdout:1811 write 0x32a1acb thru 0x32b1999 (0xfecf bytes) 2024-09-13T07:02:08.220 INFO:teuthology.orchestra.run.smithi118.stdout:1812 read 0x69f5ea thru 0x6a0056 (0xa6d bytes) 2024-09-13T07:02:08.221 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:08.217+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:08.241 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:08.241 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:08.241 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:08.242 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:08.242 INFO:teuthology.orchestra.run.smithi118.stdout:1813 write 0x9e56122 thru 0x9e5eab9 (0x8998 bytes) 2024-09-13T07:02:08.253 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:08.268 INFO:teuthology.orchestra.run.smithi118.stdout:1814 writesame 0xc9486d4 thru 0xc95576a data_size 0x1(0xd097 bytes) 2024-09-13T07:02:08.268 INFO:teuthology.orchestra.run.smithi118.stdout:1816 read 0xc536a8 thru 0xc54851 (0x11aa bytes) 2024-09-13T07:02:08.270 INFO:teuthology.orchestra.run.smithi118.stdout:1820 write 0xf3d05c thru 0xf42a6c (0x5a11 bytes) 2024-09-13T07:02:08.313 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:08.310+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:08.365 INFO:teuthology.orchestra.run.smithi118.stdout:1821 write 0x9ac6dc5 thru 0x9acb1ae (0x43ea bytes) 2024-09-13T07:02:08.374 INFO:teuthology.orchestra.run.smithi118.stdout:1825 write 0xa370d67 thru 0xa37b800 (0xaa9a bytes) 2024-09-13T07:02:08.375 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:08.375 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:08.375 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:08.375 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:08.375 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:08.386 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:08.419 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:08.413+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:08.430 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:08.430 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:08.430 INFO:teuthology.orchestra.run.smithi118.stdout:1829 write 0x7df4a2f thru 0x7df9256 (0x4828 bytes) 2024-09-13T07:02:08.441 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:08.441 INFO:teuthology.orchestra.run.smithi118.stdout:1831 write 0x6936924 thru 0x69390f2 (0x27cf bytes) 2024-09-13T07:02:08.463 INFO:teuthology.orchestra.run.smithi118.stdout:1832 write 0xd9bbb45 thru 0xd9bbcff (0x1bb bytes) 2024-09-13T07:02:08.469 INFO:teuthology.orchestra.run.smithi118.stdout:1833 read 0x2a0daf3 thru 0x2a184d5 (0xa9e3 bytes) 2024-09-13T07:02:08.469 INFO:teuthology.orchestra.run.smithi118.stdout:1834 read 0x73ceb39 thru 0x73dad29 (0xc1f1 bytes) 2024-09-13T07:02:08.470 INFO:teuthology.orchestra.run.smithi118.stdout:1837 writesame 0x9b3128e thru 0x9b357ed data_size 0x1(0x4560 bytes) 2024-09-13T07:02:08.496 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.435802903999999, 2024-09-13T07:02:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:02:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:02:08.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:02:08.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:02:08.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:02:08.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:02:08.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:02:08.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:08.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:02:08.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:02:08.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:08.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:02:08.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:02:08.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:08.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:02:08.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:02:08.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:08.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:02:08.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:02:08.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:02:08.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:02:08.502 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1534 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14365141] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.428975352999998, 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040478761000000002, 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.503 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1534 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065747+0000", 2024-09-13T07:02:08.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:08.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075258+0000", 2024-09-13T07:02:08.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095108710000000006 2024-09-13T07:02:08.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075288+0000", 2024-09-13T07:02:08.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0085000000000001e-05 2024-09-13T07:02:08.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075305+0000", 2024-09-13T07:02:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7748e-05 2024-09-13T07:02:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075367+0000", 2024-09-13T07:02:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1867999999999998e-05 2024-09-13T07:02:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075520+0000", 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015298800000000001 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.076247+0000", 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072720499999999997 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105157+0000", 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028909642999999999 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:08.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106124+0000", 2024-09-13T07:02:08.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096729400000000005 2024-09-13T07:02:08.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:08.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106173+0000", 2024-09-13T07:02:08.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8196999999999999e-05 2024-09-13T07:02:08.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106227+0000", 2024-09-13T07:02:08.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4388999999999997e-05 2024-09-13T07:02:08.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.509 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.509 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61282 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3599360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.509 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.881956+0000", 2024-09-13T07:02:08.509 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61276778799999998, 2024-09-13T07:02:08.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018514481999999999, 2024-09-13T07:02:08.509 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.509 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61282 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881956+0000", 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881956+0000", 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881955+0000", 2024-09-13T07:02:08.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881959+0000", 2024-09-13T07:02:08.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5870000000000001e-06 2024-09-13T07:02:08.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881964+0000", 2024-09-13T07:02:08.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.682e-06 2024-09-13T07:02:08.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881969+0000", 2024-09-13T07:02:08.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5050000000000001e-06 2024-09-13T07:02:08.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900353+0000", 2024-09-13T07:02:08.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018384461000000001 2024-09-13T07:02:08.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900421+0000", 2024-09-13T07:02:08.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7764999999999999e-05 2024-09-13T07:02:08.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900470+0000", 2024-09-13T07:02:08.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9162999999999997e-05 2024-09-13T07:02:08.513 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.513 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.513 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61283 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3603456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.881973+0000", 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61275076799999995, 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018634186, 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61283 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881973+0000", 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881973+0000", 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881972+0000", 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881975+0000", 2024-09-13T07:02:08.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0150000000000002e-06 2024-09-13T07:02:08.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881979+0000", 2024-09-13T07:02:08.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8359999999999999e-06 2024-09-13T07:02:08.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881984+0000", 2024-09-13T07:02:08.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2719999999999999e-06 2024-09-13T07:02:08.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900500+0000", 2024-09-13T07:02:08.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018516391 2024-09-13T07:02:08.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900558+0000", 2024-09-13T07:02:08.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7751999999999999e-05 2024-09-13T07:02:08.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900607+0000", 2024-09-13T07:02:08.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9237999999999999e-05 2024-09-13T07:02:08.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.518 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.519 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61284 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3607552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.519 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.881988+0000", 2024-09-13T07:02:08.519 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61273602699999996, 2024-09-13T07:02:08.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018734458999999998, 2024-09-13T07:02:08.519 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.519 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.519 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61284 2024-09-13T07:02:08.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.519 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881988+0000", 2024-09-13T07:02:08.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881988+0000", 2024-09-13T07:02:08.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881987+0000", 2024-09-13T07:02:08.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881989+0000", 2024-09-13T07:02:08.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3539999999999998e-06 2024-09-13T07:02:08.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881994+0000", 2024-09-13T07:02:08.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9590000000000003e-06 2024-09-13T07:02:08.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881999+0000", 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4689999999999999e-06 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900631+0000", 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018632602000000002 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900670+0000", 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8195000000000003e-05 2024-09-13T07:02:08.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900722+0000", 2024-09-13T07:02:08.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2454999999999998e-05 2024-09-13T07:02:08.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.523 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.523 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61285 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3611648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.523 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882002+0000", 2024-09-13T07:02:08.524 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61272120799999996, 2024-09-13T07:02:08.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018829511, 2024-09-13T07:02:08.524 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.524 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.524 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61285 2024-09-13T07:02:08.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.524 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882002+0000", 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882002+0000", 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882002+0000", 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882115+0000", 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112694 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882123+0000", 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5310000000000001e-06 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882129+0000", 2024-09-13T07:02:08.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1430000000000003e-06 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900752+0000", 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018622343999999999 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900789+0000", 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7113999999999997e-05 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900832+0000", 2024-09-13T07:02:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3056999999999998e-05 2024-09-13T07:02:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61286 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3615744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882135+0000", 2024-09-13T07:02:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61258855300000004, 2024-09-13T07:02:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018822186000000001, 2024-09-13T07:02:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61286 2024-09-13T07:02:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882135+0000", 2024-09-13T07:02:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882135+0000", 2024-09-13T07:02:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882134+0000", 2024-09-13T07:02:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882137+0000", 2024-09-13T07:02:08.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8870000000000001e-06 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882144+0000", 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3339999999999998e-06 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882148+0000", 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3159999999999998e-06 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900863+0000", 2024-09-13T07:02:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018715033999999998 2024-09-13T07:02:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900920+0000", 2024-09-13T07:02:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7466999999999997e-05 2024-09-13T07:02:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900957+0000", 2024-09-13T07:02:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6757000000000001e-05 2024-09-13T07:02:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.532 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61287 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3619840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882154+0000", 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61256924899999998, 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018937841, 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61287 2024-09-13T07:02:08.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882154+0000", 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882154+0000", 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882154+0000", 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882156+0000", 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882162+0000", 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.587e-06 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882170+0000", 2024-09-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3750000000000003e-06 2024-09-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900983+0000", 2024-09-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018812969999999998 2024-09-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901029+0000", 2024-09-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5952999999999999e-05 2024-09-13T07:02:08.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901092+0000", 2024-09-13T07:02:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3231000000000005e-05 2024-09-13T07:02:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61288 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3623936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882174+0000", 2024-09-13T07:02:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61254912299999997, 2024-09-13T07:02:08.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019067711000000001, 2024-09-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61288 2024-09-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882174+0000", 2024-09-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882174+0000", 2024-09-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882174+0000", 2024-09-13T07:02:08.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882187+0000", 2024-09-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3475999999999999e-05 2024-09-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882194+0000", 2024-09-13T07:02:08.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2700000000000001e-06 2024-09-13T07:02:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882203+0000", 2024-09-13T07:02:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0920000000000004e-06 2024-09-13T07:02:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901119+0000", 2024-09-13T07:02:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018916540999999999 2024-09-13T07:02:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901170+0000", 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1136e-05 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901242+0000", 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1713999999999998e-05 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61289 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3628032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.542 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882208+0000", 2024-09-13T07:02:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61251557300000004, 2024-09-13T07:02:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019174309, 2024-09-13T07:02:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61289 2024-09-13T07:02:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882208+0000", 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882208+0000", 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882207+0000", 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882210+0000", 2024-09-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6749999999999998e-06 2024-09-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882216+0000", 2024-09-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6960000000000002e-06 2024-09-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882220+0000", 2024-09-13T07:02:08.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3250000000000001e-06 2024-09-13T07:02:08.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901271+0000", 2024-09-13T07:02:08.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019051097999999999 2024-09-13T07:02:08.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901328+0000", 2024-09-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7185999999999998e-05 2024-09-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901382+0000", 2024-09-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3875999999999997e-05 2024-09-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61290 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3632128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.547 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882224+0000", 2024-09-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61249936500000002, 2024-09-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019307841999999999, 2024-09-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61290 2024-09-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882224+0000", 2024-09-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882224+0000", 2024-09-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882224+0000", 2024-09-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882226+0000", 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1490000000000001e-06 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882231+0000", 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.344e-06 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882235+0000", 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1980000000000003e-06 2024-09-13T07:02:08.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901416+0000", 2024-09-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019180127000000002 2024-09-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901480+0000", 2024-09-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4496e-05 2024-09-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901532+0000", 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1972999999999999e-05 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61291 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3636224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882239+0000", 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.612484417, 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019420103000000001, 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61291 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882239+0000", 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882239+0000", 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882239+0000", 2024-09-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882241+0000", 2024-09-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9659999999999999e-06 2024-09-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882247+0000", 2024-09-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0750000000000003e-06 2024-09-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882251+0000", 2024-09-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4739999999999999e-06 2024-09-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901559+0000", 2024-09-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019307656999999999 2024-09-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901602+0000", 2024-09-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2855000000000002e-05 2024-09-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901659+0000", 2024-09-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7355999999999998e-05 2024-09-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61292 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3640320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882255+0000", 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61246847599999998, 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019544367999999999, 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61292 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882255+0000", 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882255+0000", 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882255+0000", 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882257+0000", 2024-09-13T07:02:08.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8819999999999999e-06 2024-09-13T07:02:08.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882262+0000", 2024-09-13T07:02:08.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2619999999999998e-06 2024-09-13T07:02:08.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882266+0000", 2024-09-13T07:02:08.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3290000000000004e-06 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901685+0000", 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019418986999999999 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901752+0000", 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6927000000000005e-05 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901799+0000", 2024-09-13T07:02:08.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7298e-05 2024-09-13T07:02:08.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.561 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.561 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61293 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3644416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.561 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882270+0000", 2024-09-13T07:02:08.561 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61245350300000001, 2024-09-13T07:02:08.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019653103000000002, 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61293 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882270+0000", 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882270+0000", 2024-09-13T07:02:08.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882270+0000", 2024-09-13T07:02:08.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882273+0000", 2024-09-13T07:02:08.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9009999999999998e-06 2024-09-13T07:02:08.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882278+0000", 2024-09-13T07:02:08.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9060000000000001e-06 2024-09-13T07:02:08.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882283+0000", 2024-09-13T07:02:08.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2569999999999998e-06 2024-09-13T07:02:08.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901826+0000", 2024-09-13T07:02:08.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019543362000000002 2024-09-13T07:02:08.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901866+0000", 2024-09-13T07:02:08.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9586999999999999e-05 2024-09-13T07:02:08.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901923+0000", 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7435000000000002e-05 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61294 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3648512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882291+0000", 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61243253799999997, 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019759262999999999, 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61294 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882291+0000", 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882291+0000", 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882290+0000", 2024-09-13T07:02:08.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882293+0000", 2024-09-13T07:02:08.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6730000000000001e-06 2024-09-13T07:02:08.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882299+0000", 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.976e-06 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882304+0000", 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7040000000000002e-06 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901950+0000", 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019646101999999999 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902001+0000", 2024-09-13T07:02:08.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0853999999999998e-05 2024-09-13T07:02:08.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902050+0000", 2024-09-13T07:02:08.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9540999999999999e-05 2024-09-13T07:02:08.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.570 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.570 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61295 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3652608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882318+0000", 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61240572500000001, 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019820681, 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61295 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882318+0000", 2024-09-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882318+0000", 2024-09-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882317+0000", 2024-09-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882320+0000", 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5940000000000001e-06 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882326+0000", 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.66e-06 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882336+0000", 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424e-05 2024-09-13T07:02:08.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902065+0000", 2024-09-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01972896 2024-09-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902104+0000", 2024-09-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9276000000000001e-05 2024-09-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902139+0000", 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4288000000000001e-05 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61296 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3656704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882340+0000", 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61238317099999995, 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019912633999999999, 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.575 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61296 2024-09-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882340+0000", 2024-09-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882340+0000", 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882340+0000", 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882343+0000", 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6170000000000001e-06 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882348+0000", 2024-09-13T07:02:08.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5559999999999998e-06 2024-09-13T07:02:08.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882352+0000", 2024-09-13T07:02:08.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.369e-06 2024-09-13T07:02:08.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902151+0000", 2024-09-13T07:02:08.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019798748000000001 2024-09-13T07:02:08.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902214+0000", 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3244000000000006e-05 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902253+0000", 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8588999999999999e-05 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61297 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3660800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882356+0000", 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61236747400000002, 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020016612999999999, 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61297 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882356+0000", 2024-09-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882356+0000", 2024-09-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882356+0000", 2024-09-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882362+0000", 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.494e-06 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882369+0000", 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2859999999999996e-06 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882378+0000", 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3109999999999995e-06 2024-09-13T07:02:08.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902266+0000", 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019888059999999999 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902307+0000", 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1496999999999999e-05 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902373+0000", 2024-09-13T07:02:08.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5287e-05 2024-09-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61298 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3664896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882382+0000", 2024-09-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61234113999999995, 2024-09-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020103925000000002, 2024-09-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61298 2024-09-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882382+0000", 2024-09-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882382+0000", 2024-09-13T07:02:08.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882382+0000", 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882384+0000", 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4669999999999998e-06 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882390+0000", 2024-09-13T07:02:08.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6720000000000001e-06 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882395+0000", 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5170000000000001e-06 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902397+0000", 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020002457000000001 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902443+0000", 2024-09-13T07:02:08.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5725e-05 2024-09-13T07:02:08.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902486+0000", 2024-09-13T07:02:08.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3674000000000002e-05 2024-09-13T07:02:08.588 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.588 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.588 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.588 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61299 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3668992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:08.588 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882398+0000", 2024-09-13T07:02:08.589 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61232524399999999, 2024-09-13T07:02:08.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020201845999999999, 2024-09-13T07:02:08.589 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.589 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.589 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.589 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.589 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.589 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61299 2024-09-13T07:02:08.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.589 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882398+0000", 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882398+0000", 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882398+0000", 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882400+0000", 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.216e-06 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882405+0000", 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1290000000000004e-06 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882409+0000", 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2080000000000002e-06 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902510+0000", 2024-09-13T07:02:08.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020100129000000001 2024-09-13T07:02:08.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902544+0000", 2024-09-13T07:02:08.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4190000000000003e-05 2024-09-13T07:02:08.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902600+0000", 2024-09-13T07:02:08.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6428999999999998e-05 2024-09-13T07:02:08.593 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.593 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.593 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.593 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.593 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.593 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:08.594 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:08.595 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:08.593+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:08.695 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:08.694+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:08.711 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:08.711 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:08.711 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:08.711 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:08.721 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:08.796 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:08.793+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:08.828 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:08.828 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:08.838 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:08.870 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:02:08.917 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:08.917+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:08.951 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:08.952 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:08.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:08.952 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:08.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.952 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1174 3.13 3:cb2ed8dc:::rbd_data.11943a018f34.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 6419344~27893] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:08.952 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:02:08.952 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.065466921000002, 2024-09-13T07:02:08.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041813481999999999, 2024-09-13T07:02:08.952 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.952 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.952 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.953 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1174 2024-09-13T07:02:08.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.953 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:02:08.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:02:08.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883795+0000", 2024-09-13T07:02:08.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:08.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894234+0000", 2024-09-13T07:02:08.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010439133 2024-09-13T07:02:08.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894276+0000", 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429000000000003e-05 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894296+0000", 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9279999999999998e-05 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894477+0000", 2024-09-13T07:02:08.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018160600000000001 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894889+0000", 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041202399999999998 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.898212+0000", 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033228400000000001 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:08.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.922541+0000", 2024-09-13T07:02:08.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024329052 2024-09-13T07:02:08.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:08.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925458+0000", 2024-09-13T07:02:08.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002916891 2024-09-13T07:02:08.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:08.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925531+0000", 2024-09-13T07:02:08.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2781999999999995e-05 2024-09-13T07:02:08.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925610+0000", 2024-09-13T07:02:08.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9355000000000002e-05 2024-09-13T07:02:08.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.958 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.958 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:02:08.958 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:08.958 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.895091120000004, 2024-09-13T07:02:08.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:02:08.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:08.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:02:08.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:02:08.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:02:08.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:02:08.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:02:08.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:02:08.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:02:08.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:02:08.963 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.963 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.963 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.963 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1535 3.12 e27/22 3:4d1f9422:::rbd_object_map.11943a018f34:head v 27'27, mlcod=27'25)", 2024-09-13T07:02:08.963 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.881134289000002, 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027873574000000002, 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:02:08.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068128+0000", 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068140+0000", 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2279000000000001e-05 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068175+0000", 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4872000000000001e-05 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068193+0000", 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7540000000000001e-05 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068302+0000", 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010928 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068351+0000", 2024-09-13T07:02:08.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036000000000003e-05 2024-09-13T07:02:08.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:08.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.095950+0000", 2024-09-13T07:02:08.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027598988000000001 2024-09-13T07:02:08.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.096003+0000", 2024-09-13T07:02:08.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2722000000000001e-05 2024-09-13T07:02:08.967 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19516 3.11 3:8f71fce3:::rbd_header.119437d9d50d:head [watch reconnect cookie 140335576756256 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.657585+0000", 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.291678177, 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028756766999999999, 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.968 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19516 2024-09-13T07:02:08.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.969 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657585+0000", 2024-09-13T07:02:08.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657585+0000", 2024-09-13T07:02:08.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657584+0000", 2024-09-13T07:02:08.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:08.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657592+0000", 2024-09-13T07:02:08.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5599999999999996e-06 2024-09-13T07:02:08.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657605+0000", 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2866e-05 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657614+0000", 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1050000000000001e-06 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657870+0000", 2024-09-13T07:02:08.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000256331 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657958+0000", 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8077000000000004e-05 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.686342+0000", 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028384063000000001 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:48357 3.9 e58/42 3:953ff28c:::rbd_data.1194fc792839.0000000000000000:head v 58'53, mlcod=57'51)", 2024-09-13T07:02:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.106275+0000", 2024-09-13T07:02:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.842988132, 2024-09-13T07:02:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030129890999999999, 2024-09-13T07:02:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.106275+0000", 2024-09-13T07:02:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.106275+0000", 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.106273+0000", 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.113942+0000", 2024-09-13T07:02:08.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0076687910000000003 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.113969+0000", 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7078999999999999e-05 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.113979+0000", 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7319999999999993e-06 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.114033+0000", 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3692e-05 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.114062+0000", 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9094e-05 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.136272+0000", 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022210189000000002 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.136405+0000", 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013353 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.737209886, 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:02:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:02:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:02:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:02:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:02:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:02:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:02:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.711440175, 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:02:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:02:08.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:02:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:02:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:02:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:02:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:02:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:02:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:02:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:02:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.690289009000001, 2024-09-13T07:02:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:02:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:02:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:02:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48377 3.d 3:b0571a54:::rbd_data.1194fc792839.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13654590~2926018] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.668392725, 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037131338, 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48377 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280868+0000", 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289338+0000", 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084696370000000003 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289362+0000", 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4263000000000001e-05 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289374+0000", 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1209999999999999e-05 2024-09-13T07:02:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289460+0000", 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6883000000000001e-05 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289603+0000", 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014259299999999999 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.290036+0000", 2024-09-13T07:02:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000432545 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317645+0000", 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027609584 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317836+0000", 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190645 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317925+0000", 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8732000000000004e-05 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.318002+0000", 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7618000000000001e-05 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63292 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2419712~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152751+0000", 2024-09-13T07:02:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.79651293400000001, 2024-09-13T07:02:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025379121000000001, 2024-09-13T07:02:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63292 2024-09-13T07:02:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152751+0000", 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152751+0000", 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152750+0000", 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152758+0000", 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7780000000000007e-06 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152768+0000", 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0293e-05 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152774+0000", 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2209999999999996e-06 2024-09-13T07:02:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178038+0000", 2024-09-13T07:02:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025264063 2024-09-13T07:02:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178077+0000", 2024-09-13T07:02:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9397999999999998e-05 2024-09-13T07:02:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178130+0000", 2024-09-13T07:02:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2793000000000001e-05 2024-09-13T07:02:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63293 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2423808~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152778+0000", 2024-09-13T07:02:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.79648556699999995, 2024-09-13T07:02:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025611754, 2024-09-13T07:02:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63293 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152778+0000", 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152778+0000", 2024-09-13T07:02:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152778+0000", 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152780+0000", 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.79e-06 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152789+0000", 2024-09-13T07:02:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0159999999999997e-06 2024-09-13T07:02:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152794+0000", 2024-09-13T07:02:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8169999999999998e-06 2024-09-13T07:02:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178169+0000", 2024-09-13T07:02:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025375147000000001 2024-09-13T07:02:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178268+0000", 2024-09-13T07:02:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8894999999999994e-05 2024-09-13T07:02:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178390+0000", 2024-09-13T07:02:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121463 2024-09-13T07:02:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63294 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2427904~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152800+0000", 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.79646340999999998, 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025693872, 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63294 2024-09-13T07:02:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152800+0000", 2024-09-13T07:02:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152800+0000", 2024-09-13T07:02:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152800+0000", 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152803+0000", 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.858e-06 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152812+0000", 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5410000000000006e-06 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152817+0000", 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1290000000000004e-06 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178406+0000", 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025588507999999999 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178446+0000", 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9950999999999997e-05 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178494+0000", 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.829e-05 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63295 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2432000~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152822+0000", 2024-09-13T07:02:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.79644184600000001, 2024-09-13T07:02:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025934399, 2024-09-13T07:02:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63295 2024-09-13T07:02:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152822+0000", 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152822+0000", 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152821+0000", 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152824+0000", 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.926e-06 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152833+0000", 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1889999999999996e-06 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152839+0000", 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.096e-06 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178534+0000", 2024-09-13T07:02:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025695286000000001 2024-09-13T07:02:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178632+0000", 2024-09-13T07:02:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8189000000000005e-05 2024-09-13T07:02:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178756+0000", 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012413800000000001 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63296 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2436096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152843+0000", 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.79642055099999998, 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026017974999999999, 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63296 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152843+0000", 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152843+0000", 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152843+0000", 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152845+0000", 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.835e-06 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152854+0000", 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0210000000000005e-06 2024-09-13T07:02:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152859+0000", 2024-09-13T07:02:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8849999999999998e-06 2024-09-13T07:02:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178771+0000", 2024-09-13T07:02:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025911548999999999 2024-09-13T07:02:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178810+0000", 2024-09-13T07:02:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8943000000000003e-05 2024-09-13T07:02:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178861+0000", 2024-09-13T07:02:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1146000000000001e-05 2024-09-13T07:02:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63297 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2440192~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152864+0000", 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.79639983800000003, 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026260716999999999, 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63297 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152864+0000", 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152864+0000", 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152863+0000", 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152866+0000", 2024-09-13T07:02:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.802e-06 2024-09-13T07:02:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152875+0000", 2024-09-13T07:02:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2299999999999997e-06 2024-09-13T07:02:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152881+0000", 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1669999999999998e-06 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178902+0000", 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026021183999999999 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179002+0000", 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010067499999999999 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179124+0000", 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012207299999999999 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63298 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2444288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152885+0000", 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.79637841799999998, 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026342638000000002, 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63298 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152885+0000", 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152885+0000", 2024-09-13T07:02:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152885+0000", 2024-09-13T07:02:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152888+0000", 2024-09-13T07:02:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:02:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152897+0000", 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2750000000000001e-06 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152902+0000", 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179139+0000", 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026237176000000001 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179177+0000", 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8064e-05 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179228+0000", 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0624000000000002e-05 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63299 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2448384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152907+0000", 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.79635690800000003, 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026577495999999999, 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63299 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152907+0000", 2024-09-13T07:02:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152907+0000", 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152906+0000", 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152909+0000", 2024-09-13T07:02:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8660000000000002e-06 2024-09-13T07:02:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152918+0000", 2024-09-13T07:02:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7430000000000008e-06 2024-09-13T07:02:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152923+0000", 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8350000000000003e-06 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179267+0000", 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026344346000000001 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179364+0000", 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6932e-05 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179484+0000", 2024-09-13T07:02:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012012599999999999 2024-09-13T07:02:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63300 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2452480~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152928+0000", 2024-09-13T07:02:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.79633545500000003, 2024-09-13T07:02:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026660204999999999, 2024-09-13T07:02:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63300 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152928+0000", 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152928+0000", 2024-09-13T07:02:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152928+0000", 2024-09-13T07:02:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152930+0000", 2024-09-13T07:02:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5969999999999999e-06 2024-09-13T07:02:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152939+0000", 2024-09-13T07:02:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3760000000000008e-06 2024-09-13T07:02:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152943+0000", 2024-09-13T07:02:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6310000000000002e-06 2024-09-13T07:02:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179499+0000", 2024-09-13T07:02:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026555999 2024-09-13T07:02:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179539+0000", 2024-09-13T07:02:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9303e-05 2024-09-13T07:02:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179588+0000", 2024-09-13T07:02:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9706000000000001e-05 2024-09-13T07:02:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63301 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2456576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152947+0000", 2024-09-13T07:02:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.796316582, 2024-09-13T07:02:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026892441, 2024-09-13T07:02:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63301 2024-09-13T07:02:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152947+0000", 2024-09-13T07:02:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152947+0000", 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152947+0000", 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152949+0000", 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1730000000000002e-06 2024-09-13T07:02:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152957+0000", 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0779999999999996e-06 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152961+0000", 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2899999999999996e-06 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179616+0000", 2024-09-13T07:02:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026654589999999999 2024-09-13T07:02:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179715+0000", 2024-09-13T07:02:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9557999999999998e-05 2024-09-13T07:02:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179839+0000", 2024-09-13T07:02:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012407100000000001 2024-09-13T07:02:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.57324424600000001, 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:02:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:02:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:02:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:02:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:02:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:09.055 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:09.056 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:09.056 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:09.056 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:09.053+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:09.057 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:09.059 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:09.078 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:09.101 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:09.124 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:09.154 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:09.157 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:09.153+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:09.162 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:09.163 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:09.172 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:09.174 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:09.177 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:09.177 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:09.177 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:09.177 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:09.177 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:09.177 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:09.177 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:09.177 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:09.221 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:02:09.228 INFO:teuthology.orchestra.run.smithi118.stdout:1839 read 0x9dfc219 thru 0x9e051ac (0x8f94 bytes) 2024-09-13T07:02:09.231 INFO:teuthology.orchestra.run.smithi118.stdout:1841 writesame 0x1c3b51e thru 0x1c42119 data_size 0x2(0x6bfc bytes) 2024-09-13T07:02:09.277 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:09.277+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:09.283 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:09.283 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:09.293 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:09.296 INFO:teuthology.orchestra.run.smithi118.stdout:1842 read 0x5d5d1cd thru 0x5d61c69 (0x4a9d bytes) 2024-09-13T07:02:09.296 INFO:teuthology.orchestra.run.smithi118.stdout:1843 write 0x60fa4ac thru 0x610900c (0xeb61 bytes) 2024-09-13T07:02:09.316 INFO:teuthology.orchestra.run.smithi118.stdout:1844 write 0xb4be00 thru 0xb5104c (0x524d bytes) 2024-09-13T07:02:09.358 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:09.354+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:09.415 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:09.415 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:09.426 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:09.426 INFO:teuthology.orchestra.run.smithi118.stdout:1845 read 0x3eb0760 thru 0x3eb7835 (0x70d6 bytes) 2024-09-13T07:02:09.448 INFO:teuthology.orchestra.run.smithi118.stdout:1846 trunc from 0xd9bbd00 to 0x533c4e 2024-09-13T07:02:09.459 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:09.457+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:09.542 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:09.542 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:09.561 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:09.557+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:09.564 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:09.572 INFO:teuthology.orchestra.run.smithi118.stdout:1847 writesame 0x94a19c6 thru 0x94a83f8 data_size 0x1(0x6a33 bytes) 2024-09-13T07:02:09.586 INFO:teuthology.orchestra.run.smithi118.stdout:1848 trunc from 0x94a83f9 to 0x1ffefea 2024-09-13T07:02:09.599 INFO:teuthology.orchestra.run.smithi118.stdout:1850 punch from 0x6e8436 to 0x6ef0d1, (0x6c9b bytes) 2024-09-13T07:02:09.599 INFO:teuthology.orchestra.run.smithi118.stdout:1852 punch from 0xb7a24e to 0xb7c456, (0x2208 bytes) 2024-09-13T07:02:09.599 INFO:teuthology.orchestra.run.smithi118.stdout:1853 read 0x1b05a61 thru 0x1b0ea77 (0x9017 bytes) 2024-09-13T07:02:09.618 INFO:teuthology.orchestra.run.smithi118.stdout:1854 trunc from 0x1ffefea to 0xd754b67 2024-09-13T07:02:09.631 INFO:teuthology.orchestra.run.smithi118.stdout:1855 write 0x283f0df thru 0x283f893 (0x7b5 bytes) 2024-09-13T07:02:09.646 INFO:teuthology.orchestra.run.smithi118.stdout:1856 writesame 0x1a610a3 thru 0x1a6e1a2 data_size 0x4c(0xd100 bytes) 2024-09-13T07:02:09.658 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:09.658 INFO:teuthology.orchestra.run.smithi118.stdout:1860 write 0x465e242 thru 0x4667579 (0x9338 bytes) 2024-09-13T07:02:09.663 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:09.662+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:09.671 INFO:teuthology.orchestra.run.smithi118.stdout:1862 writesame 0x5242687 thru 0x5246f90 data_size 0x1(0x490a bytes) 2024-09-13T07:02:09.686 INFO:teuthology.orchestra.run.smithi118.stdout:1863 writesame 0x40dee0a thru 0x40e3ced data_size 0x6c(0x4ee4 bytes) 2024-09-13T07:02:09.687 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:09.687 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:09.698 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:09.705 INFO:teuthology.orchestra.run.smithi118.stdout:1864 trunc from 0xd754b67 to 0xa8985ad 2024-09-13T07:02:09.715 INFO:teuthology.orchestra.run.smithi118.stdout:1865 write 0xa108796 thru 0xa10e8a8 (0x6113 bytes) 2024-09-13T07:02:09.728 INFO:teuthology.orchestra.run.smithi118.stdout:1866 write 0x8b27e84 thru 0x8b28e20 (0xf9d bytes) 2024-09-13T07:02:09.738 INFO:teuthology.orchestra.run.smithi118.stdout:1870 trunc from 0xa8985ad to 0xb397122 2024-09-13T07:02:09.747 INFO:teuthology.orchestra.run.smithi118.stdout:1871 punch from 0x5d0d76d to 0x5d14706, (0x6f99 bytes) 2024-09-13T07:02:09.748 INFO:teuthology.orchestra.run.smithi118.stdout:1872 write 0x4086e41 thru 0x4096958 (0xfb18 bytes) 2024-09-13T07:02:09.751 INFO:teuthology.orchestra.run.smithi118.stdout:1874 write 0xa8bb447 thru 0xa8ca836 (0xf3f0 bytes) 2024-09-13T07:02:09.762 INFO:teuthology.orchestra.run.smithi118.stdout:1875 read 0x4cf1988 thru 0x4cffd4d (0xe3c6 bytes) 2024-09-13T07:02:09.762 INFO:teuthology.orchestra.run.smithi118.stdout:1876 writesame 0x164d418 thru 0x1653792 data_size 0xd(0x637b bytes) 2024-09-13T07:02:09.762 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:09.758+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:09.771 INFO:teuthology.orchestra.run.smithi118.stdout:1878 punch from 0xa35a26f to 0xa36331a, (0x90ab bytes) 2024-09-13T07:02:09.771 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:09.772 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:09.778 INFO:teuthology.orchestra.run.smithi118.stdout:1881 trunc from 0xb397122 to 0xdfca6b9 2024-09-13T07:02:09.788 INFO:teuthology.orchestra.run.smithi118.stdout:1882 read 0xbbd5ebf thru 0xbbe5315 (0xf457 bytes) 2024-09-13T07:02:09.789 INFO:teuthology.orchestra.run.smithi118.stdout:1884 write 0x67e2117 thru 0x67f1e4a (0xfd34 bytes) 2024-09-13T07:02:09.800 INFO:teuthology.orchestra.run.smithi118.stdout:1885 write 0x73d87f6 thru 0x73e4fb5 (0xc7c0 bytes) 2024-09-13T07:02:09.809 INFO:teuthology.orchestra.run.smithi118.stdout:1886 read 0xb365be5 thru 0xb373b17 (0xdf33 bytes) 2024-09-13T07:02:09.809 INFO:teuthology.orchestra.run.smithi118.stdout:1887 writesame 0x5d911f5 thru 0x5da0b3f data_size 0x15(0xf94b bytes) 2024-09-13T07:02:09.816 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:09.816 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:09.819 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:09.819 INFO:teuthology.orchestra.run.smithi118.stdout:1889 write 0x772d2d0 thru 0x773491a (0x764b bytes) 2024-09-13T07:02:09.826 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:09.828 INFO:teuthology.orchestra.run.smithi118.stdout:1890 read 0x87a3e80 thru 0x87ad8ba (0x9a3b bytes) 2024-09-13T07:02:09.828 INFO:teuthology.orchestra.run.smithi118.stdout:1891 punch from 0x22c399f to 0x22d2387, (0xe9e8 bytes) 2024-09-13T07:02:09.828 INFO:teuthology.orchestra.run.smithi118.stdout:1895 read 0x4bfea7c thru 0x4c0147b (0x2a00 bytes) 2024-09-13T07:02:09.828 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:09.828 INFO:teuthology.orchestra.run.smithi118.stdout:1897 punch from 0xcae30bb to 0xcaedfb2, (0xaef7 bytes) 2024-09-13T07:02:09.828 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:09.828 INFO:teuthology.orchestra.run.smithi118.stdout:1899 read 0x77bf4d5 thru 0x77c70ce (0x7bfa bytes) 2024-09-13T07:02:09.828 INFO:teuthology.orchestra.run.smithi118.stdout:1900 writesame 0x27bdd0c thru 0x27c252f data_size 0x6c(0x4824 bytes) 2024-09-13T07:02:09.841 INFO:teuthology.orchestra.run.smithi118.stdout:1901 write 0x29aeae8 thru 0x29b7006 (0x851f bytes) 2024-09-13T07:02:09.858 INFO:teuthology.orchestra.run.smithi118.stdout:1902 write 0xd4026a6 thru 0xd40659a (0x3ef5 bytes) 2024-09-13T07:02:09.863 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:09.861+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:09.869 INFO:teuthology.orchestra.run.smithi118.stdout:1903 writesame 0x84f6a09 thru 0x84fa1da data_size 0x5(0x37d2 bytes) 2024-09-13T07:02:09.898 INFO:teuthology.orchestra.run.smithi118.stdout:1905 trunc from 0xdfca6b9 to 0x36721a6 2024-09-13T07:02:09.928 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:09.935 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:09.936 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:09.949 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:09.955 INFO:teuthology.orchestra.run.smithi118.stdout:1907 writesame 0xdeb3057 thru 0xdeb8629 data_size 0x1(0x55d3 bytes) 2024-09-13T07:02:09.955 INFO:teuthology.orchestra.run.smithi118.stdout:1908 read 0x6c73822 thru 0x6c7ec67 (0xb446 bytes) 2024-09-13T07:02:09.955 INFO:teuthology.orchestra.run.smithi118.stdout:1910 writesame 0xbed1943 thru 0xbed2d64 data_size 0x3(0x1422 bytes) 2024-09-13T07:02:09.966 INFO:teuthology.orchestra.run.smithi118.stdout:1911 punch from 0x27bf560 to 0x27c69d4, (0x7474 bytes) 2024-09-13T07:02:09.966 INFO:teuthology.orchestra.run.smithi118.stdout:1912 writesame 0x39f4f28 thru 0x3a0441d data_size 0x5(0xf4f6 bytes) 2024-09-13T07:02:09.966 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:09.965+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:09.983 INFO:teuthology.orchestra.run.smithi118.stdout:1913 write 0x8366232 thru 0x8371bb6 (0xb985 bytes) 2024-09-13T07:02:09.992 INFO:teuthology.orchestra.run.smithi118.stdout:1915 read 0xb238310 thru 0xb2419ac (0x969d bytes) 2024-09-13T07:02:10.012 INFO:teuthology.orchestra.run.smithi118.stdout:1916 trunc from 0xdeb862a to 0x1b3d190 2024-09-13T07:02:10.050 INFO:teuthology.orchestra.run.smithi118.stdout:1917 writesame 0x2a94ef6 thru 0x2a9da6f data_size 0x21(0x8b7a bytes) 2024-09-13T07:02:10.064 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:10.062+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:10.074 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:10.074 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:10.077 INFO:teuthology.orchestra.run.smithi118.stdout:1919 writesame 0xc52c912 thru 0xc536907 data_size 0x4e(0x9ff6 bytes) 2024-09-13T07:02:10.078 INFO:teuthology.orchestra.run.smithi118.stdout:1920 write 0x5daf38 thru 0x5dc46e (0x1537 bytes) 2024-09-13T07:02:10.081 INFO:teuthology.orchestra.run.smithi118.stdout:1921 read 0x9514a1e thru 0x951686d (0x1e50 bytes) 2024-09-13T07:02:10.085 INFO:teuthology.orchestra.run.smithi118.stdout:1922 trunc from 0xc536908 to 0xe752f9f 2024-09-13T07:02:10.087 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:10.096 INFO:teuthology.orchestra.run.smithi118.stdout:1923 read 0xe69474 thru 0xe75b6b (0xc6f8 bytes) 2024-09-13T07:02:10.097 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:10.097 INFO:teuthology.orchestra.run.smithi118.stdout:1926 write 0x64f36fc thru 0x64fd360 (0x9c65 bytes) 2024-09-13T07:02:10.131 INFO:teuthology.orchestra.run.smithi118.stdout:1927 trunc from 0xe752f9f to 0x45dbd0c 2024-09-13T07:02:10.154 INFO:teuthology.orchestra.run.smithi118.stdout:1929 read 0x3faf15b thru 0x3fb4b53 (0x59f9 bytes) 2024-09-13T07:02:10.154 INFO:teuthology.orchestra.run.smithi118.stdout:1930 write 0x195f50c thru 0x19604c5 (0xfba bytes) 2024-09-13T07:02:10.165 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:10.162+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:10.193 INFO:teuthology.orchestra.run.smithi118.stdout:1932 write 0xa882b42 thru 0xa88960a (0x6ac9 bytes) 2024-09-13T07:02:10.203 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:02:10.203 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:02:10.204 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-13T07:02:10.204 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-13T07:02:10.205 INFO:teuthology.orchestra.run.smithi118.stdout:1933 clone 21 order 25 su 2097152 sc 14 2024-09-13T07:02:10.209 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:10.209 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:10.220 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:10.266 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:10.265+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:10.340 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone20 from 0xa88960b (overlap 0x533c4e) to 0x31f0fb 2024-09-13T07:02:10.367 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:10.365+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:10.387 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:10.387 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:10.397 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:10.467 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:10.465+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:10.543 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:10.544 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:10.553 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:10.597 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:10.597+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:10.604 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-13T07:02:10.670 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:10.670 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:10.670 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:10.666+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:10.681 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:10.772 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:10.770+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:10.784 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:10.784 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:10.794 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:10.872 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:10.870+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:10.901 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:10.902 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:10.912 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:10.955 INFO:teuthology.orchestra.run.smithi118.stdout:1934 trunc from 0xa88960b to 0x293ab01 2024-09-13T07:02:10.983 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:10.982+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:11.074 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:11.073+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:11.092 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:11.092 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:11.106 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:11.175 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:11.170+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:11.190 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:02:11.231 INFO:teuthology.orchestra.run.smithi118.stdout:moved 1 / 82 (1.21951%) 2024-09-13T07:02:11.232 INFO:teuthology.orchestra.run.smithi118.stdout:avg 13.6667 2024-09-13T07:02:11.232 INFO:teuthology.orchestra.run.smithi118.stdout:stddev 2.86744 -> 2.98142 (expected baseline 3.37474) 2024-09-13T07:02:11.232 INFO:teuthology.orchestra.run.smithi118.stdout:min osd.6 with 9 -> 9 pgs (0.658537 -> 0.658537 * mean) 2024-09-13T07:02:11.232 INFO:teuthology.orchestra.run.smithi118.stdout:max osd.7 with 18 -> 18 pgs (1.31707 -> 1.31707 * mean) 2024-09-13T07:02:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:02:11.232 INFO:teuthology.orchestra.run.smithi118.stdout:oload 110 2024-09-13T07:02:11.232 INFO:teuthology.orchestra.run.smithi118.stdout:max_change 0.05 2024-09-13T07:02:11.232 INFO:teuthology.orchestra.run.smithi118.stdout:max_change_osds 4 2024-09-13T07:02:11.232 INFO:teuthology.orchestra.run.smithi118.stdout:average_utilization 0.0005 2024-09-13T07:02:11.232 INFO:teuthology.orchestra.run.smithi118.stdout:overload_utilization 0.0006 2024-09-13T07:02:11.232 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 weight 1.0000 -> 0.9500 2024-09-13T07:02:11.232 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 weight 1.0000 -> 0.9500 2024-09-13T07:02:11.234 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:11.235 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:11.236 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:02:11.245 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:11.254 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-13T07:02:11.254 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-13T07:02:11.301 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:11.301+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:11.346 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:11.355 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:11.355 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:11.365 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:11.370 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:11.376 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:11.374+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:11.392 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:11.411 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:11.439 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:11.461 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:11.473 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:11.473 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:11.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:11.473 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:11.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.474 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1531 3.c 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:11.474 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:11.474 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.421669645000001, 2024-09-13T07:02:11.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063168486999999995, 2024-09-13T07:02:11.474 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.474 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:11.474 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1531 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:11.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049511+0000", 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069234+0000", 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019723920999999998 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069269+0000", 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4446999999999997e-05 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069286+0000", 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7405999999999999e-05 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069370+0000", 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-05 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069511+0000", 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014134299999999999 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070463+0000", 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095167899999999998 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105076+0000", 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034613139000000001 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112562+0000", 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074861429999999998 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112631+0000", 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112680+0000", 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9607000000000001e-05 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.479 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.480 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.480 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7628 3.1f 3:fdf2e50b:::rbd_data.1194c5936dae.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1032192~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:02:11.480 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:11.480 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 36.708743488000003, 2024-09-13T07:02:11.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.630407088, 2024-09-13T07:02:11.480 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.480 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:11.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.480 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7628 2024-09-13T07:02:11.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.481 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:11.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:11.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762435+0000", 2024-09-13T07:02:11.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:02:11.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762543+0000", 2024-09-13T07:02:11.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010850500000000001 2024-09-13T07:02:11.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762597+0000", 2024-09-13T07:02:11.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4262000000000003e-05 2024-09-13T07:02:11.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762626+0000", 2024-09-13T07:02:11.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9176e-05 2024-09-13T07:02:11.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762761+0000", 2024-09-13T07:02:11.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013405599999999999 2024-09-13T07:02:11.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:11.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762776+0000", 2024-09-13T07:02:11.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5679999999999999e-05 2024-09-13T07:02:11.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388492+0000", 2024-09-13T07:02:11.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.62571606300000004 2024-09-13T07:02:11.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for missing object", 2024-09-13T07:02:11.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388614+0000", 2024-09-13T07:02:11.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012126 2024-09-13T07:02:11.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390837+0000", 2024-09-13T07:02:11.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002223806 2024-09-13T07:02:11.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390963+0000", 2024-09-13T07:02:11.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012550300000000001 2024-09-13T07:02:11.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391127+0000", 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163995 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391989+0000", 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086230499999999995 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392662+0000", 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067260799999999999 2024-09-13T07:02:11.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:11.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392728+0000", 2024-09-13T07:02:11.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6501000000000006e-05 2024-09-13T07:02:11.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392845+0000", 2024-09-13T07:02:11.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116665 2024-09-13T07:02:11.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.487 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.488 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15862 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3440640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.488 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.772590+0000", 2024-09-13T07:02:11.488 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.698591551, 2024-09-13T07:02:11.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048639120000000001, 2024-09-13T07:02:11.488 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.488 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.488 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15862 2024-09-13T07:02:11.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.489 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772590+0000", 2024-09-13T07:02:11.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772590+0000", 2024-09-13T07:02:11.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772589+0000", 2024-09-13T07:02:11.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:11.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772592+0000", 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772598+0000", 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.463e-06 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772603+0000", 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5450000000000003e-06 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821064+0000", 2024-09-13T07:02:11.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048461046000000001 2024-09-13T07:02:11.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821137+0000", 2024-09-13T07:02:11.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2507000000000002e-05 2024-09-13T07:02:11.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821229+0000", 2024-09-13T07:02:11.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2082000000000004e-05 2024-09-13T07:02:11.492 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.493 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.493 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.493 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15863 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3465216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.493 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.772608+0000", 2024-09-13T07:02:11.493 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.698573277000001, 2024-09-13T07:02:11.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048769738, 2024-09-13T07:02:11.493 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.493 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.493 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.493 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.493 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15863 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772608+0000", 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772608+0000", 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772607+0000", 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772611+0000", 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3759999999999999e-06 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772616+0000", 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3970000000000002e-06 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.772621+0000", 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8459999999999999e-06 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821253+0000", 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048632149 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821308+0000", 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4639999999999999e-05 2024-09-13T07:02:11.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.821378+0000", 2024-09-13T07:02:11.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9987999999999995e-05 2024-09-13T07:02:11.497 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.497 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.497 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.497 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15870 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3637248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.497 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:02:11.497 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.697284874000001, 2024-09-13T07:02:11.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048771508999999998, 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15870 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773896+0000", 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773899+0000", 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1949999999999999e-06 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773904+0000", 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.356e-06 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773909+0000", 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0849999999999996e-06 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822505+0000", 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048595933000000001 2024-09-13T07:02:11.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822579+0000", 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4276999999999994e-05 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822668+0000", 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8522000000000004e-05 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15871 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.697267904, 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048913976999999997, 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15871 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773913+0000", 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773916+0000", 2024-09-13T07:02:11.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0369999999999999e-06 2024-09-13T07:02:11.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773921+0000", 2024-09-13T07:02:11.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.383e-06 2024-09-13T07:02:11.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.773927+0000", 2024-09-13T07:02:11.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.147e-06 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822692+0000", 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048765583000000001 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822753+0000", 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0637999999999998e-05 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.822827+0000", 2024-09-13T07:02:11.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4630999999999997e-05 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15884 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.776551+0000", 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.694630645, 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048644055999999998, 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15884 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776551+0000", 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776551+0000", 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776550+0000", 2024-09-13T07:02:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776553+0000", 2024-09-13T07:02:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1520000000000001e-06 2024-09-13T07:02:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776559+0000", 2024-09-13T07:02:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4890000000000003e-06 2024-09-13T07:02:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776564+0000", 2024-09-13T07:02:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0390000000000004e-06 2024-09-13T07:02:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825037+0000", 2024-09-13T07:02:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048472701 2024-09-13T07:02:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825109+0000", 2024-09-13T07:02:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1884000000000005e-05 2024-09-13T07:02:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825195+0000", 2024-09-13T07:02:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6187000000000007e-05 2024-09-13T07:02:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15885 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 4005888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.776569+0000", 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.694612640999999, 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048792239000000001, 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15885 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776569+0000", 2024-09-13T07:02:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776569+0000", 2024-09-13T07:02:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776568+0000", 2024-09-13T07:02:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776571+0000", 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0419999999999998e-06 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776577+0000", 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5709999999999997e-06 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.776582+0000", 2024-09-13T07:02:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9540000000000003e-06 2024-09-13T07:02:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825231+0000", 2024-09-13T07:02:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048649031000000002 2024-09-13T07:02:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825288+0000", 2024-09-13T07:02:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7148999999999998e-05 2024-09-13T07:02:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.825361+0000", 2024-09-13T07:02:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3133999999999995e-05 2024-09-13T07:02:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15907 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:02:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.690585231, 2024-09-13T07:02:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048713259000000002, 2024-09-13T07:02:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15907 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:02:11.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780596+0000", 2024-09-13T07:02:11.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:11.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780598+0000", 2024-09-13T07:02:11.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7e-06 2024-09-13T07:02:11.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780604+0000", 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-06 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780609+0000", 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6750000000000001e-06 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829162+0000", 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048553600000000002 2024-09-13T07:02:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829226+0000", 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3547000000000007e-05 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829309+0000", 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3547000000000005e-05 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15908 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.780613+0000", 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.690568318, 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048901932000000002, 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15908 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780613+0000", 2024-09-13T07:02:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780613+0000", 2024-09-13T07:02:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780612+0000", 2024-09-13T07:02:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780615+0000", 2024-09-13T07:02:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0120000000000001e-06 2024-09-13T07:02:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780621+0000", 2024-09-13T07:02:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8320000000000002e-06 2024-09-13T07:02:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.780626+0000", 2024-09-13T07:02:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-13T07:02:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829346+0000", 2024-09-13T07:02:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048720439999999997 2024-09-13T07:02:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829419+0000", 2024-09-13T07:02:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2347000000000004e-05 2024-09-13T07:02:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.829515+0000", 2024-09-13T07:02:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6181000000000006e-05 2024-09-13T07:02:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15928 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:02:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.686537074, 2024-09-13T07:02:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048610352000000003, 2024-09-13T07:02:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15928 2024-09-13T07:02:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:02:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784644+0000", 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784646+0000", 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3479999999999998e-06 2024-09-13T07:02:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784652+0000", 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.485e-06 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784657+0000", 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8189999999999999e-06 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833039+0000", 2024-09-13T07:02:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048382890999999997 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833116+0000", 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6371000000000005e-05 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833255+0000", 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138784 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15929 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784661+0000", 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.686520514000001, 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048815364, 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15929 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784661+0000", 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784661+0000", 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784660+0000", 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784663+0000", 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5639999999999999e-06 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784668+0000", 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4079999999999997e-06 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784673+0000", 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5120000000000002e-06 2024-09-13T07:02:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833284+0000", 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048611047999999997 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833382+0000", 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7571999999999993e-05 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833476+0000", 2024-09-13T07:02:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4653000000000003e-05 2024-09-13T07:02:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15930 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784677+0000", 2024-09-13T07:02:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.686504086999999, 2024-09-13T07:02:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048947388000000001, 2024-09-13T07:02:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15930 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784677+0000", 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784677+0000", 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784676+0000", 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784680+0000", 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1930000000000002e-06 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784685+0000", 2024-09-13T07:02:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:02:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784690+0000", 2024-09-13T07:02:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6949999999999999e-06 2024-09-13T07:02:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833496+0000", 2024-09-13T07:02:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048806041000000001 2024-09-13T07:02:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833541+0000", 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4886999999999999e-05 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833625+0000", 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4073000000000006e-05 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15931 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.784704+0000", 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.686477632999999, 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049199247000000002, 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15931 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784704+0000", 2024-09-13T07:02:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784704+0000", 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784703+0000", 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784706+0000", 2024-09-13T07:02:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1e-06 2024-09-13T07:02:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784712+0000", 2024-09-13T07:02:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8730000000000004e-06 2024-09-13T07:02:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.784718+0000", 2024-09-13T07:02:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3850000000000001e-06 2024-09-13T07:02:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833671+0000", 2024-09-13T07:02:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048952993 2024-09-13T07:02:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833787+0000", 2024-09-13T07:02:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011663300000000001 2024-09-13T07:02:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.833903+0000", 2024-09-13T07:02:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115747 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15951 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788591+0000", 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.682590814000001, 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048616683000000001, 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15951 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788591+0000", 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788591+0000", 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788590+0000", 2024-09-13T07:02:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788593+0000", 2024-09-13T07:02:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5720000000000001e-06 2024-09-13T07:02:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788598+0000", 2024-09-13T07:02:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4840000000000003e-06 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788603+0000", 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8659999999999997e-06 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837089+0000", 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048485815000000002 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837143+0000", 2024-09-13T07:02:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3794e-05 2024-09-13T07:02:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837207+0000", 2024-09-13T07:02:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4566999999999994e-05 2024-09-13T07:02:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15952 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788607+0000", 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.682574260999999, 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048772226000000002, 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15952 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788607+0000", 2024-09-13T07:02:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788607+0000", 2024-09-13T07:02:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788606+0000", 2024-09-13T07:02:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788609+0000", 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7640000000000001e-06 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788615+0000", 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8490000000000002e-06 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788620+0000", 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3859999999999998e-06 2024-09-13T07:02:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837240+0000", 2024-09-13T07:02:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048619860000000001 2024-09-13T07:02:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837301+0000", 2024-09-13T07:02:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0304999999999999e-05 2024-09-13T07:02:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837379+0000", 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8689999999999994e-05 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15953 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.682544875000001, 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04888609, 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15953 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788636+0000", 2024-09-13T07:02:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788639+0000", 2024-09-13T07:02:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4740000000000001e-06 2024-09-13T07:02:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788646+0000", 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788652+0000", 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7289999999999997e-06 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837400+0000", 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048747946 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837455+0000", 2024-09-13T07:02:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5621000000000002e-05 2024-09-13T07:02:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837523+0000", 2024-09-13T07:02:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7158999999999999e-05 2024-09-13T07:02:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:15954 3.16 3:6f384c01:::rbd_data.1194257bfc65.0000000000000002:10 [read 1507328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-13T07:02:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:02:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.682525097999999, 2024-09-13T07:02:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049039567999999999, 2024-09-13T07:02:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15954 2024-09-13T07:02:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788656+0000", 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788659+0000", 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1889999999999999e-06 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788664+0000", 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.788670+0000", 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6999999999999996e-06 2024-09-13T07:02:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837552+0000", 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048882402999999998 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837615+0000", 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2354000000000006e-05 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:48.837696+0000", 2024-09-13T07:02:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1286999999999996e-05 2024-09-13T07:02:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:02:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.352593178999999, 2024-09-13T07:02:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:02:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:02:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:02:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:02:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:02:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:02:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:02:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:02:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:02:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:02:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:02:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:02:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:02:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:02:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:02:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:02:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:02:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:02:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:02:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:02:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48376 3.c 3:350dc8a4:::rbd_data.1194fc792839.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.241972336, 2024-09-13T07:02:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.068448320000000007, 2024-09-13T07:02:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48376 2024-09-13T07:02:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229207+0000", 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.241745+0000", 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012537726000000001 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244174+0000", 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024284760000000002 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244198+0000", 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4629000000000001e-05 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244312+0000", 2024-09-13T07:02:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114282 2024-09-13T07:02:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244679+0000", 2024-09-13T07:02:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000366399 2024-09-13T07:02:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.246046+0000", 2024-09-13T07:02:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013672479999999999 2024-09-13T07:02:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.266272+0000", 2024-09-13T07:02:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020226381000000002 2024-09-13T07:02:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297510+0000", 2024-09-13T07:02:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031237576999999999 2024-09-13T07:02:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297595+0000", 2024-09-13T07:02:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5413000000000004e-05 2024-09-13T07:02:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297657+0000", 2024-09-13T07:02:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1860999999999996e-05 2024-09-13T07:02:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:11.575 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:11.576 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:11.576 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:11.577 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:11.578 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:11.574+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:11.578 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:11.578 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:11.578 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:11.578 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:11.578 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:11.579 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:11.579 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:11.579 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:11.677 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:11.677+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:11.736 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:11.736 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:11.736 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:11.736 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:11.747 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:11.778 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:11.774+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:11.850 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:11.850 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:11.850 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:11.850 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:11.851 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:11.860 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:11.878 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:11.877+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:11.979 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:11.977+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:12.015 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.954697733000003, 2024-09-13T07:02:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:02:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:02:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:02:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:02:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:02:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:02:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:02:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:02:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1534 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14365141] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.947870182000003, 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040478761000000002, 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1534 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065747+0000", 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075258+0000", 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095108710000000006 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075288+0000", 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0085000000000001e-05 2024-09-13T07:02:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075305+0000", 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7748e-05 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075367+0000", 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1867999999999998e-05 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075520+0000", 2024-09-13T07:02:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015298800000000001 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.076247+0000", 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072720499999999997 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105157+0000", 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028909642999999999 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106124+0000", 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096729400000000005 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106173+0000", 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8196999999999999e-05 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106227+0000", 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4388999999999997e-05 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61282 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3599360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.881956+0000", 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1316626169999999, 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018514481999999999, 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61282 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881956+0000", 2024-09-13T07:02:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881956+0000", 2024-09-13T07:02:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881955+0000", 2024-09-13T07:02:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881959+0000", 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5870000000000001e-06 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881964+0000", 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.682e-06 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881969+0000", 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5050000000000001e-06 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900353+0000", 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018384461000000001 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900421+0000", 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7764999999999999e-05 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900470+0000", 2024-09-13T07:02:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9162999999999997e-05 2024-09-13T07:02:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61283 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3603456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.881973+0000", 2024-09-13T07:02:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1316455970000003, 2024-09-13T07:02:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018634186, 2024-09-13T07:02:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61283 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881973+0000", 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881973+0000", 2024-09-13T07:02:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881972+0000", 2024-09-13T07:02:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881975+0000", 2024-09-13T07:02:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0150000000000002e-06 2024-09-13T07:02:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881979+0000", 2024-09-13T07:02:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8359999999999999e-06 2024-09-13T07:02:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881984+0000", 2024-09-13T07:02:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2719999999999999e-06 2024-09-13T07:02:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900500+0000", 2024-09-13T07:02:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018516391 2024-09-13T07:02:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900558+0000", 2024-09-13T07:02:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7751999999999999e-05 2024-09-13T07:02:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900607+0000", 2024-09-13T07:02:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9237999999999999e-05 2024-09-13T07:02:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61284 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3607552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.881988+0000", 2024-09-13T07:02:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1316308560000001, 2024-09-13T07:02:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018734458999999998, 2024-09-13T07:02:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61284 2024-09-13T07:02:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881988+0000", 2024-09-13T07:02:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881988+0000", 2024-09-13T07:02:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881987+0000", 2024-09-13T07:02:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881989+0000", 2024-09-13T07:02:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3539999999999998e-06 2024-09-13T07:02:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881994+0000", 2024-09-13T07:02:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9590000000000003e-06 2024-09-13T07:02:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.881999+0000", 2024-09-13T07:02:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4689999999999999e-06 2024-09-13T07:02:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900631+0000", 2024-09-13T07:02:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018632602000000002 2024-09-13T07:02:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900670+0000", 2024-09-13T07:02:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8195000000000003e-05 2024-09-13T07:02:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900722+0000", 2024-09-13T07:02:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2454999999999998e-05 2024-09-13T07:02:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61285 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3611648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882002+0000", 2024-09-13T07:02:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1316160369999997, 2024-09-13T07:02:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018829511, 2024-09-13T07:02:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61285 2024-09-13T07:02:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882002+0000", 2024-09-13T07:02:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882002+0000", 2024-09-13T07:02:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882002+0000", 2024-09-13T07:02:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882115+0000", 2024-09-13T07:02:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112694 2024-09-13T07:02:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882123+0000", 2024-09-13T07:02:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5310000000000001e-06 2024-09-13T07:02:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882129+0000", 2024-09-13T07:02:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1430000000000003e-06 2024-09-13T07:02:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900752+0000", 2024-09-13T07:02:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018622343999999999 2024-09-13T07:02:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900789+0000", 2024-09-13T07:02:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7113999999999997e-05 2024-09-13T07:02:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900832+0000", 2024-09-13T07:02:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3056999999999998e-05 2024-09-13T07:02:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61286 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3615744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882135+0000", 2024-09-13T07:02:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1314833819999999, 2024-09-13T07:02:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018822186000000001, 2024-09-13T07:02:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61286 2024-09-13T07:02:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882135+0000", 2024-09-13T07:02:12.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882135+0000", 2024-09-13T07:02:12.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882134+0000", 2024-09-13T07:02:12.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882137+0000", 2024-09-13T07:02:12.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8870000000000001e-06 2024-09-13T07:02:12.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882144+0000", 2024-09-13T07:02:12.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3339999999999998e-06 2024-09-13T07:02:12.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882148+0000", 2024-09-13T07:02:12.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3159999999999998e-06 2024-09-13T07:02:12.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900863+0000", 2024-09-13T07:02:12.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018715033999999998 2024-09-13T07:02:12.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900920+0000", 2024-09-13T07:02:12.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7466999999999997e-05 2024-09-13T07:02:12.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900957+0000", 2024-09-13T07:02:12.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6757000000000001e-05 2024-09-13T07:02:12.054 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.054 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.054 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.054 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61287 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3619840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.055 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882154+0000", 2024-09-13T07:02:12.055 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1314640779999996, 2024-09-13T07:02:12.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018937841, 2024-09-13T07:02:12.055 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.055 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.055 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.055 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.055 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.055 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61287 2024-09-13T07:02:12.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.055 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882154+0000", 2024-09-13T07:02:12.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882154+0000", 2024-09-13T07:02:12.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882154+0000", 2024-09-13T07:02:12.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882156+0000", 2024-09-13T07:02:12.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-13T07:02:12.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882162+0000", 2024-09-13T07:02:12.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.587e-06 2024-09-13T07:02:12.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882170+0000", 2024-09-13T07:02:12.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3750000000000003e-06 2024-09-13T07:02:12.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.900983+0000", 2024-09-13T07:02:12.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018812969999999998 2024-09-13T07:02:12.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901029+0000", 2024-09-13T07:02:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5952999999999999e-05 2024-09-13T07:02:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901092+0000", 2024-09-13T07:02:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3231000000000005e-05 2024-09-13T07:02:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61288 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3623936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882174+0000", 2024-09-13T07:02:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1314439519999997, 2024-09-13T07:02:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019067711000000001, 2024-09-13T07:02:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61288 2024-09-13T07:02:12.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882174+0000", 2024-09-13T07:02:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882174+0000", 2024-09-13T07:02:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882174+0000", 2024-09-13T07:02:12.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882187+0000", 2024-09-13T07:02:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3475999999999999e-05 2024-09-13T07:02:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882194+0000", 2024-09-13T07:02:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2700000000000001e-06 2024-09-13T07:02:12.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882203+0000", 2024-09-13T07:02:12.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0920000000000004e-06 2024-09-13T07:02:12.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901119+0000", 2024-09-13T07:02:12.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018916540999999999 2024-09-13T07:02:12.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901170+0000", 2024-09-13T07:02:12.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1136e-05 2024-09-13T07:02:12.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901242+0000", 2024-09-13T07:02:12.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1713999999999998e-05 2024-09-13T07:02:12.065 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.066 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.066 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.066 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61289 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3628032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.066 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882208+0000", 2024-09-13T07:02:12.066 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1314104020000002, 2024-09-13T07:02:12.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019174309, 2024-09-13T07:02:12.066 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.066 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.066 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.066 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.067 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.067 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61289 2024-09-13T07:02:12.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.067 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882208+0000", 2024-09-13T07:02:12.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882208+0000", 2024-09-13T07:02:12.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882207+0000", 2024-09-13T07:02:12.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882210+0000", 2024-09-13T07:02:12.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6749999999999998e-06 2024-09-13T07:02:12.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882216+0000", 2024-09-13T07:02:12.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6960000000000002e-06 2024-09-13T07:02:12.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882220+0000", 2024-09-13T07:02:12.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3250000000000001e-06 2024-09-13T07:02:12.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901271+0000", 2024-09-13T07:02:12.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019051097999999999 2024-09-13T07:02:12.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901328+0000", 2024-09-13T07:02:12.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7185999999999998e-05 2024-09-13T07:02:12.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901382+0000", 2024-09-13T07:02:12.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3875999999999997e-05 2024-09-13T07:02:12.071 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.071 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.071 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.071 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61290 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3632128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.071 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882224+0000", 2024-09-13T07:02:12.071 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1313941940000003, 2024-09-13T07:02:12.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019307841999999999, 2024-09-13T07:02:12.072 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.072 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.072 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61290 2024-09-13T07:02:12.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.072 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882224+0000", 2024-09-13T07:02:12.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882224+0000", 2024-09-13T07:02:12.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882224+0000", 2024-09-13T07:02:12.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882226+0000", 2024-09-13T07:02:12.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1490000000000001e-06 2024-09-13T07:02:12.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882231+0000", 2024-09-13T07:02:12.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.344e-06 2024-09-13T07:02:12.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882235+0000", 2024-09-13T07:02:12.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1980000000000003e-06 2024-09-13T07:02:12.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901416+0000", 2024-09-13T07:02:12.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019180127000000002 2024-09-13T07:02:12.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901480+0000", 2024-09-13T07:02:12.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4496e-05 2024-09-13T07:02:12.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901532+0000", 2024-09-13T07:02:12.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1972999999999999e-05 2024-09-13T07:02:12.076 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.076 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.076 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.077 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61291 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3636224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.077 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882239+0000", 2024-09-13T07:02:12.077 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1313792459999998, 2024-09-13T07:02:12.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019420103000000001, 2024-09-13T07:02:12.077 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.077 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.077 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61291 2024-09-13T07:02:12.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.078 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882239+0000", 2024-09-13T07:02:12.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882239+0000", 2024-09-13T07:02:12.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882239+0000", 2024-09-13T07:02:12.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882241+0000", 2024-09-13T07:02:12.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9659999999999999e-06 2024-09-13T07:02:12.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882247+0000", 2024-09-13T07:02:12.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0750000000000003e-06 2024-09-13T07:02:12.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882251+0000", 2024-09-13T07:02:12.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4739999999999999e-06 2024-09-13T07:02:12.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901559+0000", 2024-09-13T07:02:12.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019307656999999999 2024-09-13T07:02:12.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901602+0000", 2024-09-13T07:02:12.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2855000000000002e-05 2024-09-13T07:02:12.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901659+0000", 2024-09-13T07:02:12.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7355999999999998e-05 2024-09-13T07:02:12.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.082 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.082 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61292 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3640320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.082 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882255+0000", 2024-09-13T07:02:12.082 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1313633049999998, 2024-09-13T07:02:12.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019544367999999999, 2024-09-13T07:02:12.082 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.082 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.083 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.083 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.083 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.083 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61292 2024-09-13T07:02:12.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.083 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882255+0000", 2024-09-13T07:02:12.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882255+0000", 2024-09-13T07:02:12.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882255+0000", 2024-09-13T07:02:12.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882257+0000", 2024-09-13T07:02:12.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8819999999999999e-06 2024-09-13T07:02:12.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882262+0000", 2024-09-13T07:02:12.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2619999999999998e-06 2024-09-13T07:02:12.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882266+0000", 2024-09-13T07:02:12.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3290000000000004e-06 2024-09-13T07:02:12.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901685+0000", 2024-09-13T07:02:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019418986999999999 2024-09-13T07:02:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901752+0000", 2024-09-13T07:02:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6927000000000005e-05 2024-09-13T07:02:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901799+0000", 2024-09-13T07:02:12.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7298e-05 2024-09-13T07:02:12.087 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.087 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.087 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.087 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61293 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3644416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.087 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882270+0000", 2024-09-13T07:02:12.087 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.131348332, 2024-09-13T07:02:12.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019653103000000002, 2024-09-13T07:02:12.087 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.088 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.088 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61293 2024-09-13T07:02:12.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.088 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882270+0000", 2024-09-13T07:02:12.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882270+0000", 2024-09-13T07:02:12.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882270+0000", 2024-09-13T07:02:12.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882273+0000", 2024-09-13T07:02:12.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9009999999999998e-06 2024-09-13T07:02:12.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882278+0000", 2024-09-13T07:02:12.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9060000000000001e-06 2024-09-13T07:02:12.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882283+0000", 2024-09-13T07:02:12.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2569999999999998e-06 2024-09-13T07:02:12.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901826+0000", 2024-09-13T07:02:12.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019543362000000002 2024-09-13T07:02:12.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901866+0000", 2024-09-13T07:02:12.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9586999999999999e-05 2024-09-13T07:02:12.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901923+0000", 2024-09-13T07:02:12.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7435000000000002e-05 2024-09-13T07:02:12.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.094 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.094 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61294 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3648512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.094 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882291+0000", 2024-09-13T07:02:12.094 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1313273669999999, 2024-09-13T07:02:12.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019759262999999999, 2024-09-13T07:02:12.094 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.094 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.095 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61294 2024-09-13T07:02:12.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.095 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882291+0000", 2024-09-13T07:02:12.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882291+0000", 2024-09-13T07:02:12.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882290+0000", 2024-09-13T07:02:12.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882293+0000", 2024-09-13T07:02:12.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6730000000000001e-06 2024-09-13T07:02:12.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882299+0000", 2024-09-13T07:02:12.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.976e-06 2024-09-13T07:02:12.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882304+0000", 2024-09-13T07:02:12.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7040000000000002e-06 2024-09-13T07:02:12.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.901950+0000", 2024-09-13T07:02:12.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019646101999999999 2024-09-13T07:02:12.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902001+0000", 2024-09-13T07:02:12.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0853999999999998e-05 2024-09-13T07:02:12.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902050+0000", 2024-09-13T07:02:12.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9540999999999999e-05 2024-09-13T07:02:12.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.099 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.099 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61295 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3652608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.099 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882318+0000", 2024-09-13T07:02:12.099 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1313005540000001, 2024-09-13T07:02:12.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019820681, 2024-09-13T07:02:12.099 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.099 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.100 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61295 2024-09-13T07:02:12.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.100 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882318+0000", 2024-09-13T07:02:12.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882318+0000", 2024-09-13T07:02:12.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882317+0000", 2024-09-13T07:02:12.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882320+0000", 2024-09-13T07:02:12.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5940000000000001e-06 2024-09-13T07:02:12.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882326+0000", 2024-09-13T07:02:12.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.66e-06 2024-09-13T07:02:12.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882336+0000", 2024-09-13T07:02:12.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424e-05 2024-09-13T07:02:12.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902065+0000", 2024-09-13T07:02:12.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01972896 2024-09-13T07:02:12.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902104+0000", 2024-09-13T07:02:12.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9276000000000001e-05 2024-09-13T07:02:12.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902139+0000", 2024-09-13T07:02:12.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4288000000000001e-05 2024-09-13T07:02:12.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.104 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.104 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61296 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3656704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.105 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882340+0000", 2024-09-13T07:02:12.105 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.131278, 2024-09-13T07:02:12.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019912633999999999, 2024-09-13T07:02:12.105 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.105 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.105 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61296 2024-09-13T07:02:12.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.105 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882340+0000", 2024-09-13T07:02:12.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882340+0000", 2024-09-13T07:02:12.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882340+0000", 2024-09-13T07:02:12.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882343+0000", 2024-09-13T07:02:12.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6170000000000001e-06 2024-09-13T07:02:12.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882348+0000", 2024-09-13T07:02:12.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5559999999999998e-06 2024-09-13T07:02:12.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882352+0000", 2024-09-13T07:02:12.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.369e-06 2024-09-13T07:02:12.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902151+0000", 2024-09-13T07:02:12.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019798748000000001 2024-09-13T07:02:12.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902214+0000", 2024-09-13T07:02:12.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3244000000000006e-05 2024-09-13T07:02:12.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902253+0000", 2024-09-13T07:02:12.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8588999999999999e-05 2024-09-13T07:02:12.109 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.109 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.109 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.110 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61297 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3660800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.110 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882356+0000", 2024-09-13T07:02:12.110 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1312623029999997, 2024-09-13T07:02:12.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020016612999999999, 2024-09-13T07:02:12.110 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.110 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.110 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61297 2024-09-13T07:02:12.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.111 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882356+0000", 2024-09-13T07:02:12.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882356+0000", 2024-09-13T07:02:12.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882356+0000", 2024-09-13T07:02:12.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882362+0000", 2024-09-13T07:02:12.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.494e-06 2024-09-13T07:02:12.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882369+0000", 2024-09-13T07:02:12.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2859999999999996e-06 2024-09-13T07:02:12.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882378+0000", 2024-09-13T07:02:12.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3109999999999995e-06 2024-09-13T07:02:12.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902266+0000", 2024-09-13T07:02:12.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019888059999999999 2024-09-13T07:02:12.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902307+0000", 2024-09-13T07:02:12.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1496999999999999e-05 2024-09-13T07:02:12.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902373+0000", 2024-09-13T07:02:12.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5287e-05 2024-09-13T07:02:12.115 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.115 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.115 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.115 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61298 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3664896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.115 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882382+0000", 2024-09-13T07:02:12.115 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1312359690000005, 2024-09-13T07:02:12.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020103925000000002, 2024-09-13T07:02:12.115 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.115 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.115 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.116 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61298 2024-09-13T07:02:12.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.116 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882382+0000", 2024-09-13T07:02:12.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882382+0000", 2024-09-13T07:02:12.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882382+0000", 2024-09-13T07:02:12.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882384+0000", 2024-09-13T07:02:12.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4669999999999998e-06 2024-09-13T07:02:12.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882390+0000", 2024-09-13T07:02:12.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6720000000000001e-06 2024-09-13T07:02:12.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882395+0000", 2024-09-13T07:02:12.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5170000000000001e-06 2024-09-13T07:02:12.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902397+0000", 2024-09-13T07:02:12.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020002457000000001 2024-09-13T07:02:12.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902443+0000", 2024-09-13T07:02:12.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5725e-05 2024-09-13T07:02:12.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902486+0000", 2024-09-13T07:02:12.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3674000000000002e-05 2024-09-13T07:02:12.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.120 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.120 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:61299 3.10 3:0dbf1801:::rbd_data.1194987e4741.000000000000000b:17 [read 3668992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.120 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:07.882398+0000", 2024-09-13T07:02:12.120 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1312200729999997, 2024-09-13T07:02:12.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020201845999999999, 2024-09-13T07:02:12.120 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.120 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.121 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.121 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.121 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.121 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 61299 2024-09-13T07:02:12.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.121 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882398+0000", 2024-09-13T07:02:12.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882398+0000", 2024-09-13T07:02:12.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882398+0000", 2024-09-13T07:02:12.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882400+0000", 2024-09-13T07:02:12.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.216e-06 2024-09-13T07:02:12.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882405+0000", 2024-09-13T07:02:12.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1290000000000004e-06 2024-09-13T07:02:12.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.882409+0000", 2024-09-13T07:02:12.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2080000000000002e-06 2024-09-13T07:02:12.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902510+0000", 2024-09-13T07:02:12.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020100129000000001 2024-09-13T07:02:12.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902544+0000", 2024-09-13T07:02:12.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4190000000000003e-05 2024-09-13T07:02:12.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:07.902600+0000", 2024-09-13T07:02:12.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6428999999999998e-05 2024-09-13T07:02:12.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.125 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.126 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.126 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:12.127 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:12.128 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:12.126+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:12.229 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:12.226+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:12.240 INFO:teuthology.orchestra.run.smithi118.stdout:moved 3 / 82 (3.65854%) 2024-09-13T07:02:12.240 INFO:teuthology.orchestra.run.smithi118.stdout:avg 13.6667 2024-09-13T07:02:12.240 INFO:teuthology.orchestra.run.smithi118.stdout:stddev 2.98142 -> 3.34996 (expected baseline 3.37474) 2024-09-13T07:02:12.240 INFO:teuthology.orchestra.run.smithi118.stdout:min osd.6 with 9 -> 9 pgs (0.658537 -> 0.658537 * mean) 2024-09-13T07:02:12.240 INFO:teuthology.orchestra.run.smithi118.stdout:max osd.7 with 18 -> 20 pgs (1.31707 -> 1.46341 * mean) 2024-09-13T07:02:12.241 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:02:12.241 INFO:teuthology.orchestra.run.smithi118.stdout:oload 110 2024-09-13T07:02:12.241 INFO:teuthology.orchestra.run.smithi118.stdout:max_change 1 2024-09-13T07:02:12.241 INFO:teuthology.orchestra.run.smithi118.stdout:max_change_osds 4 2024-09-13T07:02:12.241 INFO:teuthology.orchestra.run.smithi118.stdout:average_utilization 0.0005 2024-09-13T07:02:12.241 INFO:teuthology.orchestra.run.smithi118.stdout:overload_utilization 0.0006 2024-09-13T07:02:12.241 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 weight 1.0000 -> 0.9017 2024-09-13T07:02:12.241 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 weight 0.9500 -> 0.8596 2024-09-13T07:02:12.246 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:02:12.247 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:12.248 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:12.248 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:12.248 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:12.258 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:12.259 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-13T07:02:12.259 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-13T07:02:12.329 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:12.329+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:12.414 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:12.414 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:12.414 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:12.414 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:12.414 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:12.424 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:12.431 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:12.429+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:12.527 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:12.528 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:12.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:12.528 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:12.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.528 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1174 3.13 3:cb2ed8dc:::rbd_data.11943a018f34.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 6419344~27893] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:12.528 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:02:12.528 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.643093319000002, 2024-09-13T07:02:12.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041813481999999999, 2024-09-13T07:02:12.528 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.528 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:12.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1174 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:02:12.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883795+0000", 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894234+0000", 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010439133 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894276+0000", 2024-09-13T07:02:12.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429000000000003e-05 2024-09-13T07:02:12.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894296+0000", 2024-09-13T07:02:12.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9279999999999998e-05 2024-09-13T07:02:12.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894477+0000", 2024-09-13T07:02:12.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018160600000000001 2024-09-13T07:02:12.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894889+0000", 2024-09-13T07:02:12.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041202399999999998 2024-09-13T07:02:12.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:12.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.898212+0000", 2024-09-13T07:02:12.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033228400000000001 2024-09-13T07:02:12.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:12.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.922541+0000", 2024-09-13T07:02:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024329052 2024-09-13T07:02:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925458+0000", 2024-09-13T07:02:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002916891 2024-09-13T07:02:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925531+0000", 2024-09-13T07:02:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2781999999999995e-05 2024-09-13T07:02:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925610+0000", 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9355000000000002e-05 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.472717518000003, 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:02:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:02:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:02:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:02:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:02:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:02:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:02:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:02:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:02:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:02:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:02:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:02:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:02:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:02:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:02:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1535 3.12 e27/22 3:4d1f9422:::rbd_object_map.11943a018f34:head v 27'27, mlcod=27'25)", 2024-09-13T07:02:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:02:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.458760687000002, 2024-09-13T07:02:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027873574000000002, 2024-09-13T07:02:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:02:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068129+0000", 2024-09-13T07:02:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068128+0000", 2024-09-13T07:02:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068140+0000", 2024-09-13T07:02:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2279000000000001e-05 2024-09-13T07:02:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068175+0000", 2024-09-13T07:02:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4872000000000001e-05 2024-09-13T07:02:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068193+0000", 2024-09-13T07:02:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7540000000000001e-05 2024-09-13T07:02:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068302+0000", 2024-09-13T07:02:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010928 2024-09-13T07:02:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.068351+0000", 2024-09-13T07:02:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036000000000003e-05 2024-09-13T07:02:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.095950+0000", 2024-09-13T07:02:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027598988000000001 2024-09-13T07:02:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.096003+0000", 2024-09-13T07:02:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2722000000000001e-05 2024-09-13T07:02:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19516 3.11 3:8f71fce3:::rbd_header.119437d9d50d:head [watch reconnect cookie 140335576756256 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.657585+0000", 2024-09-13T07:02:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.869304575000001, 2024-09-13T07:02:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028756766999999999, 2024-09-13T07:02:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19516 2024-09-13T07:02:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657585+0000", 2024-09-13T07:02:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657585+0000", 2024-09-13T07:02:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657584+0000", 2024-09-13T07:02:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657592+0000", 2024-09-13T07:02:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5599999999999996e-06 2024-09-13T07:02:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657605+0000", 2024-09-13T07:02:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2866e-05 2024-09-13T07:02:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657614+0000", 2024-09-13T07:02:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1050000000000001e-06 2024-09-13T07:02:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657870+0000", 2024-09-13T07:02:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000256331 2024-09-13T07:02:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.657958+0000", 2024-09-13T07:02:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8077000000000004e-05 2024-09-13T07:02:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.686342+0000", 2024-09-13T07:02:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028384063000000001 2024-09-13T07:02:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:48357 3.9 e58/42 3:953ff28c:::rbd_data.1194fc792839.0000000000000000:head v 58'53, mlcod=57'51)", 2024-09-13T07:02:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.106275+0000", 2024-09-13T07:02:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.42061453, 2024-09-13T07:02:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030129890999999999, 2024-09-13T07:02:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.106275+0000", 2024-09-13T07:02:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.106275+0000", 2024-09-13T07:02:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.106273+0000", 2024-09-13T07:02:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.113942+0000", 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0076687910000000003 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.113969+0000", 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7078999999999999e-05 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.113979+0000", 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7319999999999993e-06 2024-09-13T07:02:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.114033+0000", 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3692e-05 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.114062+0000", 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9094e-05 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.136272+0000", 2024-09-13T07:02:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022210189000000002 2024-09-13T07:02:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.136405+0000", 2024-09-13T07:02:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013353 2024-09-13T07:02:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.314836284, 2024-09-13T07:02:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:02:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:02:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:02:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:02:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:02:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:02:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:02:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:02:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:02:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:02:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:02:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:02:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:02:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:02:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:02:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:02:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:02:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:02:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.289066572999999, 2024-09-13T07:02:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:02:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:02:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:02:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:02:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:02:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:02:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:02:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:02:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:02:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:02:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:02:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:02:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:02:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:02:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:02:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:02:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:02:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.267915407, 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:02:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:02:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:02:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:02:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:02:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:02:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:02:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:02:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:02:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:02:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:02:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48377 3.d 3:b0571a54:::rbd_data.1194fc792839.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13654590~2926018] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.246019123, 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037131338, 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48377 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280868+0000", 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289338+0000", 2024-09-13T07:02:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084696370000000003 2024-09-13T07:02:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289362+0000", 2024-09-13T07:02:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4263000000000001e-05 2024-09-13T07:02:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289374+0000", 2024-09-13T07:02:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1209999999999999e-05 2024-09-13T07:02:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289460+0000", 2024-09-13T07:02:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6883000000000001e-05 2024-09-13T07:02:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289603+0000", 2024-09-13T07:02:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014259299999999999 2024-09-13T07:02:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.290036+0000", 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000432545 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317645+0000", 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027609584 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317836+0000", 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190645 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317925+0000", 2024-09-13T07:02:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8732000000000004e-05 2024-09-13T07:02:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.318002+0000", 2024-09-13T07:02:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7618000000000001e-05 2024-09-13T07:02:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63292 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2419712~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152751+0000", 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3741393320000004, 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025379121000000001, 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63292 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152751+0000", 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152751+0000", 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152750+0000", 2024-09-13T07:02:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152758+0000", 2024-09-13T07:02:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7780000000000007e-06 2024-09-13T07:02:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152768+0000", 2024-09-13T07:02:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0293e-05 2024-09-13T07:02:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152774+0000", 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2209999999999996e-06 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178038+0000", 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025264063 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178077+0000", 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9397999999999998e-05 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178130+0000", 2024-09-13T07:02:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2793000000000001e-05 2024-09-13T07:02:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63293 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2423808~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152778+0000", 2024-09-13T07:02:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.374111965, 2024-09-13T07:02:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025611754, 2024-09-13T07:02:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63293 2024-09-13T07:02:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152778+0000", 2024-09-13T07:02:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152778+0000", 2024-09-13T07:02:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152778+0000", 2024-09-13T07:02:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152780+0000", 2024-09-13T07:02:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.79e-06 2024-09-13T07:02:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152789+0000", 2024-09-13T07:02:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0159999999999997e-06 2024-09-13T07:02:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152794+0000", 2024-09-13T07:02:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8169999999999998e-06 2024-09-13T07:02:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178169+0000", 2024-09-13T07:02:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025375147000000001 2024-09-13T07:02:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178268+0000", 2024-09-13T07:02:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8894999999999994e-05 2024-09-13T07:02:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178390+0000", 2024-09-13T07:02:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121463 2024-09-13T07:02:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63294 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2427904~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152800+0000", 2024-09-13T07:02:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3740898079999999, 2024-09-13T07:02:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025693872, 2024-09-13T07:02:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63294 2024-09-13T07:02:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152800+0000", 2024-09-13T07:02:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152800+0000", 2024-09-13T07:02:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152800+0000", 2024-09-13T07:02:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152803+0000", 2024-09-13T07:02:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.858e-06 2024-09-13T07:02:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152812+0000", 2024-09-13T07:02:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5410000000000006e-06 2024-09-13T07:02:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152817+0000", 2024-09-13T07:02:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1290000000000004e-06 2024-09-13T07:02:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178406+0000", 2024-09-13T07:02:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025588507999999999 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178446+0000", 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9950999999999997e-05 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178494+0000", 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.829e-05 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63295 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2432000~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152822+0000", 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.374068244, 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025934399, 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63295 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152822+0000", 2024-09-13T07:02:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152822+0000", 2024-09-13T07:02:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152821+0000", 2024-09-13T07:02:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152824+0000", 2024-09-13T07:02:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.926e-06 2024-09-13T07:02:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152833+0000", 2024-09-13T07:02:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1889999999999996e-06 2024-09-13T07:02:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152839+0000", 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.096e-06 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178534+0000", 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025695286000000001 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178632+0000", 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8189000000000005e-05 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178756+0000", 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012413800000000001 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63296 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2436096~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152843+0000", 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3740469490000002, 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026017974999999999, 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63296 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152843+0000", 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152843+0000", 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152843+0000", 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152845+0000", 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.835e-06 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152854+0000", 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0210000000000005e-06 2024-09-13T07:02:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152859+0000", 2024-09-13T07:02:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8849999999999998e-06 2024-09-13T07:02:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178771+0000", 2024-09-13T07:02:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025911548999999999 2024-09-13T07:02:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178810+0000", 2024-09-13T07:02:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8943000000000003e-05 2024-09-13T07:02:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178861+0000", 2024-09-13T07:02:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1146000000000001e-05 2024-09-13T07:02:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63297 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2440192~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152864+0000", 2024-09-13T07:02:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3740262359999997, 2024-09-13T07:02:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026260716999999999, 2024-09-13T07:02:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63297 2024-09-13T07:02:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152864+0000", 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152864+0000", 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152863+0000", 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152866+0000", 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.802e-06 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152875+0000", 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2299999999999997e-06 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152881+0000", 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1669999999999998e-06 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.178902+0000", 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026021183999999999 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179002+0000", 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010067499999999999 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179124+0000", 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012207299999999999 2024-09-13T07:02:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63298 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2444288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152885+0000", 2024-09-13T07:02:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3740048160000002, 2024-09-13T07:02:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026342638000000002, 2024-09-13T07:02:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63298 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152885+0000", 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152885+0000", 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152885+0000", 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152888+0000", 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152897+0000", 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2750000000000001e-06 2024-09-13T07:02:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152902+0000", 2024-09-13T07:02:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9420000000000003e-06 2024-09-13T07:02:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179139+0000", 2024-09-13T07:02:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026237176000000001 2024-09-13T07:02:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179177+0000", 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8064e-05 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179228+0000", 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0624000000000002e-05 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63299 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2448384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152907+0000", 2024-09-13T07:02:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3739833060000004, 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026577495999999999, 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63299 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152907+0000", 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152907+0000", 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152906+0000", 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152909+0000", 2024-09-13T07:02:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8660000000000002e-06 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152918+0000", 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7430000000000008e-06 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152923+0000", 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8350000000000003e-06 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179267+0000", 2024-09-13T07:02:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026344346000000001 2024-09-13T07:02:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179364+0000", 2024-09-13T07:02:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6932e-05 2024-09-13T07:02:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179484+0000", 2024-09-13T07:02:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012012599999999999 2024-09-13T07:02:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63300 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2452480~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152928+0000", 2024-09-13T07:02:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.373961853, 2024-09-13T07:02:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026660204999999999, 2024-09-13T07:02:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63300 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152928+0000", 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152928+0000", 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152928+0000", 2024-09-13T07:02:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152930+0000", 2024-09-13T07:02:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5969999999999999e-06 2024-09-13T07:02:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152939+0000", 2024-09-13T07:02:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3760000000000008e-06 2024-09-13T07:02:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152943+0000", 2024-09-13T07:02:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6310000000000002e-06 2024-09-13T07:02:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179499+0000", 2024-09-13T07:02:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026555999 2024-09-13T07:02:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179539+0000", 2024-09-13T07:02:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9303e-05 2024-09-13T07:02:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179588+0000", 2024-09-13T07:02:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9706000000000001e-05 2024-09-13T07:02:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:63301 3.e 3:726f8167:::rbd_data.1194987e4741.0000000000000008:17 [read 2456576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.152947+0000", 2024-09-13T07:02:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3739429799999998, 2024-09-13T07:02:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026892441, 2024-09-13T07:02:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 63301 2024-09-13T07:02:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152947+0000", 2024-09-13T07:02:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152947+0000", 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152947+0000", 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152949+0000", 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1730000000000002e-06 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152957+0000", 2024-09-13T07:02:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0779999999999996e-06 2024-09-13T07:02:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.152961+0000", 2024-09-13T07:02:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2899999999999996e-06 2024-09-13T07:02:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179616+0000", 2024-09-13T07:02:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026654589999999999 2024-09-13T07:02:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179715+0000", 2024-09-13T07:02:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9557999999999998e-05 2024-09-13T07:02:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.179839+0000", 2024-09-13T07:02:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012407100000000001 2024-09-13T07:02:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:02:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1508706440000003, 2024-09-13T07:02:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:02:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:02:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:02:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:02:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:02:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:02:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:02:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:02:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:12.635 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:12.636 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:02:12.605+0000 7f4370da2700 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-13T07:02:12.636 INFO:teuthology.orchestra.run.smithi118.stdout:moved 2 / 82 (2.43902%) 2024-09-13T07:02:12.636 INFO:teuthology.orchestra.run.smithi118.stdout:avg 13.6667 2024-09-13T07:02:12.636 INFO:teuthology.orchestra.run.smithi118.stdout:stddev 3.34996 -> 3.29983 (expected baseline 3.37474) 2024-09-13T07:02:12.636 INFO:teuthology.orchestra.run.smithi118.stdout:min osd.6 with 9 -> 9 pgs (0.658537 -> 0.658537 * mean) 2024-09-13T07:02:12.636 INFO:teuthology.orchestra.run.smithi118.stdout:max osd.7 with 20 -> 20 pgs (1.46341 -> 1.46341 * mean) 2024-09-13T07:02:12.637 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:02:12.637 INFO:teuthology.orchestra.run.smithi118.stdout:oload 110 2024-09-13T07:02:12.637 INFO:teuthology.orchestra.run.smithi118.stdout:max_change 1 2024-09-13T07:02:12.637 INFO:teuthology.orchestra.run.smithi118.stdout:max_change_osds 4 2024-09-13T07:02:12.637 INFO:teuthology.orchestra.run.smithi118.stdout:average_utilization 0.0005 2024-09-13T07:02:12.637 INFO:teuthology.orchestra.run.smithi118.stdout:overload_utilization 0.0006 2024-09-13T07:02:12.637 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 weight 0.9017 -> 0.8132 2024-09-13T07:02:12.637 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 weight 0.8596 -> 0.7778 2024-09-13T07:02:12.638 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:12.634+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:12.638 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:12.639 INFO:teuthology.orchestra.run.smithi118.stderr:no change 2024-09-13T07:02:12.639 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-13T07:02:12.639 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-13T07:02:12.737 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:12.734+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:12.783 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:12.783 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:12.783 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:12.783 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:12.793 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:12.838 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:12.834+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:12.889 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:12.890 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:12.890 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:12.890 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:12.890 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:12.899 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:12.939 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:12.937+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:12.986 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-09-13T07:02:12.985+0000 7f4370da2700 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-13T07:02:12.986 INFO:teuthology.orchestra.run.smithi118.stdout:moved 0 / 82 (0%) 2024-09-13T07:02:12.986 INFO:teuthology.orchestra.run.smithi118.stdout:avg 13.6667 2024-09-13T07:02:12.986 INFO:teuthology.orchestra.run.smithi118.stdout:stddev 3.34996 -> 3.34996 (expected baseline 3.37474) 2024-09-13T07:02:12.986 INFO:teuthology.orchestra.run.smithi118.stdout:min osd.6 with 9 -> 9 pgs (0.658537 -> 0.658537 * mean) 2024-09-13T07:02:12.986 INFO:teuthology.orchestra.run.smithi118.stdout:max osd.7 with 20 -> 20 pgs (1.46341 -> 1.46341 * mean) 2024-09-13T07:02:12.986 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:02:12.987 INFO:teuthology.orchestra.run.smithi118.stdout:oload 110 2024-09-13T07:02:12.987 INFO:teuthology.orchestra.run.smithi118.stdout:max_change 0.05 2024-09-13T07:02:12.987 INFO:teuthology.orchestra.run.smithi118.stdout:max_change_osds 4 2024-09-13T07:02:12.987 INFO:teuthology.orchestra.run.smithi118.stdout:average_utilization 0.0005 2024-09-13T07:02:12.987 INFO:teuthology.orchestra.run.smithi118.stdout:overload_utilization 0.0006 2024-09-13T07:02:12.987 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 weight 0.9017 -> 0.8518 2024-09-13T07:02:12.987 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 weight 0.8596 -> 0.8096 2024-09-13T07:02:12.991 INFO:teuthology.orchestra.run.smithi118.stderr:no change 2024-09-13T07:02:13.001 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:13.001 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:13.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:13.001 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:13.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.001 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:13.002 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:13.002 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.495735515, 2024-09-13T07:02:13.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:02:13.002 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.002 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:13.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.002 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:02:13.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.002 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:13.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:13.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:02:13.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:13.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:02:13.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:02:13.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:02:13.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:02:13.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:02:13.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:02:13.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:02:13.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:02:13.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:02:13.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:02:13.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:13.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:02:13.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:02:13.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:13.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:02:13.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:02:13.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:13.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:02:13.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:02:13.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:13.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:02:13.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:02:13.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:02:13.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:02:13.008 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.008 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.009 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.009 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:13.009 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:13.009 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.470907805000003, 2024-09-13T07:02:13.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:02:13.009 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.009 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:13.009 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.009 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.009 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.010 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:02:13.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.010 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:13.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:13.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:02:13.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:13.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:02:13.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:02:13.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:02:13.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:02:13.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:02:13.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:02:13.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:02:13.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:02:13.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:02:13.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:02:13.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:13.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:02:13.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:02:13.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:13.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:02:13.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:02:13.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:13.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:02:13.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:02:13.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:13.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:02:13.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:02:13.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:02:13.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:02:13.015 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.015 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.015 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.016 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:13.016 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:13.016 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.932235980999998, 2024-09-13T07:02:13.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:02:13.016 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.016 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:13.016 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.016 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.016 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.016 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:02:13.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.017 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:13.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:13.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:02:13.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:13.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:02:13.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:02:13.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:02:13.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:02:13.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:02:13.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:02:13.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:02:13.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:02:13.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:02:13.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:02:13.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:13.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:02:13.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:02:13.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:13.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:02:13.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:02:13.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:13.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:02:13.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:02:13.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:13.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:02:13.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:02:13.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:02:13.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:02:13.022 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.022 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.022 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.022 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:13.022 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:13.023 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.902610787999997, 2024-09-13T07:02:13.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:02:13.023 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.023 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:13.023 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.023 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.023 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.023 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:02:13.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.023 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:13.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:13.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:02:13.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:13.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:02:13.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:02:13.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:02:13.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:02:13.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:02:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:02:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:02:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:02:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:02:13.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:02:13.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:13.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:02:13.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:02:13.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:13.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:02:13.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:02:13.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:13.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:02:13.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:02:13.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:13.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:02:13.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:02:13.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:02:13.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:02:13.029 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.029 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.029 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.029 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:13.029 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:13.030 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.369897156999997, 2024-09-13T07:02:13.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:02:13.030 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.030 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:13.030 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.030 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.030 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.030 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:02:13.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.030 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:13.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:13.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:13.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:13.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:02:13.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:02:13.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:02:13.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:02:13.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:02:13.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:02:13.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:02:13.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:02:13.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:02:13.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:02:13.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:13.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:02:13.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:02:13.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:13.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:02:13.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:02:13.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:13.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:02:13.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:02:13.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:13.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:02:13.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:02:13.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:02:13.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:02:13.036 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.036 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.036 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.036 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:13.036 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:13.037 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.369814701000003, 2024-09-13T07:02:13.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:02:13.037 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.037 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:13.037 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.037 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.037 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.037 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:02:13.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.037 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:13.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:13.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:02:13.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:13.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:02:13.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:02:13.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:02:13.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:02:13.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:02:13.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:02:13.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:02:13.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:02:13.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:02:13.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:02:13.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:13.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:02:13.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:02:13.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:13.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:02:13.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:02:13.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:13.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:02:13.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:02:13.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:13.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:02:13.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:02:13.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:02:13.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:02:13.043 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.043 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.043 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.043 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:13.043 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:13.043 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.369782891, 2024-09-13T07:02:13.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:02:13.044 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.044 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:13.044 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.044 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.044 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.044 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:02:13.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.044 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:13.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:13.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:02:13.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:02:13.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:02:13.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:02:13.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:02:13.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:02:13.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:02:13.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:02:13.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:02:13.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:02:13.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:02:13.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:13.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:02:13.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:02:13.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:13.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:02:13.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:02:13.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:13.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:02:13.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:02:13.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:13.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:02:13.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:02:13.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:02:13.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:02:13.050 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.050 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.050 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.050 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:13.050 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:13.050 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.369727732999998, 2024-09-13T07:02:13.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:02:13.050 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.051 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:13.051 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.051 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.051 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.051 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:02:13.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.051 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:13.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:13.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:02:13.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:13.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:02:13.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:02:13.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:02:13.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:02:13.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:02:13.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:02:13.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:02:13.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:02:13.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:02:13.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:02:13.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:13.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:02:13.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:02:13.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:13.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:02:13.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:02:13.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:13.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:02:13.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:02:13.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:13.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:02:13.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:02:13.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:02:13.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:02:13.057 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.057 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.057 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.057 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:13.057 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:13.057 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.369682453999999, 2024-09-13T07:02:13.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:02:13.057 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.057 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:13.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.058 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.058 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.058 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:02:13.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.058 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:13.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:13.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:02:13.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:13.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:02:13.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:02:13.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:02:13.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:02:13.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:02:13.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:02:13.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:02:13.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:02:13.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:02:13.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:02:13.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:13.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:02:13.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:02:13.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:13.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:02:13.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:02:13.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:13.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:02:13.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:02:13.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:13.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:02:13.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:02:13.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:02:13.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:02:13.063 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.063 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.064 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.064 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:13.064 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:13.064 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.369643920000001, 2024-09-13T07:02:13.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:02:13.064 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.064 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:13.064 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.064 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.064 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.065 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:02:13.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.065 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:13.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:13.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:02:13.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:13.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:02:13.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:02:13.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:02:13.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:02:13.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:02:13.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:02:13.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:02:13.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:02:13.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:02:13.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:02:13.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:13.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:02:13.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:02:13.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:13.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:02:13.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:02:13.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:13.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:02:13.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:02:13.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:13.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:02:13.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:02:13.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:02:13.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:02:13.070 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.070 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.070 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.071 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:13.071 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:13.071 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.369582301000001, 2024-09-13T07:02:13.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:02:13.071 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.071 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:13.071 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.071 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.071 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.071 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:02:13.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.072 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:13.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:13.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:02:13.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:13.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:02:13.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:02:13.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:02:13.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:02:13.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:02:13.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:02:13.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:02:13.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:02:13.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:02:13.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:02:13.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:13.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:02:13.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:02:13.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:13.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:02:13.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:02:13.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:13.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:02:13.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:02:13.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:13.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:02:13.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:02:13.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:02:13.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:02:13.077 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.077 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.077 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.077 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:13.077 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:13.077 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.355359618000001, 2024-09-13T07:02:13.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:02:13.078 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.078 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:13.078 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.078 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.078 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.078 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:02:13.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.078 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:13.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:13.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:02:13.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:13.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:02:13.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:02:13.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:02:13.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:02:13.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:02:13.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:02:13.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:02:13.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:02:13.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:02:13.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:02:13.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:13.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:02:13.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:02:13.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:13.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:02:13.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:02:13.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:13.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:02:13.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:02:13.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:13.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:02:13.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:02:13.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:02:13.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:02:13.085 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.085 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.085 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.085 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:13.085 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:13.086 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.343481097999998, 2024-09-13T07:02:13.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:02:13.086 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.086 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:13.086 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.086 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.086 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.086 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:02:13.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.086 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:13.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:13.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:02:13.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:02:13.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:02:13.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:02:13.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:02:13.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:02:13.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:02:13.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:02:13.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:02:13.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:02:13.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:02:13.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:02:13.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:02:13.090 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.090 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.091 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.091 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:13.091 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:13.091 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.342924037000003, 2024-09-13T07:02:13.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:02:13.091 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.091 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:13.091 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.091 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.091 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.091 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:02:13.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.092 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:13.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:13.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:02:13.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:02:13.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:02:13.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:02:13.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:02:13.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:02:13.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:02:13.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:02:13.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:02:13.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:02:13.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:02:13.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:02:13.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:02:13.096 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.096 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.096 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.096 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:13.096 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:13.096 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.342901882, 2024-09-13T07:02:13.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:02:13.096 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.096 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:13.096 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.096 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.097 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.097 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:02:13.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.097 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:13.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:13.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:02:13.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:02:13.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:02:13.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:02:13.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:02:13.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:02:13.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:02:13.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:02:13.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:02:13.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:02:13.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:02:13.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:02:13.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:02:13.101 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.101 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.101 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.101 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:13.101 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:13.101 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.342882164999999, 2024-09-13T07:02:13.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:02:13.101 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.102 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:13.102 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.102 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.102 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.102 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:02:13.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.102 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:13.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:13.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:13.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:02:13.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:02:13.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:02:13.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:02:13.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:02:13.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:02:13.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:02:13.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:02:13.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:02:13.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:02:13.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:02:13.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:02:13.106 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.106 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.106 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.106 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:13.107 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:13.107 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.342864835, 2024-09-13T07:02:13.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:02:13.107 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.107 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:13.107 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.107 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.107 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.107 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:02:13.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.107 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:13.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:13.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:02:13.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:02:13.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:02:13.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:02:13.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:02:13.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:02:13.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:02:13.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:02:13.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:02:13.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:02:13.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:02:13.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:02:13.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:02:13.111 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.111 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.111 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.112 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:13.112 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:13.112 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.342569173000001, 2024-09-13T07:02:13.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:02:13.112 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.112 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:13.112 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.112 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.112 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.112 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:02:13.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.112 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:13.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:13.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:13.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:02:13.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:02:13.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:02:13.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:02:13.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:02:13.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:02:13.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:02:13.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:02:13.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:02:13.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:02:13.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:02:13.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:02:13.116 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.116 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.117 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.117 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:02:13.117 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:13.117 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.781366353999999, 2024-09-13T07:02:13.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:02:13.117 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.117 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:13.117 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.117 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.117 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.118 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:02:13.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.118 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:13.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:13.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:02:13.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:13.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:02:13.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:02:13.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:02:13.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:02:13.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:02:13.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:02:13.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:02:13.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:02:13.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:02:13.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:02:13.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:13.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:02:13.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:02:13.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:13.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:02:13.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:02:13.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:13.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:02:13.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:02:13.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:13.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:02:13.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:02:13.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:02:13.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:02:13.123 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.123 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.123 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.123 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:02:13.124 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:13.124 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.749287451000001, 2024-09-13T07:02:13.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:02:13.124 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.124 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:13.124 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:13.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:13.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:02:13.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:13.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:02:13.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:02:13.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:02:13.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:02:13.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:02:13.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:02:13.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:02:13.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:02:13.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:02:13.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:02:13.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:02:13.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:02:13.128 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.128 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.128 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.128 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.128 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.128 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:13.130 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-13T07:02:13.130 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-13T07:02:13.131 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:13.131 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:13.129+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:13.230 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:13.226+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:13.292 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:75660 3.8 3.9aa93828 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-13T07:02:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:12.257271+0000", 2024-09-13T07:02:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.0364152680000001, 2024-09-13T07:02:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0364418930000001, 2024-09-13T07:02:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "waiting for peered", 2024-09-13T07:02:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 75660 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:12.257271+0000", 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:12.257271+0000", 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:12.257269+0000", 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:12.257279+0000", 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.009e-05 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:12.257296+0000", 2024-09-13T07:02:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6612e-05 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:12.257304+0000", 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6799999999999999e-06 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:12.257362+0000", 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7398000000000002e-05 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:02:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:12.257374+0000", 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1800000000000001e-05 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:13.249254+0000", 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.99188079799999995 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:13.249280+0000", 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5870000000000001e-05 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 1 2024-09-13T07:02:13.297 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:13.303 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:13.332 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:13.330+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:13.418 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:13.418 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:13.418 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:13.418 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:13.418 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:13.428 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:13.433 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:13.430+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:13.492 INFO:teuthology.orchestra.run.smithi118.stdout:moved 0 / 82 (0%) 2024-09-13T07:02:13.493 INFO:teuthology.orchestra.run.smithi118.stdout:avg 13.6667 2024-09-13T07:02:13.493 INFO:teuthology.orchestra.run.smithi118.stdout:stddev 3.34996 -> 3.34996 (expected baseline 3.37474) 2024-09-13T07:02:13.493 INFO:teuthology.orchestra.run.smithi118.stdout:min osd.6 with 9 -> 9 pgs (0.658537 -> 0.658537 * mean) 2024-09-13T07:02:13.493 INFO:teuthology.orchestra.run.smithi118.stdout:max osd.7 with 20 -> 20 pgs (1.46341 -> 1.46341 * mean) 2024-09-13T07:02:13.493 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:02:13.493 INFO:teuthology.orchestra.run.smithi118.stdout:oload 1000 2024-09-13T07:02:13.493 INFO:teuthology.orchestra.run.smithi118.stdout:max_change 1 2024-09-13T07:02:13.493 INFO:teuthology.orchestra.run.smithi118.stdout:max_change_osds 4 2024-09-13T07:02:13.493 INFO:teuthology.orchestra.run.smithi118.stdout:average_utilization 0.0005 2024-09-13T07:02:13.493 INFO:teuthology.orchestra.run.smithi118.stdout:overload_utilization 0.0053 2024-09-13T07:02:13.496 INFO:teuthology.orchestra.run.smithi118.stderr:no change 2024-09-13T07:02:13.534 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:13.534 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:13.534 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:13.534 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:13.534 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.535 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:02:13.535 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:13.535 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 30.040160085, 2024-09-13T07:02:13.535 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:02:13.535 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.535 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:13.535 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.535 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.535 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.535 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:02:13.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.536 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.537 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.539 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.553676521, 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:02:13.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.541 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.553606592, 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:13.544 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.545 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.545 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.545 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:02:13.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.545 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:13.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.546 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:13.546 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.546 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.546 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:02:13.546 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.546 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.546 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:02:13.546 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:02:13.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:02:13.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.549 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.549 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:13.549 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.553590720000001, 2024-09-13T07:02:13.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:02:13.549 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.549 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:13.549 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.549 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.549 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.549 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:02:13.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.553520262999999, 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:02:13.553 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.554 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:13.554 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.554 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.554 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.554 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:02:13.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.554 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:13.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.555 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.555 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:13.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.555 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.555 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:13.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:02:13.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:02:13.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:02:13.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:02:13.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.557 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:02:13.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:02:13.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.557 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:02:13.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:02:13.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:13.558 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.550168225, 2024-09-13T07:02:13.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:02:13.559 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.559 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.559 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.559 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.559 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.559 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:02:13.559 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.559 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.559 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.559 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:13.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.562 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.550151924, 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:13.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.564 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:13.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.564 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:02:13.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.566 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.550132139, 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.550115866000001, 2024-09-13T07:02:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:02:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:02:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:02:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:02:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:02:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:02:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:02:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:02:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:02:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:02:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:02:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:02:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:02:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.550098041, 2024-09-13T07:02:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:02:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:02:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:02:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:02:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:02:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:02:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:02:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:02:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:02:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:02:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:02:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:02:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.550081948000001, 2024-09-13T07:02:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:02:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:02:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:02:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:02:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:02:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:02:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:02:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:02:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:02:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:02:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:02:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:02:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:02:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.550060761999999, 2024-09-13T07:02:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:02:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:02:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:02:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:02:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:02:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:02:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:02:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:02:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:02:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:02:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:02:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:02:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.550044729, 2024-09-13T07:02:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:02:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:02:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:02:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:02:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.549995490000001, 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:02:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.597 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:13.533+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:02:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.54997743, 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:02:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.549960915, 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:02:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:02:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:02:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:02:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:02:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:02:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:02:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:02:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:02:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.549942550000001, 2024-09-13T07:02:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:02:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:02:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:02:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:02:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:02:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:02:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:02:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:02:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:02:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:02:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:02:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:02:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.549926451999999, 2024-09-13T07:02:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:02:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:02:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:02:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:02:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:02:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:02:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:02:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:02:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:02:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:02:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:02:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:02:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.549910517000001, 2024-09-13T07:02:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:02:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:02:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.54989402, 2024-09-13T07:02:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:02:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:02:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:13.628 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:13.628 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:13.628 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:02:13.635 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:13.634+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:13.736 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:13.734+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:13.749 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:13.749 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:13.759 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:13.783 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:13.804 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:13.823 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:13.837 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:13.837+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:13.843 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:13.863 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:13.863 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:13.865 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:13.874 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:13.889 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:13.912 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:13.935 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:13.937 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:13.937 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:13.937 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:13.938 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:13.938 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:13.938 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:13.938 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:13.938 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:13.938 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:13.937+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:13.996 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:13.996 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:14.005 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:14.039 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:14.037+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:14.114 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:14.114 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:14.123 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:14.140 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:14.137+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:14.232 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:14.232 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:14.242 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:14.242 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:14.241+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:14.261 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:02:14.342 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:14.342+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:14.393 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:14.393 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:14.403 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:14.447 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:14.446+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:14.520 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:14.521 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:14.530 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:14.545 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:14.541+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:14.639 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:14.639 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:14.645 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:14.645+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:14.649 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:14.746 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:14.742+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:14.753 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:14.753 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:14.763 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:14.847 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:14.845+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:14.919 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:14.919 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:14.929 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:14.948 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:14.946+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:15.034 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:15.034 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:15.043 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:15.087 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:15.086+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:15.136 INFO:teuthology.orchestra.run.smithi118.stdout:1935 writesame 0xe0a4310 thru 0xe0a74f7 data_size 0x4(0x31e8 bytes) 2024-09-13T07:02:15.136 INFO:teuthology.orchestra.run.smithi118.stdout:1937 writesame 0x40ca4f7 thru 0x40d6854 data_size 0x11(0xc35e bytes) 2024-09-13T07:02:15.140 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:15.140 INFO:teuthology.orchestra.run.smithi118.stdout:1939 write 0x62e6d14 thru 0x62f3d30 (0xd01d bytes) 2024-09-13T07:02:15.142 INFO:teuthology.orchestra.run.smithi118.stdout:1940 read 0xd28d352 thru 0xd29af72 (0xdc21 bytes) 2024-09-13T07:02:15.143 INFO:teuthology.orchestra.run.smithi118.stdout:1941 read 0x2023578 thru 0x2026a6b (0x34f4 bytes) 2024-09-13T07:02:15.143 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:15.143 INFO:teuthology.orchestra.run.smithi118.stdout:1943 punch from 0x76232e7 to 0x7627cb4, (0x49cd bytes) 2024-09-13T07:02:15.150 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:15.149+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:15.158 INFO:teuthology.orchestra.run.smithi118.stdout:1944 trunc from 0xe0a74f8 to 0x6e58999 2024-09-13T07:02:15.170 INFO:teuthology.orchestra.run.smithi118.stdout:1945 punch from 0x34a8585 to 0x34b4026, (0xbaa1 bytes) 2024-09-13T07:02:15.170 INFO:teuthology.orchestra.run.smithi118.stdout:1946 write 0x6162654 thru 0x616f704 (0xd0b1 bytes) 2024-09-13T07:02:15.183 INFO:teuthology.orchestra.run.smithi118.stdout:1947 writesame 0x85d6ca8 thru 0x85ded69 data_size 0x1(0x80c2 bytes) 2024-09-13T07:02:15.203 INFO:teuthology.orchestra.run.smithi118.stdout:1949 write 0xb98f32e thru 0xb990001 (0xcd4 bytes) 2024-09-13T07:02:15.203 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:15.203 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:15.205 INFO:teuthology.orchestra.run.smithi118.stdout:1950 read 0x5afee60 thru 0x5b0533f (0x64e0 bytes) 2024-09-13T07:02:15.206 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:15.206 INFO:teuthology.orchestra.run.smithi118.stdout:1952 punch from 0xb3e7c83 to 0xb3ee2bc, (0x6639 bytes) 2024-09-13T07:02:15.206 INFO:teuthology.orchestra.run.smithi118.stdout:1953 punch from 0x579b453 to 0x579da8a, (0x2637 bytes) 2024-09-13T07:02:15.206 INFO:teuthology.orchestra.run.smithi118.stdout:1954 punch from 0x9bd739c to 0x9bd9d61, (0x29c5 bytes) 2024-09-13T07:02:15.207 INFO:teuthology.orchestra.run.smithi118.stdout:1955 punch from 0x933df68 to 0x933e172, (0x20a bytes) 2024-09-13T07:02:15.207 INFO:teuthology.orchestra.run.smithi118.stdout:1956 punch from 0xb27cd98 to 0xb28935c, (0xc5c4 bytes) 2024-09-13T07:02:15.220 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:15.220 INFO:teuthology.orchestra.run.smithi118.stdout:1957 write 0xc5badea thru 0xc5c2107 (0x731e bytes) 2024-09-13T07:02:15.222 INFO:teuthology.orchestra.run.smithi118.stdout:1958 read 0x9b9274 thru 0x9c07fb (0x7588 bytes) 2024-09-13T07:02:15.223 INFO:teuthology.orchestra.run.smithi118.stdout:1960 punch from 0x376a2bf to 0x3773fa5, (0x9ce6 bytes) 2024-09-13T07:02:15.223 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:15.223 INFO:teuthology.orchestra.run.smithi118.stdout:1962 write 0x6be445a thru 0x6be5433 (0xfda bytes) 2024-09-13T07:02:15.225 INFO:teuthology.orchestra.run.smithi118.stdout:1966 writesame 0x2ca3b3c thru 0x2ca7f41 data_size 0x1(0x4406 bytes) 2024-09-13T07:02:15.240 INFO:teuthology.orchestra.run.smithi118.stdout:1967 writesame 0xb19b2e3 thru 0xb1a2293 data_size 0x1b(0x6fb1 bytes) 2024-09-13T07:02:15.250 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:15.249+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:15.254 INFO:teuthology.orchestra.run.smithi118.stdout:1968 trunc from 0xc5c2108 to 0x7caf315 2024-09-13T07:02:15.271 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:02:15.333 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:15.333 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:15.343 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:15.350 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:15.349+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:15.447 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:15.447 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:15.451 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:15.449+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:15.457 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:15.553 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:15.550+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:15.559 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:15.559 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:15.569 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:15.654 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:15.650+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:15.728 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:15.755 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:15.753+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:15.830 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:15.830 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:15.841 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:15.856 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:15.853+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:15.940 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:15.941 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:15.949 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:02:15.950 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:15.997 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:15.997+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:16.058 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:16.057+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:16.064 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:16.064 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:16.082 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:16.099 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:16.119 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:16.139 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:16.158 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:16.159 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:16.157+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:16.179 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:16.184 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:16.185 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:16.185 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:16.185 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:16.185 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:16.195 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:16.200 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:16.223 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:16.245 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:16.247 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:16.247 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:16.247 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:16.247 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:16.247 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:16.247 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:16.247 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:16.247 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:16.301 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:16.301+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:16.307 INFO:teuthology.orchestra.run.smithi118.stdout:1969 write 0xdc58cf3 thru 0xdc5f5c1 (0x68cf bytes) 2024-09-13T07:02:16.311 INFO:teuthology.orchestra.run.smithi118.stdout:1970 write 0xb7bafb5 thru 0xb7cae16 (0xfe62 bytes) 2024-09-13T07:02:16.312 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:16.312 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:16.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:16.313 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:16.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.313 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:16.313 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:16.313 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.807864148, 2024-09-13T07:02:16.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:02:16.313 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.313 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:16.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.313 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:02:16.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.314 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:16.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:16.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:02:16.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:02:16.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:02:16.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:02:16.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:02:16.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:02:16.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:16.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:02:16.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:02:16.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:16.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:02:16.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:02:16.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:02:16.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.783036438000003, 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.319 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:02:16.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:02:16.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:16.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:02:16.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:02:16.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:16.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:02:16.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:02:16.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.244364613999998, 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.325 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:02:16.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:16.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:02:16.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:02:16.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:02:16.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:02:16.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:02:16.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.330 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.214739420999997, 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:16.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:16.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:02:16.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:16.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:02:16.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:02:16.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:02:16.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:02:16.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:02:16.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:02:16.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:02:16.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:02:16.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:02:16.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:02:16.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:16.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:02:16.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:02:16.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:16.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:02:16.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:02:16.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:16.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:02:16.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.337 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:16.337 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:16.337 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.682025789999997, 2024-09-13T07:02:16.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:02:16.337 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.337 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:16.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.337 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:02:16.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.337 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:16.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:02:16.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:02:16.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:02:16.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:16.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:02:16.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:02:16.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:16.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:02:16.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:02:16.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:16.342 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.681943334000003, 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:16.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:02:16.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.681911524, 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.348 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:16.349 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.349 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.349 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.349 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:02:16.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.349 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:16.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:02:16.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:02:16.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:02:16.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.681856365999998, 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.354 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:02:16.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.359 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.681811087, 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:16.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.681772553000002, 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:16.365 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:02:16.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:16.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:02:16.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:02:16.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:16.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:02:16.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:02:16.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.681710934000002, 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.371 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.372 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.372 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:02:16.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.372 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:16.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:16.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:02:16.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:02:16.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:02:16.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:02:16.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:02:16.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:16.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.667488251000002, 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:16.377 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:16.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:02:16.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:02:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:02:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:02:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:02:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:02:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:02:16.382 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.655609730999998, 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:02:16.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.655052670000003, 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:16.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:16.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:16.391 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.655030515, 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:02:16.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:02:16.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:02:16.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:02:16.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:02:16.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:02:16.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:02:16.395 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.395 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.395 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.655010797999999, 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.396 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:16.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.654993468000001, 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.400 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:02:16.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:02:16.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.654697806000001, 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:02:16.404 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.405 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:16.405 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.405 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.405 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.405 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:02:16.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.405 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:16.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:02:16.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:02:16.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:02:16.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:02:16.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:02:16.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:02:16.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:02:16.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:02:16.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:02:16.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:02:16.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:02:16.408 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.408 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.093494987, 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:02:16.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.410 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:16.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:16.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:02:16.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:16.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:02:16.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:02:16.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:02:16.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:02:16.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:02:16.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:02:16.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:16.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:02:16.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:02:16.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:16.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:02:16.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:02:16.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.414 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.061416084000001, 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:16.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.419 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.419 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.419 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.419 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:16.420 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:16.422 INFO:teuthology.orchestra.run.smithi118.stdout:1971 trunc from 0xdc5f5c2 to 0x67fed1f 2024-09-13T07:02:16.422 INFO:teuthology.orchestra.run.smithi118.stdout:1972 write 0x9fe473f thru 0x9ff0ea1 (0xc763 bytes) 2024-09-13T07:02:16.422 INFO:teuthology.orchestra.run.smithi118.stdout:1973 trunc from 0x9ff0ea2 to 0x9b3f2cd 2024-09-13T07:02:16.422 INFO:teuthology.orchestra.run.smithi118.stdout:1975 read 0x40ef65 thru 0x4188ca (0x9966 bytes) 2024-09-13T07:02:16.422 INFO:teuthology.orchestra.run.smithi118.stdout:1976 trunc from 0x9b3f2cd to 0xd911c37 2024-09-13T07:02:16.422 INFO:teuthology.orchestra.run.smithi118.stdout:1977 write 0x60800d thru 0x610b75 (0x8b69 bytes) 2024-09-13T07:02:16.422 INFO:teuthology.orchestra.run.smithi118.stdout:1979 read 0x27cdc01 thru 0x27d2108 (0x4508 bytes) 2024-09-13T07:02:16.422 INFO:teuthology.orchestra.run.smithi118.stdout:1981 write 0xb31fc68 thru 0xb32f7e3 (0xfb7c bytes) 2024-09-13T07:02:16.422 INFO:teuthology.orchestra.run.smithi118.stdout:1982 read 0xcd3c2ba thru 0xcd3dd4c (0x1a93 bytes) 2024-09-13T07:02:16.422 INFO:teuthology.orchestra.run.smithi118.stdout:1983 read 0x95ebbff thru 0x95f21f4 (0x65f6 bytes) 2024-09-13T07:02:16.423 INFO:teuthology.orchestra.run.smithi118.stdout:1984 write 0xbb844a thru 0xbc05f7 (0x81ae bytes) 2024-09-13T07:02:16.423 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:16.417+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:16.435 INFO:teuthology.orchestra.run.smithi118.stdout:1985 trunc from 0xd911c37 to 0x57cd45c 2024-09-13T07:02:16.448 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:16.476 INFO:teuthology.orchestra.run.smithi118.stdout:1989 write 0xcd030df thru 0xcd075ce (0x44f0 bytes) 2024-09-13T07:02:16.479 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:16.498 INFO:teuthology.orchestra.run.smithi118.stdout:1992 trunc from 0xcd075cf to 0x1e82068 2024-09-13T07:02:16.520 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:16.518+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:16.521 INFO:teuthology.orchestra.run.smithi118.stdout:1993 write 0x20ca8c0 thru 0x20ce718 (0x3e59 bytes) 2024-09-13T07:02:16.538 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:16.538 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:16.538 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:16.538 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:16.542 INFO:teuthology.orchestra.run.smithi118.stdout:1994 trunc from 0x20ce719 to 0xd4713ab 2024-09-13T07:02:16.550 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:16.554 INFO:teuthology.orchestra.run.smithi118.stdout:1995 writesame 0x944118c thru 0x944a4ad data_size 0x2(0x9322 bytes) 2024-09-13T07:02:16.574 INFO:teuthology.orchestra.run.smithi118.stdout:1996 write 0x648bc6b thru 0x648de4c (0x21e2 bytes) 2024-09-13T07:02:16.588 INFO:teuthology.orchestra.run.smithi118.stdout:1997 writesame 0xdd68f16 thru 0xdd6b05b data_size 0x1(0x2146 bytes) 2024-09-13T07:02:16.590 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:16.602 INFO:teuthology.orchestra.run.smithi118.stdout:1999 trunc from 0xdd6b05c to 0x806d34f 2024-09-13T07:02:16.617 INFO:teuthology.orchestra.run.smithi118.stdout:2000 read 0x143584e thru 0x1443567 (0xdd1a bytes) 2024-09-13T07:02:16.622 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:16.621+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:16.624 INFO:teuthology.orchestra.run.smithi118.stdout:2001 trunc from 0x806d34f to 0x4407ce1 2024-09-13T07:02:16.666 INFO:teuthology.orchestra.run.smithi118.stdout:2002 writesame 0xdc77bcf thru 0xdc855d7 data_size 0x1(0xda09 bytes) 2024-09-13T07:02:16.668 INFO:teuthology.orchestra.run.smithi118.stdout:2004 write 0x8f7a97a thru 0x8f81aa3 (0x712a bytes) 2024-09-13T07:02:16.670 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:16.670 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:16.670 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:16.670 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:16.671 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:16.671 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:16.671 INFO:teuthology.orchestra.run.smithi118.stdout:2006 read 0x9b5e5f5 thru 0x9b69110 (0xab1c bytes) 2024-09-13T07:02:16.672 INFO:teuthology.orchestra.run.smithi118.stdout:2008 read 0x3e5352c thru 0x3e5e455 (0xaf2a bytes) 2024-09-13T07:02:16.673 INFO:teuthology.orchestra.run.smithi118.stdout:2011 read 0x3142822 thru 0x3145143 (0x2922 bytes) 2024-09-13T07:02:16.673 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:16.674 INFO:teuthology.orchestra.run.smithi118.stdout:2014 read 0xdbc15ab thru 0xdbcdfb7 (0xca0d bytes) 2024-09-13T07:02:16.674 INFO:teuthology.orchestra.run.smithi118.stdout:2015 write 0xbe922a2 thru 0xbe96cc0 (0x4a1f bytes) 2024-09-13T07:02:16.676 INFO:teuthology.orchestra.run.smithi118.stdout:2016 writesame 0x279614f thru 0x2798c38 data_size 0x3(0x2aea bytes) 2024-09-13T07:02:16.681 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:16.681 INFO:teuthology.orchestra.run.smithi118.stdout:2017 read 0x15ff559 thru 0x16024cd (0x2f75 bytes) 2024-09-13T07:02:16.701 INFO:teuthology.orchestra.run.smithi118.stdout:2018 trunc from 0xdc855d8 to 0x2623ab9 2024-09-13T07:02:16.716 INFO:teuthology.orchestra.run.smithi118.stdout:2019 read 0x1fdcd1f thru 0x1fe6206 (0x94e8 bytes) 2024-09-13T07:02:16.723 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:16.721+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:16.738 INFO:teuthology.orchestra.run.smithi118.stdout:2020 write 0x8820b95 thru 0x883061a (0xfa86 bytes) 2024-09-13T07:02:16.741 INFO:teuthology.orchestra.run.smithi118.stdout:2021 writesame 0x47206a thru 0x481a42 data_size 0x1(0xf9d9 bytes) 2024-09-13T07:02:16.783 INFO:teuthology.orchestra.run.smithi118.stdout:2023 read 0x86eb397 thru 0x86f1291 (0x5efb bytes) 2024-09-13T07:02:16.783 INFO:teuthology.orchestra.run.smithi118.stdout:2026 read 0x62e120 thru 0x639104 (0xafe5 bytes) 2024-09-13T07:02:16.784 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:16.791 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:16.791 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:16.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:16.791 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:16.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.791 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:02:16.792 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:16.792 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.297992764, 2024-09-13T07:02:16.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:02:16.792 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.792 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:16.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.792 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:02:16.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.792 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:16.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:16.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:02:16.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:16.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:02:16.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:02:16.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:02:16.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:02:16.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:02:16.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:02:16.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:02:16.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:02:16.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:02:16.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:02:16.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:02:16.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:02:16.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:02:16.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:02:16.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:02:16.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:02:16.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:02:16.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.797 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.797 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.797 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:16.797 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.8115092, 2024-09-13T07:02:16.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:02:16.798 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.798 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:16.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.798 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:02:16.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.798 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:16.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:16.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:16.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:02:16.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:02:16.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:02:16.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:02:16.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:02:16.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:02:16.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:02:16.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:02:16.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:02:16.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:02:16.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:02:16.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:02:16.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.802 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.802 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.802 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:16.802 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.811439271000001, 2024-09-13T07:02:16.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:02:16.803 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.803 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:16.803 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.803 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.803 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.803 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:02:16.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.803 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:16.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:16.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:02:16.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:02:16.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:02:16.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:02:16.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:02:16.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:02:16.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:02:16.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:02:16.807 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.807 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.807 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.807 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.807 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:16.807 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.811423398999999, 2024-09-13T07:02:16.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:02:16.807 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.807 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:16.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.808 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.808 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.808 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:02:16.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.808 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:16.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.811 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:16.812 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.811352941999999, 2024-09-13T07:02:16.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:02:16.812 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.812 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:16.812 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.812 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.812 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.812 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:02:16.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.812 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:16.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:16.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:16.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:02:16.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:02:16.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:02:16.816 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.816 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.816 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.816 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.816 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:16.816 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.808000904, 2024-09-13T07:02:16.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:02:16.816 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.816 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.816 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.816 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.817 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:02:16.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.817 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:16.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:16.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:02:16.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:02:16.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.820 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.820 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.820 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:16.820 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807984603000001, 2024-09-13T07:02:16.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:02:16.820 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:16.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:02:16.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:02:16.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:02:16.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:02:16.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:02:16.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:02:16.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:02:16.824 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.824 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.824 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.824 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.824 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:16.824 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807964817999999, 2024-09-13T07:02:16.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:02:16.825 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.825 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.825 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.825 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.825 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.825 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:02:16.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.825 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:16.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:16.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:16.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:02:16.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:02:16.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:02:16.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:02:16.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:02:16.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:02:16.828 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.828 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.828 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807948544999999, 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.829 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:16.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:16.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.832 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.833 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.833 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:16.833 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807930720000002, 2024-09-13T07:02:16.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:02:16.833 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.833 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.833 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:16.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:02:16.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:02:16.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:02:16.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:02:16.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:02:16.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:02:16.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:02:16.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:02:16.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:02:16.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:02:16.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.837 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.837 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.837 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:16.837 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807914626999999, 2024-09-13T07:02:16.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:02:16.837 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.837 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.837 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.838 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:02:16.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.838 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:16.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:16.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:02:16.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:02:16.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:02:16.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:02:16.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:02:16.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:02:16.841 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.841 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.841 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.841 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.841 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:16.841 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807893441000001, 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:16.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:02:16.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:02:16.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:02:16.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:02:16.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:02:16.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:02:16.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:02:16.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:02:16.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:02:16.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:02:16.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:02:16.845 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.845 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.845 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807877408, 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:16.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:16.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:02:16.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:02:16.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:02:16.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:02:16.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:02:16.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:02:16.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:02:16.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:02:16.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:02:16.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:02:16.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:02:16.849 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.849 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807828169, 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:02:16.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.851 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:16.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:16.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:02:16.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:02:16.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:02:16.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:02:16.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:02:16.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:02:16.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.854 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.854 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.854 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:16.854 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807810109000002, 2024-09-13T07:02:16.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:02:16.854 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.854 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.854 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.854 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.855 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:02:16.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.855 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:16.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:16.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:16.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:02:16.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:02:16.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:02:16.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:02:16.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:02:16.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:02:16.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:02:16.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:02:16.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807793594, 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:02:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.859 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.859 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:02:16.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.859 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:16.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:02:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:02:16.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:02:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:02:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:02:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:02:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:02:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.862 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807775229000001, 2024-09-13T07:02:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:02:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:02:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:02:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:02:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:02:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:02:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:02:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:02:16.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807759131000001, 2024-09-13T07:02:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:02:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:02:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:16.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:02:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:02:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:02:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:02:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:02:16.870 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807743196000001, 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.871 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:02:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:02:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:02:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:02:16.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:02:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:02:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:02:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:02:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:02:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:02:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:02:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.807726699, 2024-09-13T07:02:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:02:16.875 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:02:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.876 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:16.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:02:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:02:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:02:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:02:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:02:16.879 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:16.880 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:16.880 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:16.881 INFO:teuthology.orchestra.run.smithi118.stdout:2028 write 0x8e8dea6 thru 0x8e901dc (0x2337 bytes) 2024-09-13T07:02:16.882 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:16.883 INFO:teuthology.orchestra.run.smithi118.stdout:2029 write 0x85d5c12 thru 0x85e22fb (0xc6ea bytes) 2024-09-13T07:02:16.883 INFO:teuthology.orchestra.run.smithi118.stdout:2030 read 0x3433e15 thru 0x343ea83 (0xac6f bytes) 2024-09-13T07:02:16.883 INFO:teuthology.orchestra.run.smithi118.stdout:2032 writesame 0x8936b45 thru 0x893d9f5 data_size 0x1(0x6eb1 bytes) 2024-09-13T07:02:16.883 INFO:teuthology.orchestra.run.smithi118.stdout:2035 writesame 0xd669328 thru 0xd66e72d data_size 0x2d(0x5406 bytes) 2024-09-13T07:02:16.883 INFO:teuthology.orchestra.run.smithi118.stdout:2036 read 0x685d7ec thru 0x6864a77 (0x728c bytes) 2024-09-13T07:02:16.884 INFO:teuthology.orchestra.run.smithi118.stdout:2039 write 0x13fafc8 thru 0x140a387 (0xf3c0 bytes) 2024-09-13T07:02:16.884 INFO:teuthology.orchestra.run.smithi118.stdout:2040 write 0x2944aa0 thru 0x29475bf (0x2b20 bytes) 2024-09-13T07:02:16.884 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:16.884 INFO:teuthology.orchestra.run.smithi118.stdout:2042 punch from 0x83cc2ef to 0x83d0015, (0x3d26 bytes) 2024-09-13T07:02:16.884 INFO:teuthology.orchestra.run.smithi118.stdout:2043 trunc from 0xd66e72e to 0xe68c2cd 2024-09-13T07:02:16.884 INFO:teuthology.orchestra.run.smithi118.stdout:2044 write 0xb8bc241 thru 0xb8be1dc (0x1f9c bytes) 2024-09-13T07:02:16.884 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:16.884 INFO:teuthology.orchestra.run.smithi118.stdout:2046 read 0x9e9b574 thru 0x9e9e9b7 (0x3444 bytes) 2024-09-13T07:02:16.884 INFO:teuthology.orchestra.run.smithi118.stdout:2047 punch from 0x8471ccc to 0x8474609, (0x293d bytes) 2024-09-13T07:02:16.884 INFO:teuthology.orchestra.run.smithi118.stdout:2048 punch from 0xa0c414 to 0xa0e7b3, (0x239f bytes) 2024-09-13T07:02:16.884 INFO:teuthology.orchestra.run.smithi118.stdout:2050 read 0xc1525ea thru 0xc1549b7 (0x23ce bytes) 2024-09-13T07:02:16.885 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:16.878+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:16.891 INFO:teuthology.orchestra.run.smithi118.stdout:2051 writesame 0xebd45b9 thru 0xebd946e data_size 0x1f(0x4eb6 bytes) 2024-09-13T07:02:16.894 INFO:teuthology.orchestra.run.smithi118.stdout:2052 punch from 0x8ced71a to 0x8cfd1c9, (0xfaaf bytes) 2024-09-13T07:02:16.913 INFO:teuthology.orchestra.run.smithi118.stdout:2054 trunc from 0xebd946f to 0x69f953c 2024-09-13T07:02:16.929 INFO:teuthology.orchestra.run.smithi118.stdout:2055 write 0x49596e0 thru 0x49687b4 (0xf0d5 bytes) 2024-09-13T07:02:16.932 INFO:teuthology.orchestra.run.smithi118.stdout:2057 punch from 0x1ef9db8 to 0x1efb39b, (0x15e3 bytes) 2024-09-13T07:02:16.932 INFO:teuthology.orchestra.run.smithi118.stdout:2058 read 0x39c260c thru 0x39c8ff4 (0x69e9 bytes) 2024-09-13T07:02:16.932 INFO:teuthology.orchestra.run.smithi118.stdout:2059 punch from 0x3247dc2 to 0x3255b99, (0xddd7 bytes) 2024-09-13T07:02:16.932 INFO:teuthology.orchestra.run.smithi118.stdout:2061 write 0x78fcf6 thru 0x793e93 (0x419e bytes) 2024-09-13T07:02:16.946 INFO:teuthology.orchestra.run.smithi118.stdout:2062 writesame 0x70f469c thru 0x70f5125 data_size 0x13(0xa8a bytes) 2024-09-13T07:02:16.971 INFO:teuthology.orchestra.run.smithi118.stdout:2063 write 0xcc79c8d thru 0xcc824c6 (0x883a bytes) 2024-09-13T07:02:16.982 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:16.981+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:16.987 INFO:teuthology.orchestra.run.smithi118.stdout:2064 trunc from 0xcc824c7 to 0x577dd99 2024-09-13T07:02:17.003 INFO:teuthology.orchestra.run.smithi118.stdout:2065 read 0x2b5aa07 thru 0x2b5b000 (0x5fa bytes) 2024-09-13T07:02:17.004 INFO:teuthology.orchestra.run.smithi118.stdout:2066 read 0x2dd2443 thru 0x2dd7b21 (0x56df bytes) 2024-09-13T07:02:17.005 INFO:teuthology.orchestra.run.smithi118.stdout:2067 read 0x3c270f0 thru 0x3c2a796 (0x36a7 bytes) 2024-09-13T07:02:17.005 INFO:teuthology.orchestra.run.smithi118.stdout:2068 punch from 0x15b4b15 to 0x15bfaf8, (0xafe3 bytes) 2024-09-13T07:02:17.005 INFO:teuthology.orchestra.run.smithi118.stdout:2069 read 0x35b4fd7 thru 0x35b7ebe (0x2ee8 bytes) 2024-09-13T07:02:17.006 INFO:teuthology.orchestra.run.smithi118.stdout:2071 read 0x40f992e thru 0x40fc461 (0x2b34 bytes) 2024-09-13T07:02:17.018 INFO:teuthology.orchestra.run.smithi118.stdout:2072 writesame 0x64e2d98 thru 0x64f294f data_size 0x2d(0xfbb8 bytes) 2024-09-13T07:02:17.019 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:17.019 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:17.020 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:17.020 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:17.020 INFO:teuthology.orchestra.run.smithi118.stdout:2073 punch from 0x5a418a to 0x5b296a, (0xe7e0 bytes) 2024-09-13T07:02:17.021 INFO:teuthology.orchestra.run.smithi118.stdout:2074 write 0x34ae529 thru 0x34b6cd2 (0x87aa bytes) 2024-09-13T07:02:17.032 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:17.048 INFO:teuthology.orchestra.run.smithi118.stdout:2075 write 0xcdb6283 thru 0xcdbca52 (0x67d0 bytes) 2024-09-13T07:02:17.051 INFO:teuthology.orchestra.run.smithi118.stdout:2076 read 0xc4adc49 thru 0xc4b9b7c (0xbf34 bytes) 2024-09-13T07:02:17.053 INFO:teuthology.orchestra.run.smithi118.stdout:2077 trunc from 0xcdbca53 to 0xc2f3616 2024-09-13T07:02:17.066 INFO:teuthology.orchestra.run.smithi118.stdout:2078 writesame 0x96e350a thru 0x96f22e1 data_size 0x3b(0xedd8 bytes) 2024-09-13T07:02:17.071 INFO:teuthology.orchestra.run.smithi118.stdout:2079 punch from 0xbf772ca to 0xbf7b6e5, (0x441b bytes) 2024-09-13T07:02:17.071 INFO:teuthology.orchestra.run.smithi118.stdout:2081 write 0xaf661a thru 0xafa90a (0x42f1 bytes) 2024-09-13T07:02:17.073 INFO:teuthology.orchestra.run.smithi118.stdout:2085 read 0x4d98996 thru 0x4da098f (0x7ffa bytes) 2024-09-13T07:02:17.073 INFO:teuthology.orchestra.run.smithi118.stdout:2088 read 0x803a983 thru 0x80466b4 (0xbd32 bytes) 2024-09-13T07:02:17.074 INFO:teuthology.orchestra.run.smithi118.stdout:2089 read 0x362d585 thru 0x363923a (0xbcb6 bytes) 2024-09-13T07:02:17.075 INFO:teuthology.orchestra.run.smithi118.stdout:2090 write 0x4844e5d thru 0x484c9e8 (0x7b8c bytes) 2024-09-13T07:02:17.078 INFO:teuthology.orchestra.run.smithi118.stdout:2091 read 0x1a4fc0d thru 0x1a5c177 (0xc56b bytes) 2024-09-13T07:02:17.079 INFO:teuthology.orchestra.run.smithi118.stdout:2092 write 0x165fb4e thru 0x16678f5 (0x7da8 bytes) 2024-09-13T07:02:17.082 INFO:teuthology.orchestra.run.smithi118.stdout:2094 punch from 0x5f8e2b6 to 0x5f9cb25, (0xe86f bytes) 2024-09-13T07:02:17.082 INFO:teuthology.orchestra.run.smithi118.stdout:2096 punch from 0x40942c2 to 0x4097d65, (0x3aa3 bytes) 2024-09-13T07:02:17.082 INFO:teuthology.orchestra.run.smithi118.stdout:2097 write 0x905f169 thru 0x9067fda (0x8e72 bytes) 2024-09-13T07:02:17.083 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:17.078+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:17.084 INFO:teuthology.orchestra.run.smithi118.stdout:2098 punch from 0x8438673 to 0x843ce3f, (0x47cc bytes) 2024-09-13T07:02:17.084 INFO:teuthology.orchestra.run.smithi118.stdout:2099 write 0x3ba67aa thru 0x3bac88d (0x60e4 bytes) 2024-09-13T07:02:17.087 INFO:teuthology.orchestra.run.smithi118.stdout:2100 clone 22 order 25 su 131072 sc 10 2024-09-13T07:02:17.153 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:17.154 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:17.154 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:17.154 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:17.154 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:17.163 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:17.183 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:17.182+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:17.283 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:02:17.284 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:17.282+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:17.341 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:17.341 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:17.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:17.341 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:17.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:434 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'4, mlcod=26'2)", 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.804530001000003, 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001931945, 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535695+0000", 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535709+0000", 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3577999999999999e-05 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535732+0000", 2024-09-13T07:02:17.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3085e-05 2024-09-13T07:02:17.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535742+0000", 2024-09-13T07:02:17.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.048e-05 2024-09-13T07:02:17.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535840+0000", 2024-09-13T07:02:17.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7159999999999998e-05 2024-09-13T07:02:17.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535871+0000", 2024-09-13T07:02:17.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1510999999999997e-05 2024-09-13T07:02:17.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537572+0000", 2024-09-13T07:02:17.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017008279999999999 2024-09-13T07:02:17.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537629+0000", 2024-09-13T07:02:17.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6520000000000001e-05 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:435 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'5, mlcod=26'2)", 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.803398295000001, 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017126769999999999, 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536835+0000", 2024-09-13T07:02:17.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4939999999999997e-06 2024-09-13T07:02:17.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536854+0000", 2024-09-13T07:02:17.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8655999999999999e-05 2024-09-13T07:02:17.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536862+0000", 2024-09-13T07:02:17.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6570000000000003e-06 2024-09-13T07:02:17.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536926+0000", 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4037999999999999e-05 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536958+0000", 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1584000000000001e-05 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538493+0000", 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00153513 2024-09-13T07:02:17.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538541+0000", 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7883999999999998e-05 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:483 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'24, mlcod=26'21)", 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.724919665999998, 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018371780000000001, 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.350 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615306+0000", 2024-09-13T07:02:17.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615322+0000", 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5519000000000002e-05 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615335+0000", 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3854e-05 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615345+0000", 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4930000000000004e-06 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615431+0000", 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6346000000000003e-05 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615480+0000", 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8303999999999997e-05 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617102+0000", 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016225250000000001 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617144+0000", 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1936e-05 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:518 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'38, mlcod=26'36)", 2024-09-13T07:02:17.354 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.015342928000003, 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018201179999999999, 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:17.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324883+0000", 2024-09-13T07:02:17.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:17.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324891+0000", 2024-09-13T07:02:17.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1499999999999999e-06 2024-09-13T07:02:17.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324924+0000", 2024-09-13T07:02:17.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2601999999999997e-05 2024-09-13T07:02:17.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324942+0000", 2024-09-13T07:02:17.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7796999999999999e-05 2024-09-13T07:02:17.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325060+0000", 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011786900000000001 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325123+0000", 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3474999999999998e-05 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326620+0000", 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014968830000000001 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326704+0000", 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3716999999999998e-05 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:597 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'39, mlcod=26'37)", 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.898515600000003, 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017959829999999999, 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441709+0000", 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441728+0000", 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9627999999999999e-05 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441765+0000", 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6559e-05 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441783+0000", 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7691000000000001e-05 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441904+0000", 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121184 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441966+0000", 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2428999999999995e-05 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443448+0000", 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00148197 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443507+0000", 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8699000000000002e-05 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.283635306000001, 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.363 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:02:17.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:17.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:02:17.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:02:17.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:02:17.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:02:17.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:02:17.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:02:17.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:02:17.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:02:17.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:02:17.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:02:17.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:02:17.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:02:17.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:02:17.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:02:17.368 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.368 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.368 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.368 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1548 3.4 e27/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 27'52, mlcod=27'50)", 2024-09-13T07:02:17.368 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:17.368 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.198013770000003, 2024-09-13T07:02:17.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017825779999999999, 2024-09-13T07:02:17.369 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.369 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.369 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:17.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:17.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142211+0000", 2024-09-13T07:02:17.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:17.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142232+0000", 2024-09-13T07:02:17.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0888000000000001e-05 2024-09-13T07:02:17.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142268+0000", 2024-09-13T07:02:17.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6261999999999999e-05 2024-09-13T07:02:17.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142285+0000", 2024-09-13T07:02:17.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7640000000000001e-05 2024-09-13T07:02:17.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142381+0000", 2024-09-13T07:02:17.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5171000000000001e-05 2024-09-13T07:02:17.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142450+0000", 2024-09-13T07:02:17.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9778999999999997e-05 2024-09-13T07:02:17.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143953+0000", 2024-09-13T07:02:17.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001503129 2024-09-13T07:02:17.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143995+0000", 2024-09-13T07:02:17.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1878000000000001e-05 2024-09-13T07:02:17.374 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.374 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.374 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.375 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1550 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'14, mlcod=27'12)", 2024-09-13T07:02:17.375 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:17.375 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.197366058999997, 2024-09-13T07:02:17.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002750452, 2024-09-13T07:02:17.375 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.375 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.375 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:17.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:17.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142859+0000", 2024-09-13T07:02:17.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:17.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142874+0000", 2024-09-13T07:02:17.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5149e-05 2024-09-13T07:02:17.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142911+0000", 2024-09-13T07:02:17.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6439000000000001e-05 2024-09-13T07:02:17.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142926+0000", 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5079000000000001e-05 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143012+0000", 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6230000000000006e-05 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143063+0000", 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1369000000000002e-05 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145539+0000", 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024753420000000002 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145611+0000", 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2392999999999996e-05 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.379 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1668 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'17, mlcod=27'15)", 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.998950596999997, 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033976620000000001, 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341281+0000", 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8109999999999999e-06 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341298+0000", 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6575e-05 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341308+0000", 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0369e-05 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341434+0000", 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125454 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341453+0000", 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9159000000000001e-05 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344638+0000", 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031851750000000002 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344674+0000", 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5410999999999997e-05 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1666 3.4 e28/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 28'55, mlcod=27'53)", 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.998896924999997, 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001805809, 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341329+0000", 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341349+0000", 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.999e-05 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341357+0000", 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0620000000000003e-06 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341361+0000", 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341710+0000", 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034873400000000001 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341736+0000", 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5772e-05 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343097+0000", 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013615319999999999 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343135+0000", 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8111999999999997e-05 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1800 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'20, mlcod=28'18)", 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.796526458999999, 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001874398, 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543699+0000", 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543713+0000", 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4786e-05 2024-09-13T07:02:17.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543737+0000", 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3626e-05 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543752+0000", 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.489e-05 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543832+0000", 2024-09-13T07:02:17.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0396999999999996e-05 2024-09-13T07:02:17.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543994+0000", 2024-09-13T07:02:17.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016129100000000001 2024-09-13T07:02:17.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545471+0000", 2024-09-13T07:02:17.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014771960000000001 2024-09-13T07:02:17.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545575+0000", 2024-09-13T07:02:17.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103732 2024-09-13T07:02:17.393 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.393 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.393 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.394 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1882 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'23, mlcod=28'21)", 2024-09-13T07:02:17.394 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:17.394 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.700893477999998, 2024-09-13T07:02:17.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001767172, 2024-09-13T07:02:17.394 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.394 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.394 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:17.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:17.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639332+0000", 2024-09-13T07:02:17.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:17.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639343+0000", 2024-09-13T07:02:17.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1436e-05 2024-09-13T07:02:17.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639361+0000", 2024-09-13T07:02:17.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7915e-05 2024-09-13T07:02:17.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639370+0000", 2024-09-13T07:02:17.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3189999999999995e-06 2024-09-13T07:02:17.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639425+0000", 2024-09-13T07:02:17.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4917999999999998e-05 2024-09-13T07:02:17.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639454+0000", 2024-09-13T07:02:17.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9862999999999999e-05 2024-09-13T07:02:17.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641059+0000", 2024-09-13T07:02:17.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016048950000000001 2024-09-13T07:02:17.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641100+0000", 2024-09-13T07:02:17.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0883999999999997e-05 2024-09-13T07:02:17.398 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.398 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.399 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.399 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3069 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'33, mlcod=31'31)", 2024-09-13T07:02:17.399 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:17.399 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.177753762000002, 2024-09-13T07:02:17.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018597010000000001, 2024-09-13T07:02:17.399 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.399 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.399 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:17.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:17.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162471+0000", 2024-09-13T07:02:17.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:17.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162520+0000", 2024-09-13T07:02:17.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8466e-05 2024-09-13T07:02:17.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162546+0000", 2024-09-13T07:02:17.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6089999999999999e-05 2024-09-13T07:02:17.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162562+0000", 2024-09-13T07:02:17.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5962000000000001e-05 2024-09-13T07:02:17.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162646+0000", 2024-09-13T07:02:17.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3985000000000003e-05 2024-09-13T07:02:17.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162691+0000", 2024-09-13T07:02:17.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5785999999999998e-05 2024-09-13T07:02:17.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164265+0000", 2024-09-13T07:02:17.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015740890000000001 2024-09-13T07:02:17.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164333+0000", 2024-09-13T07:02:17.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7014999999999994e-05 2024-09-13T07:02:17.404 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.404 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.404 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.404 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3121 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'36, mlcod=31'34)", 2024-09-13T07:02:17.404 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:17.404 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.063807375000003, 2024-09-13T07:02:17.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019657780000000001, 2024-09-13T07:02:17.405 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.405 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.405 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:17.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:17.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276417+0000", 2024-09-13T07:02:17.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:17.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276497+0000", 2024-09-13T07:02:17.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9729999999999997e-05 2024-09-13T07:02:17.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276537+0000", 2024-09-13T07:02:17.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0102999999999999e-05 2024-09-13T07:02:17.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276553+0000", 2024-09-13T07:02:17.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6518999999999999e-05 2024-09-13T07:02:17.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276662+0000", 2024-09-13T07:02:17.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109252 2024-09-13T07:02:17.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276729+0000", 2024-09-13T07:02:17.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7056000000000003e-05 2024-09-13T07:02:17.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278270+0000", 2024-09-13T07:02:17.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015408819999999999 2024-09-13T07:02:17.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278385+0000", 2024-09-13T07:02:17.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114613 2024-09-13T07:02:17.409 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.409 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.409 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.410 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3390 3.0 e32/22 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:head v 32'45, mlcod=31'42)", 2024-09-13T07:02:17.410 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:17.410 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.844719028, 2024-09-13T07:02:17.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002246587, 2024-09-13T07:02:17.410 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.410 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.410 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:17.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:17.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495507+0000", 2024-09-13T07:02:17.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:17.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495530+0000", 2024-09-13T07:02:17.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2592999999999999e-05 2024-09-13T07:02:17.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495550+0000", 2024-09-13T07:02:17.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0152999999999998e-05 2024-09-13T07:02:17.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495562+0000", 2024-09-13T07:02:17.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2336e-05 2024-09-13T07:02:17.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495662+0000", 2024-09-13T07:02:17.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100317 2024-09-13T07:02:17.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495717+0000", 2024-09-13T07:02:17.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4304e-05 2024-09-13T07:02:17.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497707+0000", 2024-09-13T07:02:17.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019906849999999999 2024-09-13T07:02:17.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497754+0000", 2024-09-13T07:02:17.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6715999999999998e-05 2024-09-13T07:02:17.414 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.414 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.415 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.415 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3399 3.4 e32/22 3:22863a52:::rbd_data.11945d91e99c.0000000000000013:head v 32'65, mlcod=31'62)", 2024-09-13T07:02:17.415 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:17.415 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.840782355999998, 2024-09-13T07:02:17.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028830779999999999, 2024-09-13T07:02:17.415 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.415 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.415 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:17.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:17.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499442+0000", 2024-09-13T07:02:17.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:17.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499459+0000", 2024-09-13T07:02:17.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7042e-05 2024-09-13T07:02:17.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499493+0000", 2024-09-13T07:02:17.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3788000000000002e-05 2024-09-13T07:02:17.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499512+0000", 2024-09-13T07:02:17.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9055999999999998e-05 2024-09-13T07:02:17.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499608+0000", 2024-09-13T07:02:17.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6168000000000005e-05 2024-09-13T07:02:17.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499658+0000", 2024-09-13T07:02:17.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9469999999999999e-05 2024-09-13T07:02:17.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502277+0000", 2024-09-13T07:02:17.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026188639999999998 2024-09-13T07:02:17.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502327+0000", 2024-09-13T07:02:17.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0578999999999998e-05 2024-09-13T07:02:17.420 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.420 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.420 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.420 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3395 3.0 e32/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 32'47, mlcod=31'43)", 2024-09-13T07:02:17.420 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:17.420 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.840684271000001, 2024-09-13T07:02:17.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027505870000000001, 2024-09-13T07:02:17.420 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.420 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.421 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:17.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:17.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:17.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:17.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499560+0000", 2024-09-13T07:02:17.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8416000000000001e-05 2024-09-13T07:02:17.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499576+0000", 2024-09-13T07:02:17.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5602000000000001e-05 2024-09-13T07:02:17.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499587+0000", 2024-09-13T07:02:17.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1112e-05 2024-09-13T07:02:17.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499654+0000", 2024-09-13T07:02:17.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7164999999999998e-05 2024-09-13T07:02:17.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499674+0000", 2024-09-13T07:02:17.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9638000000000001e-05 2024-09-13T07:02:17.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502210+0000", 2024-09-13T07:02:17.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025367100000000002 2024-09-13T07:02:17.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502293+0000", 2024-09-13T07:02:17.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2607000000000004e-05 2024-09-13T07:02:17.425 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.425 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.425 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.425 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:4534 3.0 e32/22 3:00392e9e:::rbd_data.11944cdc0656.000000000000000c:head v 32'51, mlcod=32'49)", 2024-09-13T07:02:17.425 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:17.426 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.428983084999999, 2024-09-13T07:02:17.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017597330000000001, 2024-09-13T07:02:17.426 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.426 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.426 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:17.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:17.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911242+0000", 2024-09-13T07:02:17.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:17.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911306+0000", 2024-09-13T07:02:17.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4480999999999995e-05 2024-09-13T07:02:17.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911338+0000", 2024-09-13T07:02:17.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1733000000000003e-05 2024-09-13T07:02:17.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911351+0000", 2024-09-13T07:02:17.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3241000000000001e-05 2024-09-13T07:02:17.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911455+0000", 2024-09-13T07:02:17.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103723 2024-09-13T07:02:17.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911495+0000", 2024-09-13T07:02:17.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0683000000000003e-05 2024-09-13T07:02:17.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.912933+0000", 2024-09-13T07:02:17.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001437219 2024-09-13T07:02:17.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.913003+0000", 2024-09-13T07:02:17.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0629000000000004e-05 2024-09-13T07:02:17.430 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.431 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.431 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.431 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11205 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'50, mlcod=38'47)", 2024-09-13T07:02:17.431 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:17.431 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 38.715642600000002, 2024-09-13T07:02:17.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031049490000000001, 2024-09-13T07:02:17.431 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.431 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.431 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:17.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:17.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624583+0000", 2024-09-13T07:02:17.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:17.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625339+0000", 2024-09-13T07:02:17.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00075563200000000001 2024-09-13T07:02:17.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625358+0000", 2024-09-13T07:02:17.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9916999999999999e-05 2024-09-13T07:02:17.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625366+0000", 2024-09-13T07:02:17.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8050000000000007e-06 2024-09-13T07:02:17.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625415+0000", 2024-09-13T07:02:17.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8231e-05 2024-09-13T07:02:17.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625447+0000", 2024-09-13T07:02:17.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2703999999999997e-05 2024-09-13T07:02:17.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627652+0000", 2024-09-13T07:02:17.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022050659999999999 2024-09-13T07:02:17.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627689+0000", 2024-09-13T07:02:17.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6612000000000001e-05 2024-09-13T07:02:17.436 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.436 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.436 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.436 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11206 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'52, mlcod=38'47)", 2024-09-13T07:02:17.436 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:17.436 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 38.714371933999999, 2024-09-13T07:02:17.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022238470000000002, 2024-09-13T07:02:17.436 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.437 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.437 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:17.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:17.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625854+0000", 2024-09-13T07:02:17.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:17.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625863+0000", 2024-09-13T07:02:17.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5380000000000008e-06 2024-09-13T07:02:17.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625883+0000", 2024-09-13T07:02:17.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9247000000000001e-05 2024-09-13T07:02:17.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625893+0000", 2024-09-13T07:02:17.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0176e-05 2024-09-13T07:02:17.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626411+0000", 2024-09-13T07:02:17.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051847600000000003 2024-09-13T07:02:17.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626431+0000", 2024-09-13T07:02:17.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0199000000000001e-05 2024-09-13T07:02:17.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628028+0000", 2024-09-13T07:02:17.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015968110000000001 2024-09-13T07:02:17.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628078+0000", 2024-09-13T07:02:17.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0170000000000002e-05 2024-09-13T07:02:17.441 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.441 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.441 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.441 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.441 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.442 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:17.443 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:17.444 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone21 from 0xc2f3616 (overlap 0x1e82068) to 0xc34029 2024-09-13T07:02:17.445 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:17.442+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:17.544 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:17.541+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:17.547 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:17.547 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:17.547 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:17.547 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:17.557 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:17.573 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-13T07:02:17.645 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:17.645+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:17.692 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:17.692 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:17.692 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:17.692 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:17.692 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:17.701 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:17.746 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:17.745+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:17.803 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:17.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1531 3.c e27/22 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head v 27'9, mlcod=27'7)", 2024-09-13T07:02:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:02:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.727464324000003, 2024-09-13T07:02:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039127038000000003, 2024-09-13T07:02:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:02:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:02:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074530+0000", 2024-09-13T07:02:17.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099434+0000", 2024-09-13T07:02:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.024903926999999999 2024-09-13T07:02:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099459+0000", 2024-09-13T07:02:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5384999999999998e-05 2024-09-13T07:02:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099469+0000", 2024-09-13T07:02:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.024e-05 2024-09-13T07:02:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099566+0000", 2024-09-13T07:02:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6533999999999994e-05 2024-09-13T07:02:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099600+0000", 2024-09-13T07:02:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4007000000000002e-05 2024-09-13T07:02:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113477+0000", 2024-09-13T07:02:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01387741 2024-09-13T07:02:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113658+0000", 2024-09-13T07:02:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000180953 2024-09-13T07:02:17.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12393 3.b 3:d3c3e30e:::rbd_data.11945fc173dd.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:02:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:02:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529531390999999, 2024-09-13T07:02:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027522214, 2024-09-13T07:02:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12393 2024-09-13T07:02:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:02:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:02:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272462+0000", 2024-09-13T07:02:17.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272471+0000", 2024-09-13T07:02:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8510000000000005e-06 2024-09-13T07:02:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272486+0000", 2024-09-13T07:02:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5409000000000001e-05 2024-09-13T07:02:17.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272494+0000", 2024-09-13T07:02:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0020000000000006e-06 2024-09-13T07:02:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272575+0000", 2024-09-13T07:02:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0467000000000001e-05 2024-09-13T07:02:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272710+0000", 2024-09-13T07:02:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135554 2024-09-13T07:02:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:02:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.279539+0000", 2024-09-13T07:02:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068285439999999998 2024-09-13T07:02:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299872+0000", 2024-09-13T07:02:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020332507 2024-09-13T07:02:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299920+0000", 2024-09-13T07:02:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8009000000000001e-05 2024-09-13T07:02:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299956+0000", 2024-09-13T07:02:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6554999999999998e-05 2024-09-13T07:02:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299986+0000", 2024-09-13T07:02:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0331000000000001e-05 2024-09-13T07:02:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.816 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:48366 3.2 3:4282dbac:::rbd_data.1194fc792839.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:58.185470+0000", 2024-09-13T07:02:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.616525638999999, 2024-09-13T07:02:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.038773378999999997, 2024-09-13T07:02:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 48366 2024-09-13T07:02:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.185470+0000", 2024-09-13T07:02:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.185470+0000", 2024-09-13T07:02:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.185467+0000", 2024-09-13T07:02:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192092+0000", 2024-09-13T07:02:17.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066248230000000002 2024-09-13T07:02:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192114+0000", 2024-09-13T07:02:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1685999999999999e-05 2024-09-13T07:02:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192122+0000", 2024-09-13T07:02:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4989999999999994e-06 2024-09-13T07:02:17.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192208+0000", 2024-09-13T07:02:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5995000000000006e-05 2024-09-13T07:02:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192454+0000", 2024-09-13T07:02:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024575200000000001 2024-09-13T07:02:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.193274+0000", 2024-09-13T07:02:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081968299999999998 2024-09-13T07:02:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.212795+0000", 2024-09-13T07:02:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019520991000000001 2024-09-13T07:02:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:17.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.224086+0000", 2024-09-13T07:02:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011291539 2024-09-13T07:02:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.224184+0000", 2024-09-13T07:02:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7850000000000007e-05 2024-09-13T07:02:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.224243+0000", 2024-09-13T07:02:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9200999999999998e-05 2024-09-13T07:02:17.823 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62027 3.18 3:195cb5a2:::rbd_data.1194987e4741.0000000000000003:17 [read 3491840~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.004536+0000", 2024-09-13T07:02:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7974596869999999, 2024-09-13T07:02:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025634061, 2024-09-13T07:02:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:17.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62027 2024-09-13T07:02:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004536+0000", 2024-09-13T07:02:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004536+0000", 2024-09-13T07:02:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004535+0000", 2024-09-13T07:02:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004538+0000", 2024-09-13T07:02:17.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.971e-06 2024-09-13T07:02:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004545+0000", 2024-09-13T07:02:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3629999999999999e-06 2024-09-13T07:02:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004549+0000", 2024-09-13T07:02:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9400000000000001e-06 2024-09-13T07:02:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030001+0000", 2024-09-13T07:02:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025451613000000001 2024-09-13T07:02:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030074+0000", 2024-09-13T07:02:17.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2525000000000001e-05 2024-09-13T07:02:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030170+0000", 2024-09-13T07:02:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6383000000000002e-05 2024-09-13T07:02:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62028 3.18 3:195cb5a2:::rbd_data.1194987e4741.0000000000000003:17 [read 3495936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.004555+0000", 2024-09-13T07:02:17.829 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7974408620000002, 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025812420999999999, 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62028 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004555+0000", 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004555+0000", 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004554+0000", 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004557+0000", 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.033e-06 2024-09-13T07:02:17.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004563+0000", 2024-09-13T07:02:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:02:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004567+0000", 2024-09-13T07:02:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.228e-06 2024-09-13T07:02:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030212+0000", 2024-09-13T07:02:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025644851 2024-09-13T07:02:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030286+0000", 2024-09-13T07:02:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3973000000000005e-05 2024-09-13T07:02:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030367+0000", 2024-09-13T07:02:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0889000000000004e-05 2024-09-13T07:02:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.833 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62029 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.004570+0000", 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7974252839999991, 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025821687999999999, 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62029 2024-09-13T07:02:17.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004570+0000", 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004570+0000", 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004570+0000", 2024-09-13T07:02:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004573+0000", 2024-09-13T07:02:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2660000000000002e-06 2024-09-13T07:02:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004578+0000", 2024-09-13T07:02:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7859999999999996e-06 2024-09-13T07:02:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004583+0000", 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5469999999999998e-06 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030386+0000", 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025803578000000001 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030392+0000", 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9089999999999997e-06 2024-09-13T07:02:17.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62030 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.004586+0000", 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7974095170000002, 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025834207000000001, 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.838 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62030 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004586+0000", 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004586+0000", 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004586+0000", 2024-09-13T07:02:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004588+0000", 2024-09-13T07:02:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1749999999999999e-06 2024-09-13T07:02:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004593+0000", 2024-09-13T07:02:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0590000000000002e-06 2024-09-13T07:02:17.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004597+0000", 2024-09-13T07:02:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3159999999999998e-06 2024-09-13T07:02:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030410+0000", 2024-09-13T07:02:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025812630999999999 2024-09-13T07:02:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030420+0000", 2024-09-13T07:02:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0321e-05 2024-09-13T07:02:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62261 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005557+0000", 2024-09-13T07:02:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7964384980000005, 2024-09-13T07:02:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025596205, 2024-09-13T07:02:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62261 2024-09-13T07:02:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005557+0000", 2024-09-13T07:02:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005557+0000", 2024-09-13T07:02:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005557+0000", 2024-09-13T07:02:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005558+0000", 2024-09-13T07:02:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.328e-06 2024-09-13T07:02:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005561+0000", 2024-09-13T07:02:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8140000000000001e-06 2024-09-13T07:02:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005564+0000", 2024-09-13T07:02:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.475e-06 2024-09-13T07:02:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031148+0000", 2024-09-13T07:02:17.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025584928999999999 2024-09-13T07:02:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031153+0000", 2024-09-13T07:02:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8640000000000004e-06 2024-09-13T07:02:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62262 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005568+0000", 2024-09-13T07:02:17.846 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7964274109999998, 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025612378000000002, 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62262 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005568+0000", 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005568+0000", 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005568+0000", 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005569+0000", 2024-09-13T07:02:17.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.141e-06 2024-09-13T07:02:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005572+0000", 2024-09-13T07:02:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8229999999999999e-06 2024-09-13T07:02:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005574+0000", 2024-09-13T07:02:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5009999999999999e-06 2024-09-13T07:02:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031171+0000", 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025596071000000001 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031181+0000", 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0040999999999999e-05 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62263 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005577+0000", 2024-09-13T07:02:17.850 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7964191130000007, 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025620249000000001, 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62263 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005577+0000", 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005577+0000", 2024-09-13T07:02:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005576+0000", 2024-09-13T07:02:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005578+0000", 2024-09-13T07:02:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.37e-06 2024-09-13T07:02:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005581+0000", 2024-09-13T07:02:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8459999999999999e-06 2024-09-13T07:02:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005584+0000", 2024-09-13T07:02:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4920000000000002e-06 2024-09-13T07:02:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031191+0000", 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025607396000000001 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031197+0000", 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.854 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62264 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005587+0000", 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7964086239999997, 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025636381, 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62264 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005587+0000", 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005587+0000", 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005587+0000", 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005588+0000", 2024-09-13T07:02:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1680000000000001e-06 2024-09-13T07:02:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005591+0000", 2024-09-13T07:02:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8909999999999999e-06 2024-09-13T07:02:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005593+0000", 2024-09-13T07:02:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6539999999999999e-06 2024-09-13T07:02:17.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031214+0000", 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025620165 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031223+0000", 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7070000000000004e-06 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.858 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62265 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005596+0000", 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7963999699999995, 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025644693999999999, 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62265 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005596+0000", 2024-09-13T07:02:17.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005596+0000", 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005595+0000", 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005598+0000", 2024-09-13T07:02:17.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.999e-06 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005602+0000", 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.157e-06 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005606+0000", 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9260000000000002e-06 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031235+0000", 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025629880000000001 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031240+0000", 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9409999999999998e-06 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62266 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005608+0000", 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7963877769999996, 2024-09-13T07:02:17.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025664411000000002, 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62266 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005608+0000", 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005608+0000", 2024-09-13T07:02:17.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005608+0000", 2024-09-13T07:02:17.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:17.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005609+0000", 2024-09-13T07:02:17.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.24e-06 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005612+0000", 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005614+0000", 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6960000000000001e-06 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031262+0000", 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025647463999999998 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031272+0000", 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0379e-05 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62267 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005617+0000", 2024-09-13T07:02:17.866 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7963790300000007, 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025672828000000002, 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62267 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005617+0000", 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005617+0000", 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005616+0000", 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005618+0000", 2024-09-13T07:02:17.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3549999999999999e-06 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005621+0000", 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.791e-06 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005623+0000", 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6850000000000001e-06 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031284+0000", 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025660867 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031289+0000", 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999999e-06 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62268 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005626+0000", 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.796369533, 2024-09-13T07:02:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025689170000000001, 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62268 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005626+0000", 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005626+0000", 2024-09-13T07:02:17.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005626+0000", 2024-09-13T07:02:17.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:17.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005627+0000", 2024-09-13T07:02:17.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3519999999999999e-06 2024-09-13T07:02:17.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005630+0000", 2024-09-13T07:02:17.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9440000000000001e-06 2024-09-13T07:02:17.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005633+0000", 2024-09-13T07:02:17.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7999999999999999e-06 2024-09-13T07:02:17.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031305+0000", 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025672513000000001 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031315+0000", 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7620000000000007e-06 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62269 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005635+0000", 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7963605440000006, 2024-09-13T07:02:17.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025697040000000001, 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62269 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005635+0000", 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005635+0000", 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005635+0000", 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005637+0000", 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6899999999999999e-06 2024-09-13T07:02:17.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005640+0000", 2024-09-13T07:02:17.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7069999999999998e-06 2024-09-13T07:02:17.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005644+0000", 2024-09-13T07:02:17.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1069999999999998e-06 2024-09-13T07:02:17.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031327+0000", 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025682987000000001 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031332+0000", 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7539999999999997e-06 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.878 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62270 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005647+0000", 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.796349051, 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025711939, 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62270 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005647+0000", 2024-09-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005647+0000", 2024-09-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005646+0000", 2024-09-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005648+0000", 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.575e-06 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005651+0000", 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9009999999999998e-06 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005654+0000", 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7020000000000001e-06 2024-09-13T07:02:17.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031349+0000", 2024-09-13T07:02:17.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025695246000000001 2024-09-13T07:02:17.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031359+0000", 2024-09-13T07:02:17.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7259999999999997e-06 2024-09-13T07:02:17.882 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.882 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.882 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62271 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005769+0000", 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7962269069999994, 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025610022, 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62271 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005769+0000", 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005769+0000", 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005655+0000", 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9998865 2024-09-13T07:02:17.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005773+0000", 2024-09-13T07:02:17.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117217 2024-09-13T07:02:17.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005781+0000", 2024-09-13T07:02:17.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.507e-06 2024-09-13T07:02:17.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005789+0000", 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5190000000000003e-06 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031372+0000", 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025583155999999999 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031379+0000", 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8789999999999997e-06 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62272 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005795+0000", 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7962008279999999, 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025624159000000001, 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62272 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005795+0000", 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005795+0000", 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005794+0000", 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005798+0000", 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1480000000000002e-06 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005805+0000", 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4669999999999998e-06 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005810+0000", 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2519999999999999e-06 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031407+0000", 2024-09-13T07:02:17.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025596900999999998 2024-09-13T07:02:17.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031419+0000", 2024-09-13T07:02:17.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1807000000000001e-05 2024-09-13T07:02:17.890 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.890 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.890 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.891 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:76024 3.18 3:1ec5831f:::rbd_data.1194685901c0.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T07:02:17.891 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:16.744033+0000", 2024-09-13T07:02:17.891 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.0579622099999999, 2024-09-13T07:02:17.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041308280000000003, 2024-09-13T07:02:17.891 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:17.891 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:17.891 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:17.891 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:17.891 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 76024 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744033+0000", 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744033+0000", 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744032+0000", 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744043+0000", 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1739000000000001e-05 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744065+0000", 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1189999999999999e-05 2024-09-13T07:02:17.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:17.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744075+0000", 2024-09-13T07:02:17.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0550000000000001e-05 2024-09-13T07:02:17.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:17.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744132+0000", 2024-09-13T07:02:17.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6604999999999998e-05 2024-09-13T07:02:17.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:17.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744210+0000", 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7862999999999996e-05 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.751836+0000", 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076268289999999999 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.785260+0000", 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033423184000000002 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:17.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.785272+0000", 2024-09-13T07:02:17.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1973e-05 2024-09-13T07:02:17.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:17.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.785311+0000", 2024-09-13T07:02:17.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9928999999999997e-05 2024-09-13T07:02:17.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:17.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:17.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:17.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.785342+0000", 2024-09-13T07:02:17.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0190000000000001e-05 2024-09-13T07:02:17.896 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.896 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.897 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.897 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:17.897 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:17.897 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:17.898 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:17.894+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:17.899 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:17.999 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:17.998+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:18.026 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:18.026 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:18.026 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:18.027 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:18.038 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:18.057 INFO:teuthology.orchestra.run.smithi118.stdout:2101 read 0x55de2bc thru 0x55ec562 (0xe2a7 bytes) 2024-09-13T07:02:18.059 INFO:teuthology.orchestra.run.smithi118.stdout:2102 read 0x3e6c3ca thru 0x3e734e5 (0x711c bytes) 2024-09-13T07:02:18.061 INFO:teuthology.orchestra.run.smithi118.stdout:2103 read 0x452468e thru 0x452addd (0x6750 bytes) 2024-09-13T07:02:18.062 INFO:teuthology.orchestra.run.smithi118.stdout:2104 write 0xfed999 thru 0xff1626 (0x3c8e bytes) 2024-09-13T07:02:18.087 INFO:teuthology.orchestra.run.smithi118.stdout:2105 write 0x38012cf thru 0x380a7c2 (0x94f4 bytes) 2024-09-13T07:02:18.100 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:18.098+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:18.100 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:18.101 INFO:teuthology.orchestra.run.smithi118.stdout:2107 write 0xabaf40f thru 0xabb72f6 (0x7ee8 bytes) 2024-09-13T07:02:18.113 INFO:teuthology.orchestra.run.smithi118.stdout:2108 writesame 0x767cd2 thru 0x772757 data_size 0x49(0xaa86 bytes) 2024-09-13T07:02:18.128 INFO:teuthology.orchestra.run.smithi118.stdout:2110 writesame 0x2460fb7 thru 0x246abc2 data_size 0x6(0x9c0c bytes) 2024-09-13T07:02:18.143 INFO:teuthology.orchestra.run.smithi118.stdout:2111 write 0xb348954 thru 0xb3545c5 (0xbc72 bytes) 2024-09-13T07:02:18.156 INFO:teuthology.orchestra.run.smithi118.stdout:2113 read 0x4e2d492 thru 0x4e37046 (0x9bb5 bytes) 2024-09-13T07:02:18.156 INFO:teuthology.orchestra.run.smithi118.stdout:2114 punch from 0x212d930 to 0x213d129, (0xf7f9 bytes) 2024-09-13T07:02:18.159 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:18.160 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:18.170 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:18.170 INFO:teuthology.orchestra.run.smithi118.stdout:2116 writesame 0xe5f85cf thru 0xe603f58 data_size 0x2(0xb98a bytes) 2024-09-13T07:02:18.184 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:18.184 INFO:teuthology.orchestra.run.smithi118.stdout:2119 write 0x761f7ad thru 0x762ae39 (0xb68d bytes) 2024-09-13T07:02:18.198 INFO:teuthology.orchestra.run.smithi118.stdout:2120 write 0x176df8f thru 0x1779265 (0xb2d7 bytes) 2024-09-13T07:02:18.200 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:18.200 INFO:teuthology.orchestra.run.smithi118.stdout:2122 write 0x5a25e44 thru 0x5a34ac0 (0xec7d bytes) 2024-09-13T07:02:18.202 INFO:teuthology.orchestra.run.smithi118.stdout:2125 write 0x3790487 thru 0x3798a63 (0x85dd bytes) 2024-09-13T07:02:18.205 INFO:teuthology.orchestra.run.smithi118.stdout:2126 read 0x8d04ae9 thru 0x8d0fd48 (0xb260 bytes) 2024-09-13T07:02:18.205 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:18.205 INFO:teuthology.orchestra.run.smithi118.stdout:2128 write 0xc10f110 thru 0xc11a5eb (0xb4dc bytes) 2024-09-13T07:02:18.208 INFO:teuthology.orchestra.run.smithi118.stdout:2130 read 0x2fba6cb thru 0x2fbbee2 (0x1818 bytes) 2024-09-13T07:02:18.211 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:18.210+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:18.213 INFO:teuthology.orchestra.run.smithi118.stdout:2131 trunc from 0xe603f59 to 0xc960b00 2024-09-13T07:02:18.240 INFO:teuthology.orchestra.run.smithi118.stdout:2133 punch from 0xadc7111 to 0xadd396f, (0xc85e bytes) 2024-09-13T07:02:18.240 INFO:teuthology.orchestra.run.smithi118.stdout:2134 write 0x83c46ab thru 0x83cdb65 (0x94bb bytes) 2024-09-13T07:02:18.241 INFO:teuthology.orchestra.run.smithi118.stdout:2137 read 0x3e63675 thru 0x3e70b9f (0xd52b bytes) 2024-09-13T07:02:18.242 INFO:teuthology.orchestra.run.smithi118.stdout:2139 read 0x13ea165 thru 0x13f82a1 (0xe13d bytes) 2024-09-13T07:02:18.243 INFO:teuthology.orchestra.run.smithi118.stdout:2140 punch from 0x20ccc61 to 0x20cef57, (0x22f6 bytes) 2024-09-13T07:02:18.243 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:18.243 INFO:teuthology.orchestra.run.smithi118.stdout:2144 writesame 0x6494e99 thru 0x6499708 data_size 0x8(0x4870 bytes) 2024-09-13T07:02:18.245 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:18.255 INFO:teuthology.orchestra.run.smithi118.stdout:2146 write 0xd085c3b thru 0xd088cbc (0x3082 bytes) 2024-09-13T07:02:18.257 INFO:teuthology.orchestra.run.smithi118.stdout:2147 punch from 0x214945f to 0x2149b69, (0x70a bytes) 2024-09-13T07:02:18.258 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:02:18.261 INFO:teuthology.orchestra.run.smithi118.stdout:2148 trunc from 0xd088cbd to 0xe82e398 2024-09-13T07:02:18.272 INFO:teuthology.orchestra.run.smithi118.stdout:2149 read 0xd5257e0 thru 0xd52ef42 (0x9763 bytes) 2024-09-13T07:02:18.273 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:18.273 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:18.273 INFO:teuthology.orchestra.run.smithi118.stdout:2153 write 0x800b48c thru 0x80165c0 (0xb135 bytes) 2024-09-13T07:02:18.276 INFO:teuthology.orchestra.run.smithi118.stdout:2154 writesame 0x99d5ac8 thru 0x99d6973 data_size 0x6(0xeac bytes) 2024-09-13T07:02:18.279 INFO:teuthology.orchestra.run.smithi118.stdout:2155 read 0x1de469e thru 0x1df2178 (0xdadb bytes) 2024-09-13T07:02:18.280 INFO:teuthology.orchestra.run.smithi118.stdout:2156 clone 23 order 21 su 16384 sc 14 2024-09-13T07:02:18.293 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:02:18.301 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:18.298+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:18.321 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:18.321 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:18.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:18.321 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:18.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.321 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1524 3.14 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:18.321 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:18.321 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.275252537999997, 2024-09-13T07:02:18.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061190880000000003, 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1524 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:18.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044767+0000", 2024-09-13T07:02:18.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:18.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052670+0000", 2024-09-13T07:02:18.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079022450000000004 2024-09-13T07:02:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052682+0000", 2024-09-13T07:02:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2336e-05 2024-09-13T07:02:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052696+0000", 2024-09-13T07:02:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-13T07:02:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052755+0000", 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9298e-05 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052947+0000", 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019153900000000001 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.053694+0000", 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074663099999999999 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.102148+0000", 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048454329999999997 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105816+0000", 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036675950000000001 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105921+0000", 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105874 2024-09-13T07:02:18.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105959+0000", 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7991000000000003e-05 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19083 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3817472~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870786727999999, 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049218062999999999, 2024-09-13T07:02:18.327 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.328 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.328 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19083 2024-09-13T07:02:18.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.328 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:18.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:18.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:18.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:18.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449236+0000", 2024-09-13T07:02:18.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.502e-06 2024-09-13T07:02:18.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449238+0000", 2024-09-13T07:02:18.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.689e-06 2024-09-13T07:02:18.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449241+0000", 2024-09-13T07:02:18.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5119999999999998e-06 2024-09-13T07:02:18.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498359+0000", 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049117630000000002 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498399+0000", 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0726000000000003e-05 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498452+0000", 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3220999999999997e-05 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.331 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.332 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19084 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3465216~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.332 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:18.332 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870778017999999, 2024-09-13T07:02:18.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049328261999999998, 2024-09-13T07:02:18.332 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.332 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.332 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19084 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:18.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:18.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449244+0000", 2024-09-13T07:02:18.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.454e-06 2024-09-13T07:02:18.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449247+0000", 2024-09-13T07:02:18.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.717e-06 2024-09-13T07:02:18.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449249+0000", 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.424e-06 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498477+0000", 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049227235000000001 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498518+0000", 2024-09-13T07:02:18.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0846000000000002e-05 2024-09-13T07:02:18.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498571+0000", 2024-09-13T07:02:18.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3810000000000001e-05 2024-09-13T07:02:18.336 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.336 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.336 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.336 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19085 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3842048~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.336 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449252+0000", 2024-09-13T07:02:18.336 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870769405000001, 2024-09-13T07:02:18.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049435828000000001, 2024-09-13T07:02:18.337 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.337 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.337 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19085 2024-09-13T07:02:18.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.337 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449252+0000", 2024-09-13T07:02:18.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449252+0000", 2024-09-13T07:02:18.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449251+0000", 2024-09-13T07:02:18.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:18.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449253+0000", 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4389999999999999e-06 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449256+0000", 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449258+0000", 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.435e-06 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498594+0000", 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049335639000000001 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498635+0000", 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1229000000000001e-05 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498688+0000", 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2555000000000001e-05 2024-09-13T07:02:18.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.341 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.341 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19086 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3489792~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.341 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:18.341 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870760748999999, 2024-09-13T07:02:18.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049540534999999997, 2024-09-13T07:02:18.341 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.341 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.342 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19086 2024-09-13T07:02:18.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.342 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:18.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:18.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:18.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:18.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449269+0000", 2024-09-13T07:02:18.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6459999999999999e-06 2024-09-13T07:02:18.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449273+0000", 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9469999999999996e-06 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449276+0000", 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498707+0000", 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049431286999999997 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498748+0000", 2024-09-13T07:02:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0976999999999997e-05 2024-09-13T07:02:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498801+0000", 2024-09-13T07:02:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3090000000000002e-05 2024-09-13T07:02:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.345 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19087 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3866624~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870742918000001, 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049644714999999999, 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19087 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449280+0000", 2024-09-13T07:02:18.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.655e-06 2024-09-13T07:02:18.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449282+0000", 2024-09-13T07:02:18.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8600000000000001e-06 2024-09-13T07:02:18.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449285+0000", 2024-09-13T07:02:18.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4600000000000002e-06 2024-09-13T07:02:18.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498819+0000", 2024-09-13T07:02:18.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049534340000000003 2024-09-13T07:02:18.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498871+0000", 2024-09-13T07:02:18.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1412000000000002e-05 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498923+0000", 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2252e-05 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19088 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3514368~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870734007999999, 2024-09-13T07:02:18.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049750612999999999, 2024-09-13T07:02:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19088 2024-09-13T07:02:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449288+0000", 2024-09-13T07:02:18.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.469e-06 2024-09-13T07:02:18.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449291+0000", 2024-09-13T07:02:18.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:02:18.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449294+0000", 2024-09-13T07:02:18.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5069999999999999e-06 2024-09-13T07:02:18.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498941+0000", 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049647494 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498983+0000", 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1771999999999999e-05 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499038+0000", 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4888e-05 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.354 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19089 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3891200~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870725274000002, 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049857261, 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19089 2024-09-13T07:02:18.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.356 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:18.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:18.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449297+0000", 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4330000000000001e-06 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449301+0000", 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5260000000000002e-06 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449303+0000", 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5110000000000002e-06 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499050+0000", 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049747121999999998 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499091+0000", 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0580000000000001e-05 2024-09-13T07:02:18.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499153+0000", 2024-09-13T07:02:18.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2305999999999996e-05 2024-09-13T07:02:18.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.359 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.360 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19090 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3538944~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.360 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:18.360 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870715790999999, 2024-09-13T07:02:18.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049959569000000002, 2024-09-13T07:02:18.360 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.360 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.360 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19090 2024-09-13T07:02:18.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.360 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449307+0000", 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4279999999999999e-06 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449309+0000", 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.706e-06 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449312+0000", 2024-09-13T07:02:18.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4930000000000001e-06 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499169+0000", 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049857745000000002 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499210+0000", 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0706e-05 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499265+0000", 2024-09-13T07:02:18.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4719000000000002e-05 2024-09-13T07:02:18.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.364 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.364 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19091 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3915776~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.364 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:18.364 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870707149000001, 2024-09-13T07:02:18.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050065959, 2024-09-13T07:02:18.364 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.364 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.364 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.364 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.365 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19091 2024-09-13T07:02:18.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.365 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:18.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:18.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449315+0000", 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.485e-06 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449318+0000", 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6740000000000001e-06 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449320+0000", 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.413e-06 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499283+0000", 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049962196 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499323+0000", 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0799e-05 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499380+0000", 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6610999999999997e-05 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19092 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3563520~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449323+0000", 2024-09-13T07:02:18.368 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870698458, 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050176212999999997, 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19092 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449323+0000", 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449323+0000", 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449322+0000", 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449324+0000", 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.424e-06 2024-09-13T07:02:18.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449327+0000", 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449329+0000", 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4820000000000001e-06 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499397+0000", 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050068362999999998 2024-09-13T07:02:18.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499441+0000", 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3668999999999998e-05 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499499+0000", 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7778999999999997e-05 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.373 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19093 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3940352~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.373 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:18.373 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870689675000001, 2024-09-13T07:02:18.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050278704, 2024-09-13T07:02:18.373 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.373 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.373 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19093 2024-09-13T07:02:18.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.373 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449333+0000", 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5570000000000001e-06 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449335+0000", 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449338+0000", 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.491e-06 2024-09-13T07:02:18.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499513+0000", 2024-09-13T07:02:18.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050175143999999998 2024-09-13T07:02:18.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499554+0000", 2024-09-13T07:02:18.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0729000000000003e-05 2024-09-13T07:02:18.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499610+0000", 2024-09-13T07:02:18.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6307000000000001e-05 2024-09-13T07:02:18.377 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.377 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.377 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.377 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19094 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3588096~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.377 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:18.377 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870680772, 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050385554999999999, 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19094 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449342+0000", 2024-09-13T07:02:18.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.652e-06 2024-09-13T07:02:18.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449345+0000", 2024-09-13T07:02:18.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:02:18.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449348+0000", 2024-09-13T07:02:18.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5849999999999998e-06 2024-09-13T07:02:18.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499626+0000", 2024-09-13T07:02:18.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050278728000000002 2024-09-13T07:02:18.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499669+0000", 2024-09-13T07:02:18.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2400000000000001e-05 2024-09-13T07:02:18.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499726+0000", 2024-09-13T07:02:18.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7191999999999997e-05 2024-09-13T07:02:18.382 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.382 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.382 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.382 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19095 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3964928~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.382 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:18.382 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870671060999999, 2024-09-13T07:02:18.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050492171000000002, 2024-09-13T07:02:18.382 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.382 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19095 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449351+0000", 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.449e-06 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449354+0000", 2024-09-13T07:02:18.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.836e-06 2024-09-13T07:02:18.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449357+0000", 2024-09-13T07:02:18.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5459999999999998e-06 2024-09-13T07:02:18.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499744+0000", 2024-09-13T07:02:18.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050387339000000003 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499786+0000", 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2293999999999999e-05 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499842+0000", 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5927000000000002e-05 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.387 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19096 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3612672~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.387 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:18.387 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.870661991999999, 2024-09-13T07:02:18.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050599058000000002, 2024-09-13T07:02:18.387 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.387 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.387 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.387 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.387 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.387 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19096 2024-09-13T07:02:18.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.387 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:18.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449360+0000", 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4529999999999999e-06 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449363+0000", 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6680000000000001e-06 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449366+0000", 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6359999999999998e-06 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499860+0000", 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050494820000000003 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499902+0000", 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1111e-05 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499958+0000", 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6594000000000001e-05 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19110 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 4186112~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.867538642, 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049302258000000002, 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19110 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:18.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452487+0000", 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9980000000000003e-06 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452490+0000", 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1410000000000001e-06 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452493+0000", 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6010000000000002e-06 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501587+0000", 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049094282000000003 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501680+0000", 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2675000000000003e-05 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501785+0000", 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000104993 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19111 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3760128~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.395 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:18.396 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.86752598, 2024-09-13T07:02:18.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049401467999999997, 2024-09-13T07:02:18.396 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.396 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.396 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.396 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.396 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.396 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19111 2024-09-13T07:02:18.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.396 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452497+0000", 2024-09-13T07:02:18.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7099999999999999e-06 2024-09-13T07:02:18.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452499+0000", 2024-09-13T07:02:18.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6929999999999999e-06 2024-09-13T07:02:18.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452502+0000", 2024-09-13T07:02:18.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5900000000000002e-06 2024-09-13T07:02:18.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501801+0000", 2024-09-13T07:02:18.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049299005 2024-09-13T07:02:18.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501838+0000", 2024-09-13T07:02:18.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7282e-05 2024-09-13T07:02:18.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501897+0000", 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8405000000000001e-05 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19112 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3784704~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.867516841, 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049618640999999998, 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.400 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19112 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452505+0000", 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.364e-06 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452508+0000", 2024-09-13T07:02:18.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7470000000000001e-06 2024-09-13T07:02:18.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452511+0000", 2024-09-13T07:02:18.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-13T07:02:18.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501932+0000", 2024-09-13T07:02:18.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049421395999999999 2024-09-13T07:02:18.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502019+0000", 2024-09-13T07:02:18.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7247e-05 2024-09-13T07:02:18.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502123+0000", 2024-09-13T07:02:18.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103589 2024-09-13T07:02:18.404 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.404 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.404 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.404 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19113 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3809280~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.867507816, 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049709729000000001, 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19113 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452515+0000", 2024-09-13T07:02:18.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659999999999999e-06 2024-09-13T07:02:18.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452517+0000", 2024-09-13T07:02:18.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6189999999999998e-06 2024-09-13T07:02:18.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452520+0000", 2024-09-13T07:02:18.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7760000000000002e-06 2024-09-13T07:02:18.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502134+0000", 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049614232000000001 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502173+0000", 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8486999999999999e-05 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502223+0000", 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0392999999999997e-05 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.408 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:48369 3.f e58/22 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head v 58'78, mlcod=57'75)", 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.079554673000001, 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.057439072000000001, 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:18.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240464+0000", 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245716+0000", 2024-09-13T07:02:18.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005252329 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245745+0000", 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8844e-05 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245758+0000", 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2242000000000001e-05 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245849+0000", 2024-09-13T07:02:18.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1187999999999995e-05 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245898+0000", 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8881000000000002e-05 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297837+0000", 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051938947999999999 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:18.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297906+0000", 2024-09-13T07:02:18.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8955999999999995e-05 2024-09-13T07:02:18.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.413 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:18.413 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:18.413 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:18.414 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:18.415 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:18.415 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:18.413+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:18.424 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:18.445 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:18.464 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:18.485 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:18.507 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:02:18.507 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:02:18.508 INFO:tasks.thrashosds.thrasher:Reweighting osd 1 to 0.184435573226338 2024-09-13T07:02:18.508 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 0.184435573226338 2024-09-13T07:02:18.509 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:18.514 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:18.513+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:18.525 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:18.525 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:18.531 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:18.535 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:18.552 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:18.554 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:18.554 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:18.555 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:18.555 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:18.555 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:18.555 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:18.555 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:18.555 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:18.616 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:18.613+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:18.637 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:18.637 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:18.647 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:18.717 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:18.714+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:18.754 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:18.754 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:18.763 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:18.817 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:18.817+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:18.870 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:18.870 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:18.880 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:18.919 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:18.918+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:18.988 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:18.988 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:18.997 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:19.019 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:19.017+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:19.120 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:19.118+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:19.149 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:19.149 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:19.158 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:19.221 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:19.218+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:19.269 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:19.269 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:19.278 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:19.304 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.1 to 0.184436 (2f37) 2024-09-13T07:02:19.322 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:19.318+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:19.385 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:19.385 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:19.395 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:19.439 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:19.438+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:19.523 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:19.522+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:19.548 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:19.548 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:19.557 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:19.624 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:19.622+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:19.675 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:19.675 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:19.685 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:19.725 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:19.725+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:19.787 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:19.787 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:19.797 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:19.845 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:19.845+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:19.901 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:19.901 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:19.910 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:19.927 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:19.925+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:20.015 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:20.015 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:20.024 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:20.069 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:20.069+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:20.127 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:20.128 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:20.128 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:20.125+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:20.137 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:20.229 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:20.226+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:20.246 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:20.246 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:20.256 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:20.306 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:02:20.329 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:20.329+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:20.371 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:20.372 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:20.382 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:20.430 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:20.428+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:20.480 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:20.480 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:20.490 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:20.532 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:20.530+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:20.566 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:02:20.632 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:20.630+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:20.639 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:20.639 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:20.649 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:20.711 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:20.730 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:20.733 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:20.730+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:20.752 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:20.752 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:20.757 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:20.762 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:20.785 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:20.807 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:20.833 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:20.832+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:20.834 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:20.861 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:20.861 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:20.861 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:20.861 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:20.862 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:20.865 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:20.871 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:20.886 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:20.889 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:20.889 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:20.889 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:20.889 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:20.889 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:20.889 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:20.889 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:20.889 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:20.935 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:20.930+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:20.973 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:20.973 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:20.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:20.973 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:20.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.973 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1531 3.c e27/22 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head v 27'9, mlcod=27'7)", 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.898986473000001, 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.039127038000000003, 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:20.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074531+0000", 2024-09-13T07:02:20.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:20.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:20.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.074530+0000", 2024-09-13T07:02:20.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:20.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:20.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099434+0000", 2024-09-13T07:02:20.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.024903926999999999 2024-09-13T07:02:20.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:20.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099459+0000", 2024-09-13T07:02:20.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5384999999999998e-05 2024-09-13T07:02:20.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:20.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099469+0000", 2024-09-13T07:02:20.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.024e-05 2024-09-13T07:02:20.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:20.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099566+0000", 2024-09-13T07:02:20.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6533999999999994e-05 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099600+0000", 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4007000000000002e-05 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113477+0000", 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01387741 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.113658+0000", 2024-09-13T07:02:20.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000180953 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12393 3.b 3:d3c3e30e:::rbd_data.11945fc173dd.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 40.701053539999997, 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027522214, 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:20.978 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12393 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272464+0000", 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:20.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272462+0000", 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272471+0000", 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8510000000000005e-06 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272486+0000", 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5409000000000001e-05 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:20.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272494+0000", 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0020000000000006e-06 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272575+0000", 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0467000000000001e-05 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.272710+0000", 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000135554 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 0", 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.279539+0000", 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068285439999999998 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299872+0000", 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.020332507 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299920+0000", 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8009000000000001e-05 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299956+0000", 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6554999999999998e-05 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.299986+0000", 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0331000000000001e-05 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:48366 3.2 3:4282dbac:::rbd_data.1194fc792839.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:20.983 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:58.185470+0000", 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.788047788, 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.038773378999999997, 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 48366 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.185470+0000", 2024-09-13T07:02:20.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.185470+0000", 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.185467+0000", 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:20.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192092+0000", 2024-09-13T07:02:20.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0066248230000000002 2024-09-13T07:02:20.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:20.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192114+0000", 2024-09-13T07:02:20.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1685999999999999e-05 2024-09-13T07:02:20.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:20.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192122+0000", 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4989999999999994e-06 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192208+0000", 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5995000000000006e-05 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.192454+0000", 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024575200000000001 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:20.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.193274+0000", 2024-09-13T07:02:20.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081968299999999998 2024-09-13T07:02:20.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:20.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.212795+0000", 2024-09-13T07:02:20.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019520991000000001 2024-09-13T07:02:20.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:20.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.224086+0000", 2024-09-13T07:02:20.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011291539 2024-09-13T07:02:20.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:20.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.224184+0000", 2024-09-13T07:02:20.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7850000000000007e-05 2024-09-13T07:02:20.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:20.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.224243+0000", 2024-09-13T07:02:20.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9200999999999998e-05 2024-09-13T07:02:20.990 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:20.990 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:20.990 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:20.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.990 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62027 3.18 3:195cb5a2:::rbd_data.1194987e4741.0000000000000003:17 [read 3491840~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:20.990 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.004536+0000", 2024-09-13T07:02:20.990 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.968981835999999, 2024-09-13T07:02:20.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025634061, 2024-09-13T07:02:20.990 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:20.990 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:20.990 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:20.990 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:20.991 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:20.991 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62027 2024-09-13T07:02:20.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.991 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:20.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:20.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004536+0000", 2024-09-13T07:02:20.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:20.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:20.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004536+0000", 2024-09-13T07:02:20.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:20.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:20.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004535+0000", 2024-09-13T07:02:20.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:20.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:20.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004538+0000", 2024-09-13T07:02:20.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.971e-06 2024-09-13T07:02:20.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:20.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004545+0000", 2024-09-13T07:02:20.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3629999999999999e-06 2024-09-13T07:02:20.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:20.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004549+0000", 2024-09-13T07:02:20.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9400000000000001e-06 2024-09-13T07:02:20.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:20.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030001+0000", 2024-09-13T07:02:20.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025451613000000001 2024-09-13T07:02:20.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:20.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030074+0000", 2024-09-13T07:02:20.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2525000000000001e-05 2024-09-13T07:02:20.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:20.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030170+0000", 2024-09-13T07:02:20.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6383000000000002e-05 2024-09-13T07:02:20.995 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:20.995 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:20.995 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:20.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.996 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62028 3.18 3:195cb5a2:::rbd_data.1194987e4741.0000000000000003:17 [read 3495936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:20.996 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.004555+0000", 2024-09-13T07:02:20.996 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.968963011, 2024-09-13T07:02:20.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025812420999999999, 2024-09-13T07:02:20.996 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:20.996 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:20.996 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:20.996 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:20.997 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:20.997 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62028 2024-09-13T07:02:20.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.997 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:20.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:20.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004555+0000", 2024-09-13T07:02:20.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:20.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:20.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004555+0000", 2024-09-13T07:02:20.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:20.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:20.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004554+0000", 2024-09-13T07:02:20.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:20.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:20.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004557+0000", 2024-09-13T07:02:20.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.033e-06 2024-09-13T07:02:20.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:20.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:20.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:20.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004563+0000", 2024-09-13T07:02:20.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:02:20.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004567+0000", 2024-09-13T07:02:21.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.228e-06 2024-09-13T07:02:21.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030212+0000", 2024-09-13T07:02:21.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025644851 2024-09-13T07:02:21.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030286+0000", 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3973000000000005e-05 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030367+0000", 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0889000000000004e-05 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62029 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.004570+0000", 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.968947433, 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025821687999999999, 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62029 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004570+0000", 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004570+0000", 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004570+0000", 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004573+0000", 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2660000000000002e-06 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004578+0000", 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7859999999999996e-06 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004583+0000", 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5469999999999998e-06 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030386+0000", 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025803578000000001 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030392+0000", 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9089999999999997e-06 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.005 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62030 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.004586+0000", 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.968931666, 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025834207000000001, 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62030 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004586+0000", 2024-09-13T07:02:21.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004586+0000", 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004586+0000", 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004588+0000", 2024-09-13T07:02:21.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1749999999999999e-06 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004593+0000", 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0590000000000002e-06 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.004597+0000", 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3159999999999998e-06 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030410+0000", 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025812630999999999 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.030420+0000", 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0321e-05 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62261 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005557+0000", 2024-09-13T07:02:21.009 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.967960647, 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025596205, 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62261 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005557+0000", 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005557+0000", 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005557+0000", 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005558+0000", 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.328e-06 2024-09-13T07:02:21.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005561+0000", 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8140000000000001e-06 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005564+0000", 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.475e-06 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031148+0000", 2024-09-13T07:02:21.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025584928999999999 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031153+0000", 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8640000000000004e-06 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62262 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005568+0000", 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.967949559999999, 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025612378000000002, 2024-09-13T07:02:21.013 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62262 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005568+0000", 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005568+0000", 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005568+0000", 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005569+0000", 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.141e-06 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005572+0000", 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8229999999999999e-06 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005574+0000", 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5009999999999999e-06 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031171+0000", 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025596071000000001 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031181+0000", 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0040999999999999e-05 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62263 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005577+0000", 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.967941262, 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025620249000000001, 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.017 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62263 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005577+0000", 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005577+0000", 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005576+0000", 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005578+0000", 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.37e-06 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005581+0000", 2024-09-13T07:02:21.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8459999999999999e-06 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005584+0000", 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4920000000000002e-06 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031191+0000", 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025607396000000001 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031197+0000", 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62264 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005587+0000", 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.967930772999999, 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025636381, 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62264 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005587+0000", 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005587+0000", 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005587+0000", 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005588+0000", 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1680000000000001e-06 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005591+0000", 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8909999999999999e-06 2024-09-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005593+0000", 2024-09-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6539999999999999e-06 2024-09-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031214+0000", 2024-09-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025620165 2024-09-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031223+0000", 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7070000000000004e-06 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62265 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005596+0000", 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.967922119000001, 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025644693999999999, 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62265 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005596+0000", 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005596+0000", 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005595+0000", 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005598+0000", 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.999e-06 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005602+0000", 2024-09-13T07:02:21.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.157e-06 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005606+0000", 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9260000000000002e-06 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031235+0000", 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025629880000000001 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031240+0000", 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9409999999999998e-06 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62266 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005608+0000", 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.967909926000001, 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025664411000000002, 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.029 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62266 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005608+0000", 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005608+0000", 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005608+0000", 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005609+0000", 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.24e-06 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005612+0000", 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005614+0000", 2024-09-13T07:02:21.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6960000000000001e-06 2024-09-13T07:02:21.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031262+0000", 2024-09-13T07:02:21.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025647463999999998 2024-09-13T07:02:21.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031272+0000", 2024-09-13T07:02:21.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0379e-05 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62267 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005617+0000", 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.967901179, 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025672828000000002, 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62267 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005617+0000", 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005617+0000", 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005616+0000", 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005618+0000", 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3549999999999999e-06 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005621+0000", 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.791e-06 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005623+0000", 2024-09-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6850000000000001e-06 2024-09-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031284+0000", 2024-09-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025660867 2024-09-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031289+0000", 2024-09-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999999e-06 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62268 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005626+0000", 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.967891681999999, 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025689170000000001, 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62268 2024-09-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005626+0000", 2024-09-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005626+0000", 2024-09-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005626+0000", 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005627+0000", 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3519999999999999e-06 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005630+0000", 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9440000000000001e-06 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005633+0000", 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7999999999999999e-06 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031305+0000", 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025672513000000001 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031315+0000", 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7620000000000007e-06 2024-09-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62269 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005635+0000", 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.967882693, 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025697040000000001, 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62269 2024-09-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005635+0000", 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005635+0000", 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005635+0000", 2024-09-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005637+0000", 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6899999999999999e-06 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005640+0000", 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7069999999999998e-06 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005644+0000", 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1069999999999998e-06 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031327+0000", 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025682987000000001 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031332+0000", 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7539999999999997e-06 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62270 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005647+0000", 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.967871199999999, 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025711939, 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62270 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005647+0000", 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005647+0000", 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005646+0000", 2024-09-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005648+0000", 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.575e-06 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005651+0000", 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9009999999999998e-06 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005654+0000", 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7020000000000001e-06 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031349+0000", 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025695246000000001 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031359+0000", 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7259999999999997e-06 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62271 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005769+0000", 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.967749056000001, 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025610022, 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62271 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005769+0000", 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005769+0000", 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005655+0000", 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9998865 2024-09-13T07:02:21.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005773+0000", 2024-09-13T07:02:21.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000117217 2024-09-13T07:02:21.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005781+0000", 2024-09-13T07:02:21.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.507e-06 2024-09-13T07:02:21.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005789+0000", 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5190000000000003e-06 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031372+0000", 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025583155999999999 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031379+0000", 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8789999999999997e-06 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.052 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.053 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:62272 3.18 3.45ad3a98 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-13T07:02:21.053 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:08.005795+0000", 2024-09-13T07:02:21.053 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.967722976999999, 2024-09-13T07:02:21.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025624159000000001, 2024-09-13T07:02:21.053 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.053 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:21.053 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.053 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.053 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.053 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 62272 2024-09-13T07:02:21.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005795+0000", 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005795+0000", 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005794+0000", 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005798+0000", 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1480000000000002e-06 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005805+0000", 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4669999999999998e-06 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.005810+0000", 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2519999999999999e-06 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031407+0000", 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025596900999999998 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:08.031419+0000", 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1807000000000001e-05 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.056 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:76024 3.18 3:1ec5831f:::rbd_data.1194685901c0.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:16.744033+0000", 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.2294843589999997, 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041308280000000003, 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 76024 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.057 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:21.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:21.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744033+0000", 2024-09-13T07:02:21.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:21.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744033+0000", 2024-09-13T07:02:21.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:21.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:21.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744032+0000", 2024-09-13T07:02:21.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:21.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:21.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744043+0000", 2024-09-13T07:02:21.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1739000000000001e-05 2024-09-13T07:02:21.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:21.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744065+0000", 2024-09-13T07:02:21.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1189999999999999e-05 2024-09-13T07:02:21.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744075+0000", 2024-09-13T07:02:21.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0550000000000001e-05 2024-09-13T07:02:21.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:21.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744132+0000", 2024-09-13T07:02:21.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6604999999999998e-05 2024-09-13T07:02:21.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:21.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.744210+0000", 2024-09-13T07:02:21.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7862999999999996e-05 2024-09-13T07:02:21.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 1", 2024-09-13T07:02:21.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.751836+0000", 2024-09-13T07:02:21.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0076268289999999999 2024-09-13T07:02:21.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:21.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.785260+0000", 2024-09-13T07:02:21.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033423184000000002 2024-09-13T07:02:21.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:21.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.785272+0000", 2024-09-13T07:02:21.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1973e-05 2024-09-13T07:02:21.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:21.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.785311+0000", 2024-09-13T07:02:21.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9928999999999997e-05 2024-09-13T07:02:21.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:21.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:21.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:21.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:16.785342+0000", 2024-09-13T07:02:21.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0190000000000001e-05 2024-09-13T07:02:21.063 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.063 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.063 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.063 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:21.063 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:21.063 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:21.065 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:21.066 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:21.064+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:21.166 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:21.164+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:21.182 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:21.182 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:21.192 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:21.267 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:21.266+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:21.295 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:21.295 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:21.295 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:21.295 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:21.295 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:21.305 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:21.310 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:02:21.368 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:21.364+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:21.423 INFO:teuthology.orchestra.run.smithi118.stdout:flattening image image_client.0-clone22 2024-09-13T07:02:21.443 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:21.443 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:21.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:21.443 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:21.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.443 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1524 3.14 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:21.443 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:21.443 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.397151073000003, 2024-09-13T07:02:21.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061190880000000003, 2024-09-13T07:02:21.444 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.444 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:21.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.444 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1524 2024-09-13T07:02:21.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.444 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:21.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044767+0000", 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052670+0000", 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079022450000000004 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052682+0000", 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2336e-05 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052696+0000", 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052755+0000", 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9298e-05 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052947+0000", 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019153900000000001 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.053694+0000", 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074663099999999999 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.102148+0000", 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048454329999999997 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105816+0000", 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036675950000000001 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105921+0000", 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105874 2024-09-13T07:02:21.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105959+0000", 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7991000000000003e-05 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19083 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3817472~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.992685262999998, 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049218062999999999, 2024-09-13T07:02:21.449 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19083 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449234+0000", 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449236+0000", 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.502e-06 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449238+0000", 2024-09-13T07:02:21.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.689e-06 2024-09-13T07:02:21.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449241+0000", 2024-09-13T07:02:21.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5119999999999998e-06 2024-09-13T07:02:21.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498359+0000", 2024-09-13T07:02:21.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049117630000000002 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498399+0000", 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0726000000000003e-05 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498452+0000", 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3220999999999997e-05 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19084 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3465216~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.992676552999999, 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049328261999999998, 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19084 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449243+0000", 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449244+0000", 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.454e-06 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449247+0000", 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.717e-06 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449249+0000", 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.424e-06 2024-09-13T07:02:21.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498477+0000", 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049227235000000001 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498518+0000", 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0846000000000002e-05 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498571+0000", 2024-09-13T07:02:21.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3810000000000001e-05 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19085 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3842048~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449252+0000", 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.99266794, 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049435828000000001, 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.458 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19085 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449252+0000", 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449252+0000", 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449251+0000", 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449253+0000", 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4389999999999999e-06 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449256+0000", 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449258+0000", 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.435e-06 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498594+0000", 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049335639000000001 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498635+0000", 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1229000000000001e-05 2024-09-13T07:02:21.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498688+0000", 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2555000000000001e-05 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19086 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3489792~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.992659283999998, 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049540534999999997, 2024-09-13T07:02:21.462 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19086 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449260+0000", 2024-09-13T07:02:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449269+0000", 2024-09-13T07:02:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6459999999999999e-06 2024-09-13T07:02:21.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449273+0000", 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9469999999999996e-06 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449276+0000", 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498707+0000", 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049431286999999997 2024-09-13T07:02:21.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498748+0000", 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0976999999999997e-05 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498801+0000", 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3090000000000002e-05 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19087 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3866624~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.466 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.992641453000001, 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049644714999999999, 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19087 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449278+0000", 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449280+0000", 2024-09-13T07:02:21.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.655e-06 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449282+0000", 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8600000000000001e-06 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449285+0000", 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4600000000000002e-06 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498819+0000", 2024-09-13T07:02:21.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049534340000000003 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498871+0000", 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1412000000000002e-05 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498923+0000", 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2252e-05 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.471 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19088 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3514368~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.471 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:21.471 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.992632542999999, 2024-09-13T07:02:21.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049750612999999999, 2024-09-13T07:02:21.471 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.471 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.471 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19088 2024-09-13T07:02:21.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.471 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449287+0000", 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:21.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449288+0000", 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.469e-06 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449291+0000", 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449294+0000", 2024-09-13T07:02:21.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5069999999999999e-06 2024-09-13T07:02:21.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498941+0000", 2024-09-13T07:02:21.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049647494 2024-09-13T07:02:21.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.498983+0000", 2024-09-13T07:02:21.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1771999999999999e-05 2024-09-13T07:02:21.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499038+0000", 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4888e-05 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19089 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3891200~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.992623809000001, 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049857261, 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19089 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:21.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449296+0000", 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449297+0000", 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4330000000000001e-06 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449301+0000", 2024-09-13T07:02:21.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5260000000000002e-06 2024-09-13T07:02:21.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449303+0000", 2024-09-13T07:02:21.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5110000000000002e-06 2024-09-13T07:02:21.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499050+0000", 2024-09-13T07:02:21.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049747121999999998 2024-09-13T07:02:21.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499091+0000", 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0580000000000001e-05 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499153+0000", 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2305999999999996e-05 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19090 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3538944~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.479 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.992614326000002, 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049959569000000002, 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19090 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:21.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449305+0000", 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449307+0000", 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4279999999999999e-06 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449309+0000", 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.706e-06 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449312+0000", 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4930000000000001e-06 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499169+0000", 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049857745000000002 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499210+0000", 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0706e-05 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499265+0000", 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4719000000000002e-05 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.483 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.484 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19091 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3915776~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.484 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:21.484 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.992605684000001, 2024-09-13T07:02:21.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050065959, 2024-09-13T07:02:21.484 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.484 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.484 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.484 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.484 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.484 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19091 2024-09-13T07:02:21.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449314+0000", 2024-09-13T07:02:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449315+0000", 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.485e-06 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449318+0000", 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6740000000000001e-06 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449320+0000", 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.413e-06 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499283+0000", 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049962196 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499323+0000", 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0799e-05 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499380+0000", 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6610999999999997e-05 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19092 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3563520~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449323+0000", 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.992596992999999, 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050176212999999997, 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19092 2024-09-13T07:02:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449323+0000", 2024-09-13T07:02:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449323+0000", 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449322+0000", 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449324+0000", 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.424e-06 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449327+0000", 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449329+0000", 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4820000000000001e-06 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499397+0000", 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050068362999999998 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499441+0000", 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3668999999999998e-05 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499499+0000", 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7778999999999997e-05 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19093 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3940352~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.992588210000001, 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050278704, 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19093 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449331+0000", 2024-09-13T07:02:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449333+0000", 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5570000000000001e-06 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449335+0000", 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449338+0000", 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.491e-06 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499513+0000", 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050175143999999998 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499554+0000", 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0729000000000003e-05 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499610+0000", 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6307000000000001e-05 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19094 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3588096~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.992579307, 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050385554999999999, 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19094 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449340+0000", 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449342+0000", 2024-09-13T07:02:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.652e-06 2024-09-13T07:02:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449345+0000", 2024-09-13T07:02:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:02:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449348+0000", 2024-09-13T07:02:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5849999999999998e-06 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499626+0000", 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050278728000000002 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499669+0000", 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2400000000000001e-05 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499726+0000", 2024-09-13T07:02:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7191999999999997e-05 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19095 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3964928~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.992569595999999, 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050492171000000002, 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19095 2024-09-13T07:02:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449350+0000", 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449351+0000", 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.449e-06 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449354+0000", 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.836e-06 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449357+0000", 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5459999999999998e-06 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499744+0000", 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050387339000000003 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499786+0000", 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2293999999999999e-05 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499842+0000", 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5927000000000002e-05 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19096 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3612672~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.992560526999998, 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050599058000000002, 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19096 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449359+0000", 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449360+0000", 2024-09-13T07:02:21.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4529999999999999e-06 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449363+0000", 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6680000000000001e-06 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.449366+0000", 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6359999999999998e-06 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499860+0000", 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050494820000000003 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499902+0000", 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1111e-05 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.499958+0000", 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6594000000000001e-05 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19110 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 4186112~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.989437176999999, 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049302258000000002, 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19110 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452482+0000", 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:21.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452487+0000", 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9980000000000003e-06 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452490+0000", 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1410000000000001e-06 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452493+0000", 2024-09-13T07:02:21.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6010000000000002e-06 2024-09-13T07:02:21.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501587+0000", 2024-09-13T07:02:21.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049094282000000003 2024-09-13T07:02:21.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501680+0000", 2024-09-13T07:02:21.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2675000000000003e-05 2024-09-13T07:02:21.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501785+0000", 2024-09-13T07:02:21.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000104993 2024-09-13T07:02:21.514 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.514 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.514 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.514 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19111 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3760128~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.514 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:21.514 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.989424515, 2024-09-13T07:02:21.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049401467999999997, 2024-09-13T07:02:21.514 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.514 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19111 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452495+0000", 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452497+0000", 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7099999999999999e-06 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452499+0000", 2024-09-13T07:02:21.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6929999999999999e-06 2024-09-13T07:02:21.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452502+0000", 2024-09-13T07:02:21.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5900000000000002e-06 2024-09-13T07:02:21.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501801+0000", 2024-09-13T07:02:21.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049299005 2024-09-13T07:02:21.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501838+0000", 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7282e-05 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501897+0000", 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8405000000000001e-05 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19112 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3784704~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.518 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.989415376, 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049618640999999998, 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19112 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452504+0000", 2024-09-13T07:02:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452505+0000", 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.364e-06 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452508+0000", 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7470000000000001e-06 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452511+0000", 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.501932+0000", 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049421395999999999 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502019+0000", 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7247e-05 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502123+0000", 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103589 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.522 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:19113 3.1 3:850364ae:::rbd_data.1194257bfc65.0000000000000000:10 [read 3809280~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.989406351, 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049709729000000001, 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19113 2024-09-13T07:02:21.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452513+0000", 2024-09-13T07:02:21.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:21.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452515+0000", 2024-09-13T07:02:21.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659999999999999e-06 2024-09-13T07:02:21.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452517+0000", 2024-09-13T07:02:21.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6189999999999998e-06 2024-09-13T07:02:21.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.452520+0000", 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7760000000000002e-06 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502134+0000", 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.049614232000000001 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502173+0000", 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8486999999999999e-05 2024-09-13T07:02:21.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:53.502223+0000", 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0392999999999997e-05 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:48369 3.f e58/22 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head v 58'78, mlcod=57'75)", 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.201453208, 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.057439072000000001, 2024-09-13T07:02:21.527 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:21.528 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:21.528 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:21.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:21.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:21.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:21.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:21.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:21.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240464+0000", 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245716+0000", 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005252329 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245745+0000", 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8844e-05 2024-09-13T07:02:21.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245758+0000", 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2242000000000001e-05 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245849+0000", 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1187999999999995e-05 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245898+0000", 2024-09-13T07:02:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8881000000000002e-05 2024-09-13T07:02:21.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:21.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297837+0000", 2024-09-13T07:02:21.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051938947999999999 2024-09-13T07:02:21.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:21.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:21.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:21.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297906+0000", 2024-09-13T07:02:21.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8955999999999995e-05 2024-09-13T07:02:21.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.531 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.532 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:21.532 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:21.532 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:21.533 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:21.534 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:21.530+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:21.634 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:21.632+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:21.684 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-13T07:02:21.692 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:21.692 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:21.703 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:21.718 INFO:teuthology.orchestra.run.smithi118.stdout:2157 write 0xc72e294 thru 0xc730d1e (0x2a8b bytes) 2024-09-13T07:02:21.734 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:21.732+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:21.747 INFO:teuthology.orchestra.run.smithi118.stdout:2158 read 0x78d647f thru 0x78d938b (0x2f0d bytes) 2024-09-13T07:02:21.748 INFO:teuthology.orchestra.run.smithi118.stdout:2160 write 0xc4b40f4 thru 0xc4bf71d (0xb62a bytes) 2024-09-13T07:02:21.770 INFO:teuthology.orchestra.run.smithi118.stdout:2161 writesame 0x62fd3a2 thru 0x6302c85 data_size 0x2(0x58e4 bytes) 2024-09-13T07:02:21.788 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:21.789 INFO:teuthology.orchestra.run.smithi118.stdout:2165 writesame 0xa58747f thru 0xa594c85 data_size 0x1(0xd807 bytes) 2024-09-13T07:02:21.835 INFO:teuthology.orchestra.run.smithi118.stdout:2166 read 0xbce4532 thru 0xbce9206 (0x4cd5 bytes) 2024-09-13T07:02:21.835 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:21.834+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:21.836 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:21.836 INFO:teuthology.orchestra.run.smithi118.stdout:2168 write 0x448a532 thru 0x448ed72 (0x4841 bytes) 2024-09-13T07:02:21.841 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:21.841 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:21.841 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:21.841 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:21.841 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:21.855 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:21.861 INFO:teuthology.orchestra.run.smithi118.stdout:2169 punch from 0x9302245 to 0x930d1e6, (0xafa1 bytes) 2024-09-13T07:02:21.872 INFO:teuthology.orchestra.run.smithi118.stdout:2170 trunc from 0xe82e398 to 0x8b6713b 2024-09-13T07:02:21.939 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:21.934+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:21.987 INFO:teuthology.orchestra.run.smithi118.stdout:2171 read 0x70319d5 thru 0x7037962 (0x5f8e bytes) 2024-09-13T07:02:21.988 INFO:teuthology.orchestra.run.smithi118.stdout:2172 read 0x5a07742 thru 0x5a11710 (0x9fcf bytes) 2024-09-13T07:02:21.989 INFO:teuthology.orchestra.run.smithi118.stdout:2174 writesame 0x8016462 thru 0x8024088 data_size 0x1(0xdc27 bytes) 2024-09-13T07:02:22.010 INFO:teuthology.orchestra.run.smithi118.stdout:2175 read 0x3a0788c thru 0x3a08dcd (0x1542 bytes) 2024-09-13T07:02:22.011 INFO:teuthology.orchestra.run.smithi118.stdout:2177 write 0x410a480 thru 0x410d004 (0x2b85 bytes) 2024-09-13T07:02:22.013 INFO:teuthology.orchestra.run.smithi118.stdout:2178 read 0x284d6be thru 0x2854000 (0x6943 bytes) 2024-09-13T07:02:22.014 INFO:teuthology.orchestra.run.smithi118.stdout:2179 write 0x858226 thru 0x8641d2 (0xbfad bytes) 2024-09-13T07:02:22.037 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:22.034+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:22.060 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:22.061 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:22.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:22.061 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:22.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.061 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1531 3.c 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:22.061 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:22.061 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.010410778999997, 2024-09-13T07:02:22.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063168486999999995, 2024-09-13T07:02:22.061 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.061 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:22.061 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.061 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1531 2024-09-13T07:02:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049511+0000", 2024-09-13T07:02:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069234+0000", 2024-09-13T07:02:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019723920999999998 2024-09-13T07:02:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069269+0000", 2024-09-13T07:02:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4446999999999997e-05 2024-09-13T07:02:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069286+0000", 2024-09-13T07:02:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7405999999999999e-05 2024-09-13T07:02:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069370+0000", 2024-09-13T07:02:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-05 2024-09-13T07:02:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069511+0000", 2024-09-13T07:02:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014134299999999999 2024-09-13T07:02:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070463+0000", 2024-09-13T07:02:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095167899999999998 2024-09-13T07:02:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105076+0000", 2024-09-13T07:02:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034613139000000001 2024-09-13T07:02:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112562+0000", 2024-09-13T07:02:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074861429999999998 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112631+0000", 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112680+0000", 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9607000000000001e-05 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7628 3.1f 3:fdf2e50b:::rbd_data.1194c5936dae.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1032192~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:02:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.297484621999999, 2024-09-13T07:02:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.630407088, 2024-09-13T07:02:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7628 2024-09-13T07:02:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762435+0000", 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762543+0000", 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010850500000000001 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762597+0000", 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4262000000000003e-05 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762626+0000", 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9176e-05 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762761+0000", 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013405599999999999 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762776+0000", 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5679999999999999e-05 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388492+0000", 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.62571606300000004 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for missing object", 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388614+0000", 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012126 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390837+0000", 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002223806 2024-09-13T07:02:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390963+0000", 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012550300000000001 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391127+0000", 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163995 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391989+0000", 2024-09-13T07:02:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086230499999999995 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392662+0000", 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067260799999999999 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392728+0000", 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6501000000000006e-05 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392845+0000", 2024-09-13T07:02:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116665 2024-09-13T07:02:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:02:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.941334312999999, 2024-09-13T07:02:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:02:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:02:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:02:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:02:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:02:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:02:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:02:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:02:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:02:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:02:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:02:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:02:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:02:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:02:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:02:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:02:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:02:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:02:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:22.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:02:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:02:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:02:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:02:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48376 3.c 3:350dc8a4:::rbd_data.1194fc792839.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.830713469999999, 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.068448320000000007, 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48376 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229207+0000", 2024-09-13T07:02:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.241745+0000", 2024-09-13T07:02:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012537726000000001 2024-09-13T07:02:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244174+0000", 2024-09-13T07:02:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024284760000000002 2024-09-13T07:02:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244198+0000", 2024-09-13T07:02:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4629000000000001e-05 2024-09-13T07:02:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244312+0000", 2024-09-13T07:02:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114282 2024-09-13T07:02:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244679+0000", 2024-09-13T07:02:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000366399 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.246046+0000", 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013672479999999999 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.266272+0000", 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020226381000000002 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297510+0000", 2024-09-13T07:02:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031237576999999999 2024-09-13T07:02:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297595+0000", 2024-09-13T07:02:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5413000000000004e-05 2024-09-13T07:02:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297657+0000", 2024-09-13T07:02:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1860999999999996e-05 2024-09-13T07:02:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78084 3.17 3:ecbd88ca:::rbd_data.1194987e4741.0000000000000004:17 [read 225280~1024] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903205+0000", 2024-09-13T07:02:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1567174759999999, 2024-09-13T07:02:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050967852000000001, 2024-09-13T07:02:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78084 2024-09-13T07:02:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903205+0000", 2024-09-13T07:02:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903205+0000", 2024-09-13T07:02:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903205+0000", 2024-09-13T07:02:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903207+0000", 2024-09-13T07:02:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2800000000000002e-06 2024-09-13T07:02:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903215+0000", 2024-09-13T07:02:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-13T07:02:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903219+0000", 2024-09-13T07:02:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.313e-06 2024-09-13T07:02:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.953989+0000", 2024-09-13T07:02:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050770176 2024-09-13T07:02:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954056+0000", 2024-09-13T07:02:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7148000000000002e-05 2024-09-13T07:02:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954173+0000", 2024-09-13T07:02:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011643599999999999 2024-09-13T07:02:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78085 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903222+0000", 2024-09-13T07:02:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.156700013, 2024-09-13T07:02:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050982131, 2024-09-13T07:02:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78085 2024-09-13T07:02:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903222+0000", 2024-09-13T07:02:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903222+0000", 2024-09-13T07:02:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903222+0000", 2024-09-13T07:02:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903225+0000", 2024-09-13T07:02:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0740000000000001e-06 2024-09-13T07:02:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903232+0000", 2024-09-13T07:02:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5579999999999997e-06 2024-09-13T07:02:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903237+0000", 2024-09-13T07:02:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6840000000000001e-06 2024-09-13T07:02:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954194+0000", 2024-09-13T07:02:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050957055000000001 2024-09-13T07:02:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954205+0000", 2024-09-13T07:02:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0297e-05 2024-09-13T07:02:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78086 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903242+0000", 2024-09-13T07:02:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1566801130000002, 2024-09-13T07:02:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051002389000000002, 2024-09-13T07:02:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78086 2024-09-13T07:02:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903242+0000", 2024-09-13T07:02:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903242+0000", 2024-09-13T07:02:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903242+0000", 2024-09-13T07:02:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903245+0000", 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903251+0000", 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.122e-06 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903257+0000", 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:02:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954230+0000", 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050972904999999999 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954245+0000", 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4882000000000001e-05 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.103 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78087 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.104 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903262+0000", 2024-09-13T07:02:22.104 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.156660456, 2024-09-13T07:02:22.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051013920999999997, 2024-09-13T07:02:22.104 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.104 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.104 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78087 2024-09-13T07:02:22.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.104 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903262+0000", 2024-09-13T07:02:22.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903262+0000", 2024-09-13T07:02:22.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903261+0000", 2024-09-13T07:02:22.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903264+0000", 2024-09-13T07:02:22.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.05e-06 2024-09-13T07:02:22.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903271+0000", 2024-09-13T07:02:22.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1310000000000003e-06 2024-09-13T07:02:22.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903277+0000", 2024-09-13T07:02:22.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5849999999999996e-06 2024-09-13T07:02:22.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954267+0000", 2024-09-13T07:02:22.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050990129000000002 2024-09-13T07:02:22.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954276+0000", 2024-09-13T07:02:22.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6270000000000006e-06 2024-09-13T07:02:22.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.108 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.108 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78088 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.108 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903282+0000", 2024-09-13T07:02:22.108 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1566403410000001, 2024-09-13T07:02:22.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051023737, 2024-09-13T07:02:22.108 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.108 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.109 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78088 2024-09-13T07:02:22.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.109 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903282+0000", 2024-09-13T07:02:22.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903282+0000", 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903282+0000", 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903289+0000", 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7440000000000007e-06 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903298+0000", 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3790000000000006e-06 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903304+0000", 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3219999999999998e-06 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954297+0000", 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050992643999999997 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954306+0000", 2024-09-13T07:02:22.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0580000000000004e-06 2024-09-13T07:02:22.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.112 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.112 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78089 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.112 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903310+0000", 2024-09-13T07:02:22.112 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1566127740000001, 2024-09-13T07:02:22.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051026754000000001, 2024-09-13T07:02:22.112 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.113 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.113 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78089 2024-09-13T07:02:22.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.113 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903310+0000", 2024-09-13T07:02:22.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903310+0000", 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903309+0000", 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903312+0000", 2024-09-13T07:02:22.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.241e-06 2024-09-13T07:02:22.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903319+0000", 2024-09-13T07:02:22.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.613e-06 2024-09-13T07:02:22.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903324+0000", 2024-09-13T07:02:22.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3020000000000002e-06 2024-09-13T07:02:22.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954326+0000", 2024-09-13T07:02:22.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051001610000000003 2024-09-13T07:02:22.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954336+0000", 2024-09-13T07:02:22.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0728e-05 2024-09-13T07:02:22.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.116 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.117 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78090 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.117 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903329+0000", 2024-09-13T07:02:22.117 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1565933370000003, 2024-09-13T07:02:22.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051046853000000003, 2024-09-13T07:02:22.117 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.117 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.117 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78090 2024-09-13T07:02:22.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.117 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903329+0000", 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903329+0000", 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903329+0000", 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903332+0000", 2024-09-13T07:02:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8559999999999998e-06 2024-09-13T07:02:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903338+0000", 2024-09-13T07:02:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4649999999999999e-06 2024-09-13T07:02:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903344+0000", 2024-09-13T07:02:22.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6019999999999999e-06 2024-09-13T07:02:22.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954369+0000", 2024-09-13T07:02:22.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051025484000000003 2024-09-13T07:02:22.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954376+0000", 2024-09-13T07:02:22.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9129999999999997e-06 2024-09-13T07:02:22.121 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.121 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.121 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.121 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78091 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.121 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903349+0000", 2024-09-13T07:02:22.121 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1565737829999998, 2024-09-13T07:02:22.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051057573000000002, 2024-09-13T07:02:22.121 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.121 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.121 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.121 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.122 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78091 2024-09-13T07:02:22.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.122 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903349+0000", 2024-09-13T07:02:22.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903349+0000", 2024-09-13T07:02:22.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903348+0000", 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903351+0000", 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7920000000000001e-06 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903358+0000", 2024-09-13T07:02:22.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8009999999999998e-06 2024-09-13T07:02:22.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903363+0000", 2024-09-13T07:02:22.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2939999999999996e-06 2024-09-13T07:02:22.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954397+0000", 2024-09-13T07:02:22.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051033862999999999 2024-09-13T07:02:22.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954406+0000", 2024-09-13T07:02:22.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2799999999999992e-06 2024-09-13T07:02:22.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.125 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.125 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78092 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.125 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903368+0000", 2024-09-13T07:02:22.125 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1565543509999996, 2024-09-13T07:02:22.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051071887000000003, 2024-09-13T07:02:22.126 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.126 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.126 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78092 2024-09-13T07:02:22.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.126 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903368+0000", 2024-09-13T07:02:22.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903368+0000", 2024-09-13T07:02:22.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903368+0000", 2024-09-13T07:02:22.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903371+0000", 2024-09-13T07:02:22.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.909e-06 2024-09-13T07:02:22.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903377+0000", 2024-09-13T07:02:22.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5529999999999998e-06 2024-09-13T07:02:22.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903383+0000", 2024-09-13T07:02:22.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9139999999999996e-06 2024-09-13T07:02:22.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954428+0000", 2024-09-13T07:02:22.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051044524000000001 2024-09-13T07:02:22.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954440+0000", 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2442e-05 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78093 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903388+0000", 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1565347509999997, 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051086009000000002, 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.130 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.131 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78093 2024-09-13T07:02:22.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.131 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903388+0000", 2024-09-13T07:02:22.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903388+0000", 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903387+0000", 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903390+0000", 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7329999999999999e-06 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903396+0000", 2024-09-13T07:02:22.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4659999999999996e-06 2024-09-13T07:02:22.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903402+0000", 2024-09-13T07:02:22.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9710000000000001e-06 2024-09-13T07:02:22.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954463+0000", 2024-09-13T07:02:22.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051060881000000002 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954474+0000", 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0397000000000001e-05 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78094 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903408+0000", 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.156514016, 2024-09-13T07:02:22.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051097747999999998, 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78094 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903408+0000", 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903408+0000", 2024-09-13T07:02:22.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903408+0000", 2024-09-13T07:02:22.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903411+0000", 2024-09-13T07:02:22.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0790000000000001e-06 2024-09-13T07:02:22.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903418+0000", 2024-09-13T07:02:22.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5289999999999997e-06 2024-09-13T07:02:22.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903423+0000", 2024-09-13T07:02:22.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6679999999999998e-06 2024-09-13T07:02:22.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954497+0000", 2024-09-13T07:02:22.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051073674999999999 2024-09-13T07:02:22.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954506+0000", 2024-09-13T07:02:22.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2469999999999997e-06 2024-09-13T07:02:22.138 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.138 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.139 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78095 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.139 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903428+0000", 2024-09-13T07:02:22.139 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1564942870000001, 2024-09-13T07:02:22.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051108246000000003, 2024-09-13T07:02:22.139 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.139 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.139 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.139 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.139 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.139 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78095 2024-09-13T07:02:22.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.140 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903428+0000", 2024-09-13T07:02:22.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903428+0000", 2024-09-13T07:02:22.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903428+0000", 2024-09-13T07:02:22.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903431+0000", 2024-09-13T07:02:22.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7970000000000001e-06 2024-09-13T07:02:22.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903437+0000", 2024-09-13T07:02:22.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5180000000000001e-06 2024-09-13T07:02:22.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903443+0000", 2024-09-13T07:02:22.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8300000000000001e-06 2024-09-13T07:02:22.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954528+0000", 2024-09-13T07:02:22.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051085182 2024-09-13T07:02:22.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954536+0000", 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3650000000000004e-06 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78096 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903473+0000", 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1564492599999996, 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051095058999999998, 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78096 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903473+0000", 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903473+0000", 2024-09-13T07:02:22.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903473+0000", 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903475+0000", 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6709999999999999e-06 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903481+0000", 2024-09-13T07:02:22.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4600000000000002e-06 2024-09-13T07:02:22.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903485+0000", 2024-09-13T07:02:22.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-13T07:02:22.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954559+0000", 2024-09-13T07:02:22.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051074018999999998 2024-09-13T07:02:22.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954568+0000", 2024-09-13T07:02:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5410000000000006e-06 2024-09-13T07:02:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78097 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903488+0000", 2024-09-13T07:02:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1564349439999999, 2024-09-13T07:02:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051110567000000003, 2024-09-13T07:02:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.148 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.148 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78097 2024-09-13T07:02:22.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.148 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903488+0000", 2024-09-13T07:02:22.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903488+0000", 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903487+0000", 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903489+0000", 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.906e-06 2024-09-13T07:02:22.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903494+0000", 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2020000000000003e-06 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903498+0000", 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.399e-06 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954590+0000", 2024-09-13T07:02:22.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051091932 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954598+0000", 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3979999999999999e-06 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78098 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903501+0000", 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1564215359999999, 2024-09-13T07:02:22.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051126591999999998, 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78098 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903501+0000", 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903501+0000", 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903501+0000", 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903503+0000", 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1289999999999999e-06 2024-09-13T07:02:22.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903506+0000", 2024-09-13T07:02:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4580000000000002e-06 2024-09-13T07:02:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903510+0000", 2024-09-13T07:02:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3150000000000002e-06 2024-09-13T07:02:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954618+0000", 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051108029999999999 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954628+0000", 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0699999999999996e-06 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:78099 3.17 3.5311bd37 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:17.903513+0000", 2024-09-13T07:02:22.155 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1564092370000001, 2024-09-13T07:02:22.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051143515000000001, 2024-09-13T07:02:22.156 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.156 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:22.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.156 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 78099 2024-09-13T07:02:22.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.156 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903513+0000", 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903513+0000", 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903513+0000", 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903515+0000", 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8700000000000001e-06 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903518+0000", 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.523e-06 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.903522+0000", 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3189999999999999e-06 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954648+0000", 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051126434999999998 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:17.954657+0000", 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6270000000000006e-06 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.159 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:22.161 INFO:teuthology.orchestra.run.smithi118.stdout:2180 read 0x1c643c4 thru 0x1c6e9bc (0xa5f9 bytes) 2024-09-13T07:02:22.161 INFO:teuthology.orchestra.run.smithi118.stdout:2182 read 0x48039ec thru 0x4812f9e (0xf5b3 bytes) 2024-09-13T07:02:22.161 INFO:teuthology.orchestra.run.smithi118.stdout:2183 trunc from 0x8b6713b to 0x455f72b 2024-09-13T07:02:22.161 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:22.156+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:22.162 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:22.185 INFO:teuthology.orchestra.run.smithi118.stdout:2184 read 0x33d9b32 thru 0x33e173e (0x7c0d bytes) 2024-09-13T07:02:22.186 INFO:teuthology.orchestra.run.smithi118.stdout:2186 read 0x42ad33 thru 0x437c06 (0xced4 bytes) 2024-09-13T07:02:22.203 INFO:teuthology.orchestra.run.smithi118.stdout:2188 trunc from 0x455f72b to 0xb886c7e 2024-09-13T07:02:22.214 INFO:teuthology.orchestra.run.smithi118.stdout:2189 clone 24 order 22 su 524288 sc 7 2024-09-13T07:02:22.262 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:22.260+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:22.311 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:22.311 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:22.311 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:22.311 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:22.324 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:22.362 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:22.360+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:22.418 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:22.418 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:22.427 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:22.463 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:22.462+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:22.538 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:22.538 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:22.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:22.538 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:22.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.538 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:22.538 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.476981864000003, 2024-09-13T07:02:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:02:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:02:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:02:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:02:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:02:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:02:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:02:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:02:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:02:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:02:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:02:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:02:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:02:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:02:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:02:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:02:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:02:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:02:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:02:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:02:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1534 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14365141] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:02:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.470154313000002, 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040478761000000002, 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1534 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:02:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065747+0000", 2024-09-13T07:02:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075258+0000", 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095108710000000006 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075288+0000", 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0085000000000001e-05 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075305+0000", 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7748e-05 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075367+0000", 2024-09-13T07:02:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1867999999999998e-05 2024-09-13T07:02:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075520+0000", 2024-09-13T07:02:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015298800000000001 2024-09-13T07:02:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.076247+0000", 2024-09-13T07:02:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072720499999999997 2024-09-13T07:02:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105157+0000", 2024-09-13T07:02:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028909642999999999 2024-09-13T07:02:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106124+0000", 2024-09-13T07:02:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096729400000000005 2024-09-13T07:02:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106173+0000", 2024-09-13T07:02:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8196999999999999e-05 2024-09-13T07:02:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106227+0000", 2024-09-13T07:02:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4388999999999997e-05 2024-09-13T07:02:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84407 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 12206080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901446+0000", 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63445651400000003, 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030719218, 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84407 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901446+0000", 2024-09-13T07:02:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901446+0000", 2024-09-13T07:02:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901446+0000", 2024-09-13T07:02:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901448+0000", 2024-09-13T07:02:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3080000000000002e-06 2024-09-13T07:02:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901454+0000", 2024-09-13T07:02:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1319999999999999e-06 2024-09-13T07:02:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901459+0000", 2024-09-13T07:02:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.464e-06 2024-09-13T07:02:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.924872+0000", 2024-09-13T07:02:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023413633 2024-09-13T07:02:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.924953+0000", 2024-09-13T07:02:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0773999999999996e-05 2024-09-13T07:02:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932165+0000", 2024-09-13T07:02:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072122640000000003 2024-09-13T07:02:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84408 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 12402688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901464+0000", 2024-09-13T07:02:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63443876399999999, 2024-09-13T07:02:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026240949999999999, 2024-09-13T07:02:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84408 2024-09-13T07:02:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901464+0000", 2024-09-13T07:02:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901464+0000", 2024-09-13T07:02:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901463+0000", 2024-09-13T07:02:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901502+0000", 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8288999999999999e-05 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901509+0000", 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0269999999999998e-06 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901514+0000", 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2209999999999996e-06 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927501+0000", 2024-09-13T07:02:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025987059 2024-09-13T07:02:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927587+0000", 2024-09-13T07:02:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5603e-05 2024-09-13T07:02:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927705+0000", 2024-09-13T07:02:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118062 2024-09-13T07:02:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84409 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 12763136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901519+0000", 2024-09-13T07:02:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63438366300000004, 2024-09-13T07:02:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026402450000000001, 2024-09-13T07:02:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84409 2024-09-13T07:02:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901519+0000", 2024-09-13T07:02:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901519+0000", 2024-09-13T07:02:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901518+0000", 2024-09-13T07:02:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901521+0000", 2024-09-13T07:02:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5119999999999998e-06 2024-09-13T07:02:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901526+0000", 2024-09-13T07:02:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.293e-06 2024-09-13T07:02:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901531+0000", 2024-09-13T07:02:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4619999999999998e-06 2024-09-13T07:02:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927729+0000", 2024-09-13T07:02:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026198321 2024-09-13T07:02:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927806+0000", 2024-09-13T07:02:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7243000000000006e-05 2024-09-13T07:02:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927921+0000", 2024-09-13T07:02:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011509400000000001 2024-09-13T07:02:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84410 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 12959744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901534+0000", 2024-09-13T07:02:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63436830600000005, 2024-09-13T07:02:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026609322000000001, 2024-09-13T07:02:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84410 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901534+0000", 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901534+0000", 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901534+0000", 2024-09-13T07:02:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901537+0000", 2024-09-13T07:02:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6299999999999998e-06 2024-09-13T07:02:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901541+0000", 2024-09-13T07:02:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7099999999999998e-06 2024-09-13T07:02:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901545+0000", 2024-09-13T07:02:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8680000000000001e-06 2024-09-13T07:02:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927944+0000", 2024-09-13T07:02:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026398402000000001 2024-09-13T07:02:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928018+0000", 2024-09-13T07:02:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4985000000000001e-05 2024-09-13T07:02:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928144+0000", 2024-09-13T07:02:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012506400000000001 2024-09-13T07:02:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84411 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 13123584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901549+0000", 2024-09-13T07:02:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63435378799999997, 2024-09-13T07:02:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026813632, 2024-09-13T07:02:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84411 2024-09-13T07:02:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901549+0000", 2024-09-13T07:02:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901549+0000", 2024-09-13T07:02:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901548+0000", 2024-09-13T07:02:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901551+0000", 2024-09-13T07:02:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.615e-06 2024-09-13T07:02:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901556+0000", 2024-09-13T07:02:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4659999999999998e-06 2024-09-13T07:02:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901562+0000", 2024-09-13T07:02:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:02:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928171+0000", 2024-09-13T07:02:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026608643000000001 2024-09-13T07:02:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928247+0000", 2024-09-13T07:02:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6058999999999998e-05 2024-09-13T07:02:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928362+0000", 2024-09-13T07:02:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115562 2024-09-13T07:02:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84412 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 13320192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901566+0000", 2024-09-13T07:02:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63433654799999994, 2024-09-13T07:02:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027009352, 2024-09-13T07:02:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84412 2024-09-13T07:02:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901566+0000", 2024-09-13T07:02:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901566+0000", 2024-09-13T07:02:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901566+0000", 2024-09-13T07:02:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901569+0000", 2024-09-13T07:02:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8959999999999999e-06 2024-09-13T07:02:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901574+0000", 2024-09-13T07:02:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8989999999999998e-06 2024-09-13T07:02:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901579+0000", 2024-09-13T07:02:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1640000000000003e-06 2024-09-13T07:02:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928385+0000", 2024-09-13T07:02:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026806469999999999 2024-09-13T07:02:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928459+0000", 2024-09-13T07:02:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3396e-05 2024-09-13T07:02:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928575+0000", 2024-09-13T07:02:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116846 2024-09-13T07:02:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84413 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 13680640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901583+0000", 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63431995299999999, 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027205637000000001, 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84413 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901583+0000", 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901583+0000", 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901582+0000", 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901585+0000", 2024-09-13T07:02:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6299999999999998e-06 2024-09-13T07:02:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901591+0000", 2024-09-13T07:02:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7350000000000001e-06 2024-09-13T07:02:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901595+0000", 2024-09-13T07:02:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.318e-06 2024-09-13T07:02:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928597+0000", 2024-09-13T07:02:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027002307999999999 2024-09-13T07:02:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928673+0000", 2024-09-13T07:02:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5430000000000001e-05 2024-09-13T07:02:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928788+0000", 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115568 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84414 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901599+0000", 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63430372899999998, 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027406803, 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84414 2024-09-13T07:02:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901599+0000", 2024-09-13T07:02:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901599+0000", 2024-09-13T07:02:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901598+0000", 2024-09-13T07:02:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901601+0000", 2024-09-13T07:02:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.221e-06 2024-09-13T07:02:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901606+0000", 2024-09-13T07:02:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2310000000000004e-06 2024-09-13T07:02:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901610+0000", 2024-09-13T07:02:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9489999999999998e-06 2024-09-13T07:02:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928812+0000", 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027201843 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928886+0000", 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4538999999999999e-05 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929006+0000", 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119336 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84415 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14041088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901614+0000", 2024-09-13T07:02:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63428889099999997, 2024-09-13T07:02:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027605469000000001, 2024-09-13T07:02:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84415 2024-09-13T07:02:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901614+0000", 2024-09-13T07:02:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901614+0000", 2024-09-13T07:02:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901613+0000", 2024-09-13T07:02:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901616+0000", 2024-09-13T07:02:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3130000000000001e-06 2024-09-13T07:02:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901654+0000", 2024-09-13T07:02:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7914999999999999e-05 2024-09-13T07:02:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901660+0000", 2024-09-13T07:02:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:02:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929028+0000", 2024-09-13T07:02:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027368380000000001 2024-09-13T07:02:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929103+0000", 2024-09-13T07:02:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4609000000000003e-05 2024-09-13T07:02:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929219+0000", 2024-09-13T07:02:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116168 2024-09-13T07:02:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84416 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14237696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901666+0000", 2024-09-13T07:02:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63423670399999998, 2024-09-13T07:02:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027766724999999999, 2024-09-13T07:02:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84416 2024-09-13T07:02:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901666+0000", 2024-09-13T07:02:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901666+0000", 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901665+0000", 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901669+0000", 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3289999999999998e-06 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901676+0000", 2024-09-13T07:02:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2620000000000001e-06 2024-09-13T07:02:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901680+0000", 2024-09-13T07:02:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:02:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929242+0000", 2024-09-13T07:02:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027561326000000001 2024-09-13T07:02:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929316+0000", 2024-09-13T07:02:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4019999999999999e-05 2024-09-13T07:02:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929433+0000", 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116735 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84417 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901684+0000", 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63421819599999996, 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027976127, 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84417 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901684+0000", 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901684+0000", 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901684+0000", 2024-09-13T07:02:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901686+0000", 2024-09-13T07:02:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1449999999999998e-06 2024-09-13T07:02:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901691+0000", 2024-09-13T07:02:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0690000000000001e-06 2024-09-13T07:02:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901696+0000", 2024-09-13T07:02:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4730000000000002e-06 2024-09-13T07:02:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929456+0000", 2024-09-13T07:02:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027760164 2024-09-13T07:02:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929547+0000", 2024-09-13T07:02:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0623000000000003e-05 2024-09-13T07:02:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929660+0000", 2024-09-13T07:02:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113962 2024-09-13T07:02:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84540 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.905105+0000", 2024-09-13T07:02:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63079748800000002, 2024-09-13T07:02:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026262232, 2024-09-13T07:02:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84540 2024-09-13T07:02:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905105+0000", 2024-09-13T07:02:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905105+0000", 2024-09-13T07:02:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905105+0000", 2024-09-13T07:02:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905107+0000", 2024-09-13T07:02:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2120000000000002e-06 2024-09-13T07:02:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905112+0000", 2024-09-13T07:02:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8439999999999997e-06 2024-09-13T07:02:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905117+0000", 2024-09-13T07:02:22.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6879999999999996e-06 2024-09-13T07:02:22.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931155+0000", 2024-09-13T07:02:22.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026037976000000001 2024-09-13T07:02:22.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931233+0000", 2024-09-13T07:02:22.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7733999999999998e-05 2024-09-13T07:02:22.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931367+0000", 2024-09-13T07:02:22.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013415599999999999 2024-09-13T07:02:22.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.612 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.612 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84541 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 13697024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.905121+0000", 2024-09-13T07:02:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63078144000000003, 2024-09-13T07:02:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026510446999999999, 2024-09-13T07:02:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84541 2024-09-13T07:02:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905121+0000", 2024-09-13T07:02:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905121+0000", 2024-09-13T07:02:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905121+0000", 2024-09-13T07:02:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905123+0000", 2024-09-13T07:02:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2409999999999998e-06 2024-09-13T07:02:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905128+0000", 2024-09-13T07:02:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-13T07:02:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905132+0000", 2024-09-13T07:02:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1509999999999997e-06 2024-09-13T07:02:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931390+0000", 2024-09-13T07:02:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026258055999999998 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931475+0000", 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4452999999999999e-05 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931632+0000", 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000156804 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84542 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14057472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.905136+0000", 2024-09-13T07:02:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63076663099999997, 2024-09-13T07:02:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026726409999999999, 2024-09-13T07:02:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84542 2024-09-13T07:02:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905136+0000", 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905136+0000", 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905136+0000", 2024-09-13T07:02:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905138+0000", 2024-09-13T07:02:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.221e-06 2024-09-13T07:02:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905143+0000", 2024-09-13T07:02:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7419999999999997e-06 2024-09-13T07:02:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905147+0000", 2024-09-13T07:02:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2860000000000001e-06 2024-09-13T07:02:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931656+0000", 2024-09-13T07:02:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026509064999999998 2024-09-13T07:02:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931743+0000", 2024-09-13T07:02:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7548999999999998e-05 2024-09-13T07:02:22.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931862+0000", 2024-09-13T07:02:22.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118879 2024-09-13T07:02:22.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.622 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.623 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84543 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14254080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.623 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.905150+0000", 2024-09-13T07:02:22.623 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63075224699999999, 2024-09-13T07:02:22.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026942713, 2024-09-13T07:02:22.624 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.624 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.624 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84543 2024-09-13T07:02:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905150+0000", 2024-09-13T07:02:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905150+0000", 2024-09-13T07:02:22.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905150+0000", 2024-09-13T07:02:22.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905152+0000", 2024-09-13T07:02:22.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9810000000000002e-06 2024-09-13T07:02:22.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905157+0000", 2024-09-13T07:02:22.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5839999999999996e-06 2024-09-13T07:02:22.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905160+0000", 2024-09-13T07:02:22.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8630000000000001e-06 2024-09-13T07:02:22.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931887+0000", 2024-09-13T07:02:22.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026726481 2024-09-13T07:02:22.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931961+0000", 2024-09-13T07:02:22.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4626e-05 2024-09-13T07:02:22.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932093+0000", 2024-09-13T07:02:22.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013151699999999999 2024-09-13T07:02:22.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.629 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84544 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.905166+0000", 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63073674099999999, 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027182738000000001, 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84544 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905166+0000", 2024-09-13T07:02:22.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905166+0000", 2024-09-13T07:02:22.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905165+0000", 2024-09-13T07:02:22.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905168+0000", 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.181e-06 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905173+0000", 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905177+0000", 2024-09-13T07:02:22.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1779999999999996e-06 2024-09-13T07:02:22.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932117+0000", 2024-09-13T07:02:22.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026940153000000001 2024-09-13T07:02:22.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932226+0000", 2024-09-13T07:02:22.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108751 2024-09-13T07:02:22.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932349+0000", 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000122762 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84545 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.905180+0000", 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63072209300000004, 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027392868000000001, 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84545 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905180+0000", 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905180+0000", 2024-09-13T07:02:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905180+0000", 2024-09-13T07:02:22.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905183+0000", 2024-09-13T07:02:22.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.537e-06 2024-09-13T07:02:22.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905187+0000", 2024-09-13T07:02:22.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6739999999999996e-06 2024-09-13T07:02:22.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905193+0000", 2024-09-13T07:02:22.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5570000000000003e-06 2024-09-13T07:02:22.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932377+0000", 2024-09-13T07:02:22.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027184069000000002 2024-09-13T07:02:22.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932453+0000", 2024-09-13T07:02:22.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5635000000000003e-05 2024-09-13T07:02:22.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932573+0000", 2024-09-13T07:02:22.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120701 2024-09-13T07:02:22.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.638 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.639 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84793 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 12615680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:22.639 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.917918+0000", 2024-09-13T07:02:22.639 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61798443000000003, 2024-09-13T07:02:22.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026248368000000001, 2024-09-13T07:02:22.639 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.639 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:22.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.639 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84793 2024-09-13T07:02:22.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.640 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.917918+0000", 2024-09-13T07:02:22.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.917918+0000", 2024-09-13T07:02:22.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.917918+0000", 2024-09-13T07:02:22.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:22.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.917920+0000", 2024-09-13T07:02:22.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9850000000000001e-06 2024-09-13T07:02:22.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.917925+0000", 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0749999999999997e-06 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.917929+0000", 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4560000000000002e-06 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.938144+0000", 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020214567999999999 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.939071+0000", 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00092679800000000005 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.944166+0000", 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005095746 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.643 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:22.645 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:22.646 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:22.644+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:22.746 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:22.744+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:22.757 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:22.757 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:22.757 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:22.757 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:22.767 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:22.847 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:22.846+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:22.867 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:22.868 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:22.868 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:22.868 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:22.868 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:22.877 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:22.900 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:02:22.949 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:22.948+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:22.984 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:22.984 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:22.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:22.984 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:22.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.985 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1174 3.13 3:cb2ed8dc:::rbd_data.11943a018f34.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 6419344~27893] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:22.985 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:02:22.985 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 57.099866622999997, 2024-09-13T07:02:22.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041813481999999999, 2024-09-13T07:02:22.985 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.985 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:22.985 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.985 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.985 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.986 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1174 2024-09-13T07:02:22.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.986 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:02:22.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:02:22.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883795+0000", 2024-09-13T07:02:22.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:22.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894234+0000", 2024-09-13T07:02:22.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010439133 2024-09-13T07:02:22.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894276+0000", 2024-09-13T07:02:22.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429000000000003e-05 2024-09-13T07:02:22.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894296+0000", 2024-09-13T07:02:22.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9279999999999998e-05 2024-09-13T07:02:22.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894477+0000", 2024-09-13T07:02:22.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018160600000000001 2024-09-13T07:02:22.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894889+0000", 2024-09-13T07:02:22.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041202399999999998 2024-09-13T07:02:22.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:22.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.898212+0000", 2024-09-13T07:02:22.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033228400000000001 2024-09-13T07:02:22.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:22.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.922541+0000", 2024-09-13T07:02:22.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024329052 2024-09-13T07:02:22.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:22.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925458+0000", 2024-09-13T07:02:22.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002916891 2024-09-13T07:02:22.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925531+0000", 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2781999999999995e-05 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925610+0000", 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9355000000000002e-05 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:02:22.991 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:22.992 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.929490821999998, 2024-09-13T07:02:22.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:02:22.992 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.992 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:22.992 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:22.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:22.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:02:22.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:22.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:02:22.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:02:22.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:22.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:02:22.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:02:22.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:22.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:02:22.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:02:22.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:22.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:02:22.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:02:22.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:22.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:02:22.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:02:22.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:22.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:02:22.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:02:22.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:22.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:02:22.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:02:22.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.997 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:22.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:22.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.997 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:22.997 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:22.997 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.771609588, 2024-09-13T07:02:22.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:02:22.997 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:22.998 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:22.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:22.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:22.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:22.998 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:02:22.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.998 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:22.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:22.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:22.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:02:22.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:02:23.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:02:23.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:02:23.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:02:23.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:02:23.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:02:23.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:02:23.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:02:23.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:02:23.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:02:23.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:23.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:02:23.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:02:23.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:02:23.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:02:23.003 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.003 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.003 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.745839877000002, 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.004 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:23.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:23.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:02:23.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:23.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:02:23.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:02:23.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:02:23.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:02:23.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:02:23.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:02:23.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:02:23.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:02:23.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:02:23.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:02:23.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:23.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:02:23.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:02:23.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:23.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:02:23.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:02:23.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:23.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:02:23.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:02:23.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:23.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:02:23.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:02:23.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:02:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:02:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.724688710999999, 2024-09-13T07:02:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:02:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:02:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:02:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:02:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:02:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:02:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:02:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:02:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:02:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:02:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:02:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:02:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:02:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:02:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:02:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:02:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:02:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:02:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:02:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:02:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:02:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:02:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:02:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48377 3.d 3:b0571a54:::rbd_data.1194fc792839.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13654590~2926018] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.702792426999999, 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037131338, 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48377 2024-09-13T07:02:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280868+0000", 2024-09-13T07:02:23.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289338+0000", 2024-09-13T07:02:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084696370000000003 2024-09-13T07:02:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289362+0000", 2024-09-13T07:02:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4263000000000001e-05 2024-09-13T07:02:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289374+0000", 2024-09-13T07:02:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1209999999999999e-05 2024-09-13T07:02:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289460+0000", 2024-09-13T07:02:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6883000000000001e-05 2024-09-13T07:02:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289603+0000", 2024-09-13T07:02:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014259299999999999 2024-09-13T07:02:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.290036+0000", 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000432545 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317645+0000", 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027609584 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317836+0000", 2024-09-13T07:02:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190645 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317925+0000", 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8732000000000004e-05 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.318002+0000", 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7618000000000001e-05 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:02:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.607643948, 2024-09-13T07:02:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:02:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:02:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:02:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:02:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:02:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:02:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:02:23.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:02:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:02:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:02:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:02:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:02:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:02:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:02:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:02:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:02:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:02:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:02:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:02:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:02:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84839 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 14516224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930431+0000", 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0532323640000001, 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031861486000000001, 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84839 2024-09-13T07:02:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930431+0000", 2024-09-13T07:02:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930431+0000", 2024-09-13T07:02:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930431+0000", 2024-09-13T07:02:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930432+0000", 2024-09-13T07:02:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.344e-06 2024-09-13T07:02:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930435+0000", 2024-09-13T07:02:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:02:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930437+0000", 2024-09-13T07:02:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5739999999999998e-06 2024-09-13T07:02:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962158+0000", 2024-09-13T07:02:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031720554999999998 2024-09-13T07:02:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962206+0000", 2024-09-13T07:02:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8250000000000001e-05 2024-09-13T07:02:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962292+0000", 2024-09-13T07:02:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6131000000000006e-05 2024-09-13T07:02:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84956 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 11943936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930666+0000", 2024-09-13T07:02:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0529975899999999, 2024-09-13T07:02:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031897930999999997, 2024-09-13T07:02:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84956 2024-09-13T07:02:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930666+0000", 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930666+0000", 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930665+0000", 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930667+0000", 2024-09-13T07:02:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8929999999999999e-06 2024-09-13T07:02:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930670+0000", 2024-09-13T07:02:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2030000000000001e-06 2024-09-13T07:02:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930673+0000", 2024-09-13T07:02:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6390000000000001e-06 2024-09-13T07:02:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962315+0000", 2024-09-13T07:02:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031641759999999998 2024-09-13T07:02:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962401+0000", 2024-09-13T07:02:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6378000000000005e-05 2024-09-13T07:02:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962564+0000", 2024-09-13T07:02:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016248 2024-09-13T07:02:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84957 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 12140544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930675+0000", 2024-09-13T07:02:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.052988064, 2024-09-13T07:02:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032070254999999999, 2024-09-13T07:02:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84957 2024-09-13T07:02:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930675+0000", 2024-09-13T07:02:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930675+0000", 2024-09-13T07:02:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930675+0000", 2024-09-13T07:02:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930676+0000", 2024-09-13T07:02:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2580000000000001e-06 2024-09-13T07:02:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930679+0000", 2024-09-13T07:02:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8080000000000001e-06 2024-09-13T07:02:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930682+0000", 2024-09-13T07:02:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-13T07:02:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962607+0000", 2024-09-13T07:02:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031925289000000003 2024-09-13T07:02:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962660+0000", 2024-09-13T07:02:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2666e-05 2024-09-13T07:02:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962745+0000", 2024-09-13T07:02:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5797999999999994e-05 2024-09-13T07:02:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84958 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 12500992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930684+0000", 2024-09-13T07:02:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.052979442, 2024-09-13T07:02:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032330946999999999, 2024-09-13T07:02:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84958 2024-09-13T07:02:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930684+0000", 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930684+0000", 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930683+0000", 2024-09-13T07:02:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930685+0000", 2024-09-13T07:02:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.669e-06 2024-09-13T07:02:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930689+0000", 2024-09-13T07:02:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.126e-06 2024-09-13T07:02:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930692+0000", 2024-09-13T07:02:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5179999999999999e-06 2024-09-13T07:02:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962768+0000", 2024-09-13T07:02:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032076219000000003 2024-09-13T07:02:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962859+0000", 2024-09-13T07:02:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1508999999999994e-05 2024-09-13T07:02:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963015+0000", 2024-09-13T07:02:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000155301 2024-09-13T07:02:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84959 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 12697600~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930694+0000", 2024-09-13T07:02:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.052969544, 2024-09-13T07:02:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032672771000000003, 2024-09-13T07:02:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84959 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930694+0000", 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930694+0000", 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930694+0000", 2024-09-13T07:02:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930695+0000", 2024-09-13T07:02:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.658e-06 2024-09-13T07:02:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930699+0000", 2024-09-13T07:02:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0629999999999999e-06 2024-09-13T07:02:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930702+0000", 2024-09-13T07:02:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5560000000000002e-06 2024-09-13T07:02:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963058+0000", 2024-09-13T07:02:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032356318000000002 2024-09-13T07:02:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963161+0000", 2024-09-13T07:02:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103368 2024-09-13T07:02:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963367+0000", 2024-09-13T07:02:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000205031 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84960 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 12861440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930704+0000", 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.052959325, 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032951687, 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84960 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930704+0000", 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930704+0000", 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930704+0000", 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930705+0000", 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.669e-06 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930711+0000", 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1039999999999998e-06 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930714+0000", 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7620000000000001e-06 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963411+0000", 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032696814999999997 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963500+0000", 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8982999999999998e-05 2024-09-13T07:02:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963656+0000", 2024-09-13T07:02:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000155563 2024-09-13T07:02:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84961 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 13058048~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930718+0000", 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.052945053, 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033209935000000003, 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84961 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930718+0000", 2024-09-13T07:02:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930718+0000", 2024-09-13T07:02:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930718+0000", 2024-09-13T07:02:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930720+0000", 2024-09-13T07:02:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4820000000000001e-06 2024-09-13T07:02:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930727+0000", 2024-09-13T07:02:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.939e-06 2024-09-13T07:02:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930733+0000", 2024-09-13T07:02:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3369999999999996e-06 2024-09-13T07:02:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963687+0000", 2024-09-13T07:02:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032953627999999999 2024-09-13T07:02:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963772+0000", 2024-09-13T07:02:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4588999999999999e-05 2024-09-13T07:02:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963928+0000", 2024-09-13T07:02:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015654800000000001 2024-09-13T07:02:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84962 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 13418496~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930736+0000", 2024-09-13T07:02:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.052926931, 2024-09-13T07:02:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033524994000000002, 2024-09-13T07:02:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84962 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930736+0000", 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930736+0000", 2024-09-13T07:02:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930736+0000", 2024-09-13T07:02:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930738+0000", 2024-09-13T07:02:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.903e-06 2024-09-13T07:02:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930741+0000", 2024-09-13T07:02:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1520000000000001e-06 2024-09-13T07:02:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930744+0000", 2024-09-13T07:02:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3060000000000002e-06 2024-09-13T07:02:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963978+0000", 2024-09-13T07:02:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033233608999999997 2024-09-13T07:02:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964083+0000", 2024-09-13T07:02:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105461 2024-09-13T07:02:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964261+0000", 2024-09-13T07:02:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017797399999999999 2024-09-13T07:02:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84963 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 13615104~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930746+0000", 2024-09-13T07:02:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0529169039999999, 2024-09-13T07:02:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033877758000000001, 2024-09-13T07:02:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84963 2024-09-13T07:02:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930746+0000", 2024-09-13T07:02:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930746+0000", 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930746+0000", 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930755+0000", 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2909999999999996e-06 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930759+0000", 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3519999999999998e-06 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930762+0000", 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8080000000000001e-06 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964312+0000", 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033550705 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964413+0000", 2024-09-13T07:02:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100995 2024-09-13T07:02:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964624+0000", 2024-09-13T07:02:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021085099999999999 2024-09-13T07:02:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84964 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 13778944~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930772+0000", 2024-09-13T07:02:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0528909870000001, 2024-09-13T07:02:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034129645, 2024-09-13T07:02:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84964 2024-09-13T07:02:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930772+0000", 2024-09-13T07:02:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930772+0000", 2024-09-13T07:02:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930772+0000", 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930774+0000", 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.255e-06 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930777+0000", 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3790000000000002e-06 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930780+0000", 2024-09-13T07:02:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7420000000000002e-06 2024-09-13T07:02:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964666+0000", 2024-09-13T07:02:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033885314 2024-09-13T07:02:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964757+0000", 2024-09-13T07:02:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1279999999999993e-05 2024-09-13T07:02:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964902+0000", 2024-09-13T07:02:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000145148 2024-09-13T07:02:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84965 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 13975552~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930783+0000", 2024-09-13T07:02:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0528807040000001, 2024-09-13T07:02:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034340119000000002, 2024-09-13T07:02:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84965 2024-09-13T07:02:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930783+0000", 2024-09-13T07:02:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930783+0000", 2024-09-13T07:02:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930782+0000", 2024-09-13T07:02:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930784+0000", 2024-09-13T07:02:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.508e-06 2024-09-13T07:02:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930787+0000", 2024-09-13T07:02:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7980000000000001e-06 2024-09-13T07:02:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930789+0000", 2024-09-13T07:02:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-13T07:02:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964929+0000", 2024-09-13T07:02:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034140141999999998 2024-09-13T07:02:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964999+0000", 2024-09-13T07:02:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9647999999999995e-05 2024-09-13T07:02:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965123+0000", 2024-09-13T07:02:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001237 2024-09-13T07:02:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84966 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 14336000~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930792+0000", 2024-09-13T07:02:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.052871522, 2024-09-13T07:02:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034666583000000001, 2024-09-13T07:02:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84966 2024-09-13T07:02:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930792+0000", 2024-09-13T07:02:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930792+0000", 2024-09-13T07:02:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930791+0000", 2024-09-13T07:02:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930793+0000", 2024-09-13T07:02:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.674e-06 2024-09-13T07:02:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930796+0000", 2024-09-13T07:02:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.807e-06 2024-09-13T07:02:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930799+0000", 2024-09-13T07:02:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8559999999999998e-06 2024-09-13T07:02:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965162+0000", 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034363553999999998 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965282+0000", 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012021800000000001 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965458+0000", 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000175885 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84967 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 14532608~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930801+0000", 2024-09-13T07:02:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.052862438, 2024-09-13T07:02:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034820298, 2024-09-13T07:02:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84967 2024-09-13T07:02:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930801+0000", 2024-09-13T07:02:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930801+0000", 2024-09-13T07:02:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930801+0000", 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930802+0000", 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3939999999999999e-06 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930838+0000", 2024-09-13T07:02:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5571999999999998e-05 2024-09-13T07:02:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930843+0000", 2024-09-13T07:02:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8560000000000003e-06 2024-09-13T07:02:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965478+0000", 2024-09-13T07:02:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034635000999999999 2024-09-13T07:02:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965531+0000", 2024-09-13T07:02:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2073000000000001e-05 2024-09-13T07:02:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965621+0000", 2024-09-13T07:02:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0608999999999999e-05 2024-09-13T07:02:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:23.097 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:23.098 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:23.098 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:23.098 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:23.099 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:23.096+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:23.099 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:23.118 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:23.143 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:23.166 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:23.190 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:23.199 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:23.196+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:23.209 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:23.209 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:23.213 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:23.216 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:23.216 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:23.216 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:23.216 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:23.216 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:23.216 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:23.216 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:23.216 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:23.218 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:23.300 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:23.296+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:23.325 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:02:23.390 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:23.390 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:23.401 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:23.402 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:23.398+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:23.441 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone23 from 0xb886c7e (overlap 0x455f72b) to 0x6eff1d 2024-09-13T07:02:23.504 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:23.502+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:23.540 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:23.540 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:23.554 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:23.602 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:23.600+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:23.674 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:23.674 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:23.684 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:23.704 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:23.702+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:23.748 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-13T07:02:23.783 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:23.783 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:23.794 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:23.805 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:23.804+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:23.906 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:23.902+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:23.926 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:23.926 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:23.926 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:24.007 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:24.004+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:24.031 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:24.032 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:24.042 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:24.108 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:24.104+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:24.149 INFO:teuthology.orchestra.run.smithi118.stdout:2190 punch from 0x70dbfe4 to 0x70dc7f8, (0x814 bytes) 2024-09-13T07:02:24.156 INFO:teuthology.orchestra.run.smithi118.stdout:2191 punch from 0x6eac75d to 0x6eb46f1, (0x7f94 bytes) 2024-09-13T07:02:24.156 INFO:teuthology.orchestra.run.smithi118.stdout:2192 punch from 0x5ab2038 to 0x5ab3126, (0x10ee bytes) 2024-09-13T07:02:24.156 INFO:teuthology.orchestra.run.smithi118.stdout:2193 read 0x975f934 thru 0x976c297 (0xc964 bytes) 2024-09-13T07:02:24.156 INFO:teuthology.orchestra.run.smithi118.stdout:2195 read 0xa833f8e thru 0xa836111 (0x2184 bytes) 2024-09-13T07:02:24.156 INFO:teuthology.orchestra.run.smithi118.stdout:2196 punch from 0x11f63de to 0x1202faf, (0xcbd1 bytes) 2024-09-13T07:02:24.156 INFO:teuthology.orchestra.run.smithi118.stdout:2197 read 0xa58f2f2 thru 0xa5961b3 (0x6ec2 bytes) 2024-09-13T07:02:24.156 INFO:teuthology.orchestra.run.smithi118.stdout:2199 read 0x52a95cb thru 0x52b1ad7 (0x850d bytes) 2024-09-13T07:02:24.157 INFO:teuthology.orchestra.run.smithi118.stdout:2200 punch from 0x5e0ea97 to 0x5e10f41, (0x24aa bytes) 2024-09-13T07:02:24.157 INFO:teuthology.orchestra.run.smithi118.stdout:2201 read 0xaa0c9d thru 0xaa3668 (0x29cc bytes) 2024-09-13T07:02:24.158 INFO:teuthology.orchestra.run.smithi118.stdout:2202 punch from 0x6c77e77 to 0x6c85859, (0xd9e2 bytes) 2024-09-13T07:02:24.158 INFO:teuthology.orchestra.run.smithi118.stdout:2203 write 0x6755980 thru 0x675c0a2 (0x6723 bytes) 2024-09-13T07:02:24.165 INFO:teuthology.orchestra.run.smithi118.stdout:2204 punch from 0x539342 to 0x547c49, (0xe907 bytes) 2024-09-13T07:02:24.165 INFO:teuthology.orchestra.run.smithi118.stdout:2205 read 0x2e8a0a6 thru 0x2e99b97 (0xfaf2 bytes) 2024-09-13T07:02:24.168 INFO:teuthology.orchestra.run.smithi118.stdout:2206 trunc from 0xb886c7e to 0xa38cb19 2024-09-13T07:02:24.196 INFO:teuthology.orchestra.run.smithi118.stdout:2208 read 0x267fec7 thru 0x26830b7 (0x31f1 bytes) 2024-09-13T07:02:24.197 INFO:teuthology.orchestra.run.smithi118.stdout:2209 write 0x92c4ac thru 0x9380a8 (0xbbfd bytes) 2024-09-13T07:02:24.204 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:24.204 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:24.208 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:24.204+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:24.214 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:24.234 INFO:teuthology.orchestra.run.smithi118.stdout:2212 write 0x4512dc9 thru 0x451aed1 (0x8109 bytes) 2024-09-13T07:02:24.248 INFO:teuthology.orchestra.run.smithi118.stdout:2213 write 0x144bcb thru 0x148d23 (0x4159 bytes) 2024-09-13T07:02:24.298 INFO:teuthology.orchestra.run.smithi118.stdout:2214 write 0xd1bbf23 thru 0xd1c8d46 (0xce24 bytes) 2024-09-13T07:02:24.309 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:24.306+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:24.316 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:02:24.317 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:02:24.317 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T07:02:24.317 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.393991 2024-09-13T07:02:24.320 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0.39399095572599563 2024-09-13T07:02:24.321 INFO:teuthology.orchestra.run.smithi118.stdout:2215 write 0xebf014d thru 0xebfe47f (0xe333 bytes) 2024-09-13T07:02:24.327 INFO:teuthology.orchestra.run.smithi118.stdout:2217 writesame 0xb08ba19 thru 0xb090a17 data_size 0x1(0x4fff bytes) 2024-09-13T07:02:24.342 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:24.342 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:24.354 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:02:24.358 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:24.362 INFO:teuthology.orchestra.run.smithi118.stdout:2220 trunc from 0xebfe480 to 0x7b1a3d0 2024-09-13T07:02:24.413 INFO:teuthology.orchestra.run.smithi118.stdout:2221 write 0x7c09904 thru 0x7c15a22 (0xc11f bytes) 2024-09-13T07:02:24.413 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:24.412+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:24.416 INFO:teuthology.orchestra.run.smithi118.stdout:2222 read 0x33b4aad thru 0x33b97e3 (0x4d37 bytes) 2024-09-13T07:02:24.429 INFO:teuthology.orchestra.run.smithi118.stdout:2224 trunc from 0x7c15a23 to 0x2776ade 2024-09-13T07:02:24.484 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:24.484 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:24.498 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:24.511 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:24.510+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:24.525 INFO:teuthology.orchestra.run.smithi118.stdout:2226 read 0x1309580 thru 0x1315efe (0xc97f bytes) 2024-09-13T07:02:24.527 INFO:teuthology.orchestra.run.smithi118.stdout:2227 punch from 0x2aaf99 to 0x2abd2a, (0xd91 bytes) 2024-09-13T07:02:24.527 INFO:teuthology.orchestra.run.smithi118.stdout:2228 read 0x1ceca27 thru 0x1cf8b96 (0xc170 bytes) 2024-09-13T07:02:24.538 INFO:teuthology.orchestra.run.smithi118.stdout:2229 write 0x3fdddc9 thru 0x3fe2c5a (0x4e92 bytes) 2024-09-13T07:02:24.544 INFO:teuthology.orchestra.run.smithi118.stdout:2230 write 0x502a8f thru 0x507c24 (0x5196 bytes) 2024-09-13T07:02:24.595 INFO:teuthology.orchestra.run.smithi118.stdout:2233 trunc from 0x3fe2c5b to 0xc519e9d 2024-09-13T07:02:24.605 INFO:teuthology.orchestra.run.smithi118.stdout:2234 writesame 0x7afb7d6 thru 0x7aff443 data_size 0x23(0x3c6e bytes) 2024-09-13T07:02:24.611 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:24.611 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:24.608+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:24.624 INFO:teuthology.orchestra.run.smithi118.stdout:2238 trunc from 0xc519e9d to 0x4ec250a 2024-09-13T07:02:24.635 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:24.635 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:24.645 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:24.648 INFO:teuthology.orchestra.run.smithi118.stdout:2239 write 0x137c8b9 thru 0x137dea0 (0x15e8 bytes) 2024-09-13T07:02:24.651 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:24.652 INFO:teuthology.orchestra.run.smithi118.stdout:2241 writesame 0x2d67aa0 thru 0x2d6c69e data_size 0x5(0x4bff bytes) 2024-09-13T07:02:24.680 INFO:teuthology.orchestra.run.smithi118.stdout:2242 writesame 0x9ebe6cd thru 0x9ec2a5e data_size 0x2(0x4392 bytes) 2024-09-13T07:02:24.680 INFO:teuthology.orchestra.run.smithi118.stdout:2244 write 0x350547e thru 0x350abb9 (0x573c bytes) 2024-09-13T07:02:24.683 INFO:teuthology.orchestra.run.smithi118.stdout:2246 read 0x4086480 thru 0x408b59e (0x511f bytes) 2024-09-13T07:02:24.683 INFO:teuthology.orchestra.run.smithi118.stdout:2247 read 0x5f9c4a5 thru 0x5fa105c (0x4bb8 bytes) 2024-09-13T07:02:24.698 INFO:teuthology.orchestra.run.smithi118.stdout:2248 trunc from 0x9ec2a5f to 0xb8b6d0 2024-09-13T07:02:24.712 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:24.710+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:24.760 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:24.760 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:24.772 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:24.778 INFO:teuthology.orchestra.run.smithi118.stdout:2249 punch from 0xa4d7e2 to 0xa5379f, (0x5fbd bytes) 2024-09-13T07:02:24.778 INFO:teuthology.orchestra.run.smithi118.stdout:2251 read 0x73c472 thru 0x745a0a (0x9599 bytes) 2024-09-13T07:02:24.779 INFO:teuthology.orchestra.run.smithi118.stdout:2252 punch from 0x116677 to 0x1197fe, (0x3187 bytes) 2024-09-13T07:02:24.801 INFO:teuthology.orchestra.run.smithi118.stdout:2253 write 0x6e746d7 thru 0x6e75aba (0x13e4 bytes) 2024-09-13T07:02:24.807 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:24.823 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:24.822+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:24.831 INFO:teuthology.orchestra.run.smithi118.stdout:2256 writesame 0xca3191d thru 0xca4106e data_size 0x1(0xf752 bytes) 2024-09-13T07:02:24.832 INFO:teuthology.orchestra.run.smithi118.stdout:2257 read 0x2da4e16 thru 0x2da6d01 (0x1eec bytes) 2024-09-13T07:02:24.832 INFO:teuthology.orchestra.run.smithi118.stdout:2260 read 0x1d2d814 thru 0x1d38951 (0xb13e bytes) 2024-09-13T07:02:24.832 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:24.832 INFO:teuthology.orchestra.run.smithi118.stdout:2263 write 0x6ff7a4b thru 0x7004c24 (0xd1da bytes) 2024-09-13T07:02:24.841 INFO:teuthology.orchestra.run.smithi118.stdout:2265 punch from 0x8dbb2ef to 0x8dc347c, (0x818d bytes) 2024-09-13T07:02:24.841 INFO:teuthology.orchestra.run.smithi118.stdout:2266 punch from 0x1163be2 to 0x11708ff, (0xcd1d bytes) 2024-09-13T07:02:24.841 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:24.841 INFO:teuthology.orchestra.run.smithi118.stdout:2268 write 0x43c816 thru 0x445272 (0x8a5d bytes) 2024-09-13T07:02:24.844 INFO:teuthology.orchestra.run.smithi118.stdout:2270 write 0x8c4e54e thru 0x8c4f67c (0x112f bytes) 2024-09-13T07:02:24.850 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:24.861 INFO:teuthology.orchestra.run.smithi118.stdout:2274 trunc from 0xca4106f to 0x85d48f6 2024-09-13T07:02:24.887 INFO:teuthology.orchestra.run.smithi118.stdout:2276 punch from 0x683007 to 0x6846f4, (0x16ed bytes) 2024-09-13T07:02:24.887 INFO:teuthology.orchestra.run.smithi118.stdout:2277 clone 25 order 21 su 8192 sc 14 2024-09-13T07:02:24.889 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:24.889 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:24.899 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:24.913 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:24.910+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:25.000 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:25.000 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:25.009 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:25.051 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:25.050+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:25.114 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:25.110+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:25.162 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:25.162 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:25.173 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:25.219 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:25.218+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:25.230 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:02:25.285 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:25.285 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:25.294 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:25.316 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:25.312+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:25.357 INFO:teuthology.orchestra.run.smithi118.stderr:set osd.3 primary-affinity to 0.393991 (8258202) 2024-09-13T07:02:25.411 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:25.411 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:25.417 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:25.416+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:25.421 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:25.518 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:25.514+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:25.530 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:25.530 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:25.540 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:25.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:02:25.560+0000 7f37ff7fe700 0 --2- 172.21.15.118:0/262654863 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f380006c990 0x7f38001cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T07:02:25.597 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:25.619 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:25.618+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:25.619 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:25.639 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:25.658 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:25.687 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:25.693 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:25.693 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:25.703 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:25.710 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:25.732 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:25.751 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:25.750+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:25.753 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:25.756 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:25.756 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:25.756 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:25.756 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:25.756 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:25.756 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:25.756 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:25.757 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:25.809 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:25.809 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:25.818 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:25.821 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:25.818+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:25.922 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:25.920+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:25.925 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:25.925 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:25.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.864229463000001, 2024-09-13T07:02:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:02:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:02:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:02:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:02:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:02:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:02:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:02:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:02:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:02:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:02:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:02:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:02:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:02:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:02:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:02:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:02:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:02:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:02:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:02:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:02:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:02:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:02:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:02:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:02:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1534 3.a 3:52bd0aab:::rbd_data.11943a018f34.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14365141] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:02:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.857401912, 2024-09-13T07:02:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040478761000000002, 2024-09-13T07:02:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1534 2024-09-13T07:02:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:02:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065748+0000", 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065747+0000", 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075258+0000", 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095108710000000006 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075288+0000", 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0085000000000001e-05 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075305+0000", 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7748e-05 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075367+0000", 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1867999999999998e-05 2024-09-13T07:02:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.075520+0000", 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015298800000000001 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.076247+0000", 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072720499999999997 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105157+0000", 2024-09-13T07:02:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028909642999999999 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106124+0000", 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096729400000000005 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106173+0000", 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8196999999999999e-05 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.106227+0000", 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4388999999999997e-05 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84407 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 12206080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901446+0000", 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0217041130000002, 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030719218, 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84407 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901446+0000", 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901446+0000", 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901446+0000", 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901448+0000", 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3080000000000002e-06 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901454+0000", 2024-09-13T07:02:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1319999999999999e-06 2024-09-13T07:02:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901459+0000", 2024-09-13T07:02:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.464e-06 2024-09-13T07:02:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.924872+0000", 2024-09-13T07:02:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023413633 2024-09-13T07:02:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.924953+0000", 2024-09-13T07:02:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0773999999999996e-05 2024-09-13T07:02:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932165+0000", 2024-09-13T07:02:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072122640000000003 2024-09-13T07:02:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84408 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 12402688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901464+0000", 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0216863629999997, 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026240949999999999, 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84408 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901464+0000", 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901464+0000", 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901463+0000", 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901502+0000", 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8288999999999999e-05 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901509+0000", 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0269999999999998e-06 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901514+0000", 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2209999999999996e-06 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927501+0000", 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025987059 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927587+0000", 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5603e-05 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927705+0000", 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118062 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84409 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 12763136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901519+0000", 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0216312619999997, 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026402450000000001, 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84409 2024-09-13T07:02:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901519+0000", 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901519+0000", 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901518+0000", 2024-09-13T07:02:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901521+0000", 2024-09-13T07:02:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5119999999999998e-06 2024-09-13T07:02:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901526+0000", 2024-09-13T07:02:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.293e-06 2024-09-13T07:02:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901531+0000", 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4619999999999998e-06 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927729+0000", 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026198321 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927806+0000", 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7243000000000006e-05 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927921+0000", 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011509400000000001 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84410 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 12959744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901534+0000", 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.021615905, 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026609322000000001, 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.951 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84410 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901534+0000", 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901534+0000", 2024-09-13T07:02:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901534+0000", 2024-09-13T07:02:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901537+0000", 2024-09-13T07:02:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6299999999999998e-06 2024-09-13T07:02:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901541+0000", 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7099999999999998e-06 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901545+0000", 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8680000000000001e-06 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.927944+0000", 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026398402000000001 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928018+0000", 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4985000000000001e-05 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928144+0000", 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012506400000000001 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84411 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 13123584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901549+0000", 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0216013869999996, 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026813632, 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84411 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901549+0000", 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901549+0000", 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901548+0000", 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901551+0000", 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.615e-06 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901556+0000", 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4659999999999998e-06 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901562+0000", 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928171+0000", 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026608643000000001 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928247+0000", 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6058999999999998e-05 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928362+0000", 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115562 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84412 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 13320192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901566+0000", 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0215841470000004, 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027009352, 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84412 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901566+0000", 2024-09-13T07:02:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901566+0000", 2024-09-13T07:02:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901566+0000", 2024-09-13T07:02:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901569+0000", 2024-09-13T07:02:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8959999999999999e-06 2024-09-13T07:02:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901574+0000", 2024-09-13T07:02:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8989999999999998e-06 2024-09-13T07:02:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901579+0000", 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1640000000000003e-06 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928385+0000", 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026806469999999999 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928459+0000", 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3396e-05 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928575+0000", 2024-09-13T07:02:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116846 2024-09-13T07:02:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84413 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 13680640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901583+0000", 2024-09-13T07:02:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0215675519999996, 2024-09-13T07:02:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027205637000000001, 2024-09-13T07:02:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84413 2024-09-13T07:02:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901583+0000", 2024-09-13T07:02:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901583+0000", 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901582+0000", 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901585+0000", 2024-09-13T07:02:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6299999999999998e-06 2024-09-13T07:02:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901591+0000", 2024-09-13T07:02:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7350000000000001e-06 2024-09-13T07:02:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901595+0000", 2024-09-13T07:02:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.318e-06 2024-09-13T07:02:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928597+0000", 2024-09-13T07:02:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027002307999999999 2024-09-13T07:02:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928673+0000", 2024-09-13T07:02:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5430000000000001e-05 2024-09-13T07:02:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928788+0000", 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115568 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84414 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901599+0000", 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0215513280000001, 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027406803, 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84414 2024-09-13T07:02:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901599+0000", 2024-09-13T07:02:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901599+0000", 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901598+0000", 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901601+0000", 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.221e-06 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901606+0000", 2024-09-13T07:02:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2310000000000004e-06 2024-09-13T07:02:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901610+0000", 2024-09-13T07:02:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9489999999999998e-06 2024-09-13T07:02:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928812+0000", 2024-09-13T07:02:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027201843 2024-09-13T07:02:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.928886+0000", 2024-09-13T07:02:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4538999999999999e-05 2024-09-13T07:02:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929006+0000", 2024-09-13T07:02:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119336 2024-09-13T07:02:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84415 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14041088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901614+0000", 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0215364899999999, 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027605469000000001, 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84415 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901614+0000", 2024-09-13T07:02:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901614+0000", 2024-09-13T07:02:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901613+0000", 2024-09-13T07:02:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901616+0000", 2024-09-13T07:02:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3130000000000001e-06 2024-09-13T07:02:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901654+0000", 2024-09-13T07:02:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7914999999999999e-05 2024-09-13T07:02:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901660+0000", 2024-09-13T07:02:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:02:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929028+0000", 2024-09-13T07:02:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027368380000000001 2024-09-13T07:02:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929103+0000", 2024-09-13T07:02:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4609000000000003e-05 2024-09-13T07:02:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929219+0000", 2024-09-13T07:02:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116168 2024-09-13T07:02:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84416 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14237696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901666+0000", 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0214843030000003, 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027766724999999999, 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84416 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901666+0000", 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901666+0000", 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901665+0000", 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901669+0000", 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3289999999999998e-06 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901676+0000", 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2620000000000001e-06 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901680+0000", 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:02:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929242+0000", 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027561326000000001 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929316+0000", 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4019999999999999e-05 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929433+0000", 2024-09-13T07:02:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116735 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84417 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.901684+0000", 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0214657950000001, 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027976127, 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84417 2024-09-13T07:02:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901684+0000", 2024-09-13T07:02:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901684+0000", 2024-09-13T07:02:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901684+0000", 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901686+0000", 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1449999999999998e-06 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901691+0000", 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0690000000000001e-06 2024-09-13T07:02:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.901696+0000", 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4730000000000002e-06 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929456+0000", 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027760164 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929547+0000", 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0623000000000003e-05 2024-09-13T07:02:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.929660+0000", 2024-09-13T07:02:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113962 2024-09-13T07:02:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84540 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.905105+0000", 2024-09-13T07:02:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.018045087, 2024-09-13T07:02:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026262232, 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84540 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905105+0000", 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905105+0000", 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905105+0000", 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905107+0000", 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2120000000000002e-06 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905112+0000", 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8439999999999997e-06 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905117+0000", 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6879999999999996e-06 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931155+0000", 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026037976000000001 2024-09-13T07:02:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931233+0000", 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7733999999999998e-05 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931367+0000", 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013415599999999999 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84541 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 13697024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.905121+0000", 2024-09-13T07:02:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.018029039, 2024-09-13T07:02:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026510446999999999, 2024-09-13T07:02:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84541 2024-09-13T07:02:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905121+0000", 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905121+0000", 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905121+0000", 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905123+0000", 2024-09-13T07:02:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2409999999999998e-06 2024-09-13T07:02:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905128+0000", 2024-09-13T07:02:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-13T07:02:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905132+0000", 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1509999999999997e-06 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931390+0000", 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026258055999999998 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931475+0000", 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4452999999999999e-05 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931632+0000", 2024-09-13T07:02:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000156804 2024-09-13T07:02:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84542 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14057472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.905136+0000", 2024-09-13T07:02:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0180142300000004, 2024-09-13T07:02:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026726409999999999, 2024-09-13T07:02:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84542 2024-09-13T07:02:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905136+0000", 2024-09-13T07:02:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905136+0000", 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905136+0000", 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905138+0000", 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.221e-06 2024-09-13T07:02:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905143+0000", 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7419999999999997e-06 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905147+0000", 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2860000000000001e-06 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931656+0000", 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026509064999999998 2024-09-13T07:02:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931743+0000", 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7548999999999998e-05 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931862+0000", 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118879 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84543 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14254080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.905150+0000", 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0179998460000004, 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026942713, 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84543 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905150+0000", 2024-09-13T07:02:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905150+0000", 2024-09-13T07:02:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905150+0000", 2024-09-13T07:02:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905152+0000", 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9810000000000002e-06 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905157+0000", 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5839999999999996e-06 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905160+0000", 2024-09-13T07:02:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8630000000000001e-06 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931887+0000", 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026726481 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.931961+0000", 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4626e-05 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932093+0000", 2024-09-13T07:02:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013151699999999999 2024-09-13T07:02:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84544 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.905166+0000", 2024-09-13T07:02:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0179843399999999, 2024-09-13T07:02:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027182738000000001, 2024-09-13T07:02:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84544 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905166+0000", 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905166+0000", 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905165+0000", 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905168+0000", 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.181e-06 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905173+0000", 2024-09-13T07:02:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905177+0000", 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1779999999999996e-06 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932117+0000", 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026940153000000001 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932226+0000", 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108751 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932349+0000", 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000122762 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84545 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.905180+0000", 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0179696920000003, 2024-09-13T07:02:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027392868000000001, 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84545 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905180+0000", 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905180+0000", 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905180+0000", 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905183+0000", 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.537e-06 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905187+0000", 2024-09-13T07:02:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6739999999999996e-06 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.905193+0000", 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5570000000000003e-06 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932377+0000", 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027184069000000002 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932453+0000", 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5635000000000003e-05 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.932573+0000", 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120701 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84793 3.0 3:038b979a:::rbd_data.11948b05a38f.0000000000000002:1a [read 12615680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.917918+0000", 2024-09-13T07:02:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0052320290000001, 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026248368000000001, 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84793 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.917918+0000", 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.917918+0000", 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.917918+0000", 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.917920+0000", 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9850000000000001e-06 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.917925+0000", 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0749999999999997e-06 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.917929+0000", 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4560000000000002e-06 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.938144+0000", 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020214567999999999 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.939071+0000", 2024-09-13T07:02:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00092679800000000005 2024-09-13T07:02:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.944166+0000", 2024-09-13T07:02:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005095746 2024-09-13T07:02:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.017 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:26.018 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:26.022 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:26.020+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:26.123 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:26.120+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:26.133 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:26.133 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:26.133 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:26.133 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:26.143 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:26.224 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:26.222+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:26.249 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:26.250 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:26.259 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:26.325 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:26.322+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:26.369 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:02:26.374 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:26.374 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:26.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:26.374 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1174 3.13 3:cb2ed8dc:::rbd_data.11943a018f34.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 6419344~27893] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.489677030999999, 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041813481999999999, 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1174 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883797+0000", 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.883795+0000", 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:26.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894234+0000", 2024-09-13T07:02:26.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010439133 2024-09-13T07:02:26.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894276+0000", 2024-09-13T07:02:26.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429000000000003e-05 2024-09-13T07:02:26.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894296+0000", 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9279999999999998e-05 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894477+0000", 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018160600000000001 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.894889+0000", 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041202399999999998 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:26.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.898212+0000", 2024-09-13T07:02:26.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033228400000000001 2024-09-13T07:02:26.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:26.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.922541+0000", 2024-09-13T07:02:26.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024329052 2024-09-13T07:02:26.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925458+0000", 2024-09-13T07:02:26.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002916891 2024-09-13T07:02:26.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:26.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925531+0000", 2024-09-13T07:02:26.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2781999999999995e-05 2024-09-13T07:02:26.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:25.925610+0000", 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9355000000000002e-05 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.319301230000001, 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:26.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:02:26.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:02:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:02:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:02:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:02:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:02:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:26.385 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:26.386 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.161419995999999, 2024-09-13T07:02:26.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:02:26.386 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.386 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.386 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:02:26.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.386 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:02:26.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.135650285000001, 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:02:26.391 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.392 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.392 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:02:26.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.392 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:26.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:26.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:02:26.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:26.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:02:26.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:02:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:02:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:02:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:02:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:02:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:02:26.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:02:26.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:02:26.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.397 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.397 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:26.397 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:26.397 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.114499119000001, 2024-09-13T07:02:26.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:02:26.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:02:26.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48377 3.d 3:b0571a54:::rbd_data.1194fc792839.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13654590~2926018] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.092602835000001, 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.037131338, 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.403 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48377 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280871+0000", 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.280868+0000", 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289338+0000", 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084696370000000003 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289362+0000", 2024-09-13T07:02:26.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4263000000000001e-05 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289374+0000", 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1209999999999999e-05 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289460+0000", 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6883000000000001e-05 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.289603+0000", 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014259299999999999 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.290036+0000", 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000432545 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317645+0000", 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027609584 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:26.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317836+0000", 2024-09-13T07:02:26.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190645 2024-09-13T07:02:26.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:26.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317925+0000", 2024-09-13T07:02:26.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8732000000000004e-05 2024-09-13T07:02:26.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.318002+0000", 2024-09-13T07:02:26.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7618000000000001e-05 2024-09-13T07:02:26.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.997454355999999, 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.409 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:02:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:02:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:02:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:02:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:02:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:02:26.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:26.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:02:26.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:02:26.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:26.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:02:26.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:02:26.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:02:26.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:02:26.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:26.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:02:26.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:02:26.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84839 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 14516224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930431+0000", 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4430427720000001, 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031861486000000001, 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.415 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.416 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84839 2024-09-13T07:02:26.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.416 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930431+0000", 2024-09-13T07:02:26.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930431+0000", 2024-09-13T07:02:26.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930431+0000", 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930432+0000", 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.344e-06 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930435+0000", 2024-09-13T07:02:26.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930437+0000", 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5739999999999998e-06 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962158+0000", 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031720554999999998 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962206+0000", 2024-09-13T07:02:26.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8250000000000001e-05 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962292+0000", 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6131000000000006e-05 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84956 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 11943936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930666+0000", 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4428079980000001, 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031897930999999997, 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.419 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84956 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930666+0000", 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930666+0000", 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930665+0000", 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930667+0000", 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8929999999999999e-06 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930670+0000", 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2030000000000001e-06 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930673+0000", 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6390000000000001e-06 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962315+0000", 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031641759999999998 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962401+0000", 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6378000000000005e-05 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962564+0000", 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016248 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.423 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84957 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 12140544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.424 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930675+0000", 2024-09-13T07:02:26.424 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4427984719999998, 2024-09-13T07:02:26.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032070254999999999, 2024-09-13T07:02:26.424 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.424 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.424 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.424 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.424 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.424 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84957 2024-09-13T07:02:26.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.424 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930675+0000", 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930675+0000", 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930675+0000", 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930676+0000", 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2580000000000001e-06 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930679+0000", 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8080000000000001e-06 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930682+0000", 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-13T07:02:26.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962607+0000", 2024-09-13T07:02:26.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031925289000000003 2024-09-13T07:02:26.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962660+0000", 2024-09-13T07:02:26.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2666e-05 2024-09-13T07:02:26.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962745+0000", 2024-09-13T07:02:26.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5797999999999994e-05 2024-09-13T07:02:26.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.428 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.428 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84958 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 12500992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.428 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930684+0000", 2024-09-13T07:02:26.428 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4427898499999996, 2024-09-13T07:02:26.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032330946999999999, 2024-09-13T07:02:26.428 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.428 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.428 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84958 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930684+0000", 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930684+0000", 2024-09-13T07:02:26.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930683+0000", 2024-09-13T07:02:26.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930685+0000", 2024-09-13T07:02:26.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.669e-06 2024-09-13T07:02:26.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930689+0000", 2024-09-13T07:02:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.126e-06 2024-09-13T07:02:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930692+0000", 2024-09-13T07:02:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5179999999999999e-06 2024-09-13T07:02:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962768+0000", 2024-09-13T07:02:26.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032076219000000003 2024-09-13T07:02:26.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.962859+0000", 2024-09-13T07:02:26.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1508999999999994e-05 2024-09-13T07:02:26.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963015+0000", 2024-09-13T07:02:26.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000155301 2024-09-13T07:02:26.432 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.432 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.432 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.433 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84959 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 12697600~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.433 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930694+0000", 2024-09-13T07:02:26.433 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4427799520000004, 2024-09-13T07:02:26.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032672771000000003, 2024-09-13T07:02:26.433 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.433 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.433 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.433 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.433 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.433 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84959 2024-09-13T07:02:26.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930694+0000", 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930694+0000", 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930694+0000", 2024-09-13T07:02:26.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:26.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930695+0000", 2024-09-13T07:02:26.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.658e-06 2024-09-13T07:02:26.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930699+0000", 2024-09-13T07:02:26.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0629999999999999e-06 2024-09-13T07:02:26.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930702+0000", 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5560000000000002e-06 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963058+0000", 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032356318000000002 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963161+0000", 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103368 2024-09-13T07:02:26.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963367+0000", 2024-09-13T07:02:26.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000205031 2024-09-13T07:02:26.437 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.437 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.437 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.437 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84960 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 12861440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.437 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930704+0000", 2024-09-13T07:02:26.437 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4427697330000004, 2024-09-13T07:02:26.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032951687, 2024-09-13T07:02:26.438 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.438 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.438 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.438 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.438 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.438 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84960 2024-09-13T07:02:26.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.438 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930704+0000", 2024-09-13T07:02:26.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930704+0000", 2024-09-13T07:02:26.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930704+0000", 2024-09-13T07:02:26.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:26.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930705+0000", 2024-09-13T07:02:26.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.669e-06 2024-09-13T07:02:26.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930711+0000", 2024-09-13T07:02:26.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1039999999999998e-06 2024-09-13T07:02:26.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930714+0000", 2024-09-13T07:02:26.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7620000000000001e-06 2024-09-13T07:02:26.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963411+0000", 2024-09-13T07:02:26.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032696814999999997 2024-09-13T07:02:26.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963500+0000", 2024-09-13T07:02:26.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8982999999999998e-05 2024-09-13T07:02:26.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963656+0000", 2024-09-13T07:02:26.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000155563 2024-09-13T07:02:26.442 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.442 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.442 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.442 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84961 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 13058048~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.442 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930718+0000", 2024-09-13T07:02:26.442 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.442755461, 2024-09-13T07:02:26.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033209935000000003, 2024-09-13T07:02:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84961 2024-09-13T07:02:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930718+0000", 2024-09-13T07:02:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930718+0000", 2024-09-13T07:02:26.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930718+0000", 2024-09-13T07:02:26.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930720+0000", 2024-09-13T07:02:26.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4820000000000001e-06 2024-09-13T07:02:26.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930727+0000", 2024-09-13T07:02:26.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.939e-06 2024-09-13T07:02:26.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930733+0000", 2024-09-13T07:02:26.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3369999999999996e-06 2024-09-13T07:02:26.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963687+0000", 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.032953627999999999 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963772+0000", 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4588999999999999e-05 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963928+0000", 2024-09-13T07:02:26.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015654800000000001 2024-09-13T07:02:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84962 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 13418496~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930736+0000", 2024-09-13T07:02:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4427373389999998, 2024-09-13T07:02:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033524994000000002, 2024-09-13T07:02:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.447 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.448 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.448 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84962 2024-09-13T07:02:26.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.448 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930736+0000", 2024-09-13T07:02:26.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930736+0000", 2024-09-13T07:02:26.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930736+0000", 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930738+0000", 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.903e-06 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930741+0000", 2024-09-13T07:02:26.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1520000000000001e-06 2024-09-13T07:02:26.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930744+0000", 2024-09-13T07:02:26.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3060000000000002e-06 2024-09-13T07:02:26.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.963978+0000", 2024-09-13T07:02:26.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033233608999999997 2024-09-13T07:02:26.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964083+0000", 2024-09-13T07:02:26.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105461 2024-09-13T07:02:26.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964261+0000", 2024-09-13T07:02:26.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017797399999999999 2024-09-13T07:02:26.451 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.451 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.451 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.452 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84963 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 13615104~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.452 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930746+0000", 2024-09-13T07:02:26.452 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4427273119999997, 2024-09-13T07:02:26.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033877758000000001, 2024-09-13T07:02:26.452 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.452 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.452 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84963 2024-09-13T07:02:26.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.453 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930746+0000", 2024-09-13T07:02:26.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930746+0000", 2024-09-13T07:02:26.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930746+0000", 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930755+0000", 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2909999999999996e-06 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930759+0000", 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3519999999999998e-06 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930762+0000", 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8080000000000001e-06 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964312+0000", 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033550705 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964413+0000", 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100995 2024-09-13T07:02:26.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964624+0000", 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021085099999999999 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84964 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 13778944~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930772+0000", 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4427013950000003, 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034129645, 2024-09-13T07:02:26.456 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84964 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930772+0000", 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930772+0000", 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930772+0000", 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930774+0000", 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.255e-06 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930777+0000", 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3790000000000002e-06 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930780+0000", 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7420000000000002e-06 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964666+0000", 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033885314 2024-09-13T07:02:26.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964757+0000", 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1279999999999993e-05 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964902+0000", 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000145148 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84965 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 13975552~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930783+0000", 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4426911120000003, 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034340119000000002, 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84965 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930783+0000", 2024-09-13T07:02:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930783+0000", 2024-09-13T07:02:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930782+0000", 2024-09-13T07:02:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930784+0000", 2024-09-13T07:02:26.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.508e-06 2024-09-13T07:02:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930787+0000", 2024-09-13T07:02:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7980000000000001e-06 2024-09-13T07:02:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930789+0000", 2024-09-13T07:02:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-13T07:02:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964929+0000", 2024-09-13T07:02:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034140141999999998 2024-09-13T07:02:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.964999+0000", 2024-09-13T07:02:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9647999999999995e-05 2024-09-13T07:02:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965123+0000", 2024-09-13T07:02:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001237 2024-09-13T07:02:26.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84966 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 14336000~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930792+0000", 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.44268193, 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034666583000000001, 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.466 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84966 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930792+0000", 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930792+0000", 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930791+0000", 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930793+0000", 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.674e-06 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930796+0000", 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.807e-06 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930799+0000", 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8559999999999998e-06 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965162+0000", 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034363553999999998 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965282+0000", 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012021800000000001 2024-09-13T07:02:26.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965458+0000", 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000175885 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:84967 3.8 3:168ac795:::rbd_data.11948b05a38f.0000000000000005:1a [read 14532608~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:21.930801+0000", 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4426728459999998, 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034820298, 2024-09-13T07:02:26.470 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84967 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930801+0000", 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:26.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930801+0000", 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930801+0000", 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967296 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930802+0000", 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3939999999999999e-06 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930838+0000", 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5571999999999998e-05 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.930843+0000", 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8560000000000003e-06 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965478+0000", 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034635000999999999 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965531+0000", 2024-09-13T07:02:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2073000000000001e-05 2024-09-13T07:02:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:21.965621+0000", 2024-09-13T07:02:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0608999999999999e-05 2024-09-13T07:02:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:26.474 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:26.475 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:26.475 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone24 from 0x85d48f6 (overlap 0xb8b6d0) to 0x5c5b68 2024-09-13T07:02:26.476 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:26.477 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:26.472+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:26.572 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-13T07:02:26.577 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:26.574+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:26.583 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:26.583 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:26.583 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:26.583 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:26.594 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:26.598 INFO:teuthology.orchestra.run.smithi118.stdout:2279 write 0x118ed6b thru 0x119a322 (0xb5b8 bytes) 2024-09-13T07:02:26.677 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:26.676+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:26.703 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:26.703 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:26.703 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:26.714 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:26.759 INFO:teuthology.orchestra.run.smithi118.stdout:2282 writesame 0xe54d4ee thru 0xe556cbb data_size 0x7f(0x97ce bytes) 2024-09-13T07:02:26.759 INFO:teuthology.orchestra.run.smithi118.stdout:2283 punch from 0xd6a84b9 to 0xd6b44d0, (0xc017 bytes) 2024-09-13T07:02:26.759 INFO:teuthology.orchestra.run.smithi118.stdout:2284 punch from 0x536ac54 to 0x536dd4d, (0x30f9 bytes) 2024-09-13T07:02:26.759 INFO:teuthology.orchestra.run.smithi118.stdout:2285 write 0x402bfb4 thru 0x4032e94 (0x6ee1 bytes) 2024-09-13T07:02:26.774 INFO:teuthology.orchestra.run.smithi118.stdout:2286 punch from 0xdf00351 to 0xdf0e9cb, (0xe67a bytes) 2024-09-13T07:02:26.775 INFO:teuthology.orchestra.run.smithi118.stdout:2287 writesame 0x993572d thru 0x994381b data_size 0x1(0xe0ef bytes) 2024-09-13T07:02:26.778 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:26.774+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:26.799 INFO:teuthology.orchestra.run.smithi118.stdout:2288 read 0x929929b thru 0x92a380b (0xa571 bytes) 2024-09-13T07:02:26.799 INFO:teuthology.orchestra.run.smithi118.stdout:2289 read 0xc507440 thru 0xc5077b6 (0x377 bytes) 2024-09-13T07:02:26.799 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:26.799 INFO:teuthology.orchestra.run.smithi118.stdout:2291 writesame 0x12399e1 thru 0x124069a data_size 0x6(0x6cba bytes) 2024-09-13T07:02:26.821 INFO:teuthology.orchestra.run.smithi118.stdout:2292 trunc from 0xe556cbc to 0x55ee04a 2024-09-13T07:02:26.878 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:26.878 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:26.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:26.878 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:26.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.878 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:26.878 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:26.878 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.371469568000002, 2024-09-13T07:02:26.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:26.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:02:26.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:26.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:02:26.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:02:26.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:26.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:02:26.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:02:26.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:02:26.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:02:26.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:26.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:02:26.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:02:26.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.346641857999998, 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.885 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.885 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.885 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:02:26.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.885 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:26.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:02:26.889 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.807970034, 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:02:26.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:02:26.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:26.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:02:26.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:02:26.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:02:26.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:02:26.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:02:26.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:02:26.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.896 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:26.896 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:26.896 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.778344840999999, 2024-09-13T07:02:26.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:02:26.896 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.896 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.896 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.896 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.896 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.896 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:02:26.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.896 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:02:26.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:02:26.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:26.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:26.901 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.245631209999999, 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:02:26.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:02:26.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:02:26.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:02:26.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:02:26.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:26.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:02:26.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:02:26.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.245548753999998, 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.907 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.908 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:02:26.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.908 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:26.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:02:26.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:02:26.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.245516944000002, 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:02:26.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:02:26.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.918 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.245461786, 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:26.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:02:26.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:26.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:26.924 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.245416507000002, 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:02:26.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.245377972999997, 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.930 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:26.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.935 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.245316354000003, 2024-09-13T07:02:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:02:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.936 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:02:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:02:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:02:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:02:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:02:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:02:26.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:02:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:02:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:02:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:02:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.231093670999996, 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:02:26.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:02:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:02:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:02:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:02:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:26.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:26.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:26.947 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.219215151, 2024-09-13T07:02:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:02:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:02:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:26.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.951 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.218658089999998, 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:02:26.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:02:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:02:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:02:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:26.956 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.218635935000002, 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:26.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:02:26.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:02:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:02:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:02:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:02:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:02:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.960 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.218616218000001, 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:02:26.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:26.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.218598888000002, 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.965 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:02:26.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:02:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:02:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:02:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:02:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:26.969 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.218303226000003, 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:02:26.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:02:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:02:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:02:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:02:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:02:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:02:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.973 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.657100407000001, 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.974 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:02:26.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.979 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.625021504000003, 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:26.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:26.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:02:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:02:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:02:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:02:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:26.983 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:26.983 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:26.985 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:26.985 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:26.986 INFO:teuthology.orchestra.run.smithi118.stdout:2294 writesame 0x4b8df91 thru 0x4b936ae data_size 0x36(0x571e bytes) 2024-09-13T07:02:26.986 INFO:teuthology.orchestra.run.smithi118.stdout:2296 write 0xf37da9 thru 0xf3faaa (0x7d02 bytes) 2024-09-13T07:02:26.986 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:26.984+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:27.085 INFO:teuthology.orchestra.run.smithi118.stdout:2298 read 0x187421d thru 0x1881364 (0xd148 bytes) 2024-09-13T07:02:27.086 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:27.084+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:27.086 INFO:teuthology.orchestra.run.smithi118.stdout:2299 punch from 0x1532545 to 0x1539eed, (0x79a8 bytes) 2024-09-13T07:02:27.086 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:27.086 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:27.087 INFO:teuthology.orchestra.run.smithi118.stdout:2307 read 0x27f3447 thru 0x27fd9cb (0xa585 bytes) 2024-09-13T07:02:27.087 INFO:teuthology.orchestra.run.smithi118.stdout:2308 write 0x40ee542 thru 0x40fd216 (0xecd5 bytes) 2024-09-13T07:02:27.097 INFO:teuthology.orchestra.run.smithi118.stdout:2309 trunc from 0x55ee04a to 0x5cc4a35 2024-09-13T07:02:27.106 INFO:teuthology.orchestra.run.smithi118.stdout:2312 write 0x52d5d3e thru 0x52e273e (0xca01 bytes) 2024-09-13T07:02:27.110 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:27.110 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:27.110 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:27.110 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:27.120 INFO:teuthology.orchestra.run.smithi118.stdout:2314 punch from 0x34df1ca to 0x34e12ce, (0x2104 bytes) 2024-09-13T07:02:27.120 INFO:teuthology.orchestra.run.smithi118.stdout:2315 punch from 0x4bb7598 to 0x4bc4f2e, (0xd996 bytes) 2024-09-13T07:02:27.120 INFO:teuthology.orchestra.run.smithi118.stdout:2317 write 0x3d97f5f thru 0x3d9b882 (0x3924 bytes) 2024-09-13T07:02:27.121 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:27.125 INFO:teuthology.orchestra.run.smithi118.stdout:2318 punch from 0x490d10a to 0x4915e95, (0x8d8b bytes) 2024-09-13T07:02:27.125 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:27.149 INFO:teuthology.orchestra.run.smithi118.stdout:2320 write 0xe0d3a66 thru 0xe0e2dce (0xf369 bytes) 2024-09-13T07:02:27.175 INFO:teuthology.orchestra.run.smithi118.stdout:2322 trunc from 0xe0e2dcf to 0xa5d865b 2024-09-13T07:02:27.186 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:27.184+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:27.195 INFO:teuthology.orchestra.run.smithi118.stdout:2323 read 0x29c4945 thru 0x29c8b5f (0x421b bytes) 2024-09-13T07:02:27.195 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:27.195 INFO:teuthology.orchestra.run.smithi118.stdout:2326 write 0x2db7f7f thru 0x2dc32c1 (0xb343 bytes) 2024-09-13T07:02:27.217 INFO:teuthology.orchestra.run.smithi118.stdout:2328 trunc from 0xa5d865b to 0xe30ed0c 2024-09-13T07:02:27.226 INFO:teuthology.orchestra.run.smithi118.stdout:2330 write 0x4268a2c thru 0x426cf08 (0x44dd bytes) 2024-09-13T07:02:27.229 INFO:teuthology.orchestra.run.smithi118.stdout:2331 read 0x2a6c3ca thru 0x2a6f07d (0x2cb4 bytes) 2024-09-13T07:02:27.230 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:27.233 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:27.233 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:27.233 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:27.238 INFO:teuthology.orchestra.run.smithi118.stdout:2335 trunc from 0xe30ed0c to 0x9a1072e 2024-09-13T07:02:27.245 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:27.250 INFO:teuthology.orchestra.run.smithi118.stdout:2336 read 0x59ff2ce thru 0x5a062cb (0x6ffe bytes) 2024-09-13T07:02:27.251 INFO:teuthology.orchestra.run.smithi118.stdout:2338 write 0x2fe7cb9 thru 0x2ff79bc (0xfd04 bytes) 2024-09-13T07:02:27.261 INFO:teuthology.orchestra.run.smithi118.stdout:2339 punch from 0x7369e53 to 0x736fba7, (0x5d54 bytes) 2024-09-13T07:02:27.261 INFO:teuthology.orchestra.run.smithi118.stdout:2340 write 0x83a6100 thru 0x83a7f1c (0x1e1d bytes) 2024-09-13T07:02:27.277 INFO:teuthology.orchestra.run.smithi118.stdout:2341 write 0xb8d1b55 thru 0xb8d256b (0xa17 bytes) 2024-09-13T07:02:27.284 INFO:teuthology.orchestra.run.smithi118.stdout:2342 write 0x603387b thru 0x6039630 (0x5db6 bytes) 2024-09-13T07:02:27.287 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:27.284+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:27.287 INFO:teuthology.orchestra.run.smithi118.stdout:2345 read 0x82fec49 thru 0x830102a (0x23e2 bytes) 2024-09-13T07:02:27.292 INFO:teuthology.orchestra.run.smithi118.stdout:2347 trunc from 0xb8d256c to 0xdac3a5a 2024-09-13T07:02:27.303 INFO:teuthology.orchestra.run.smithi118.stdout:2349 writesame 0x9b8422 thru 0x9c44b0 data_size 0x1(0xc08f bytes) 2024-09-13T07:02:27.348 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:27.348 INFO:teuthology.orchestra.run.smithi118.stdout:2352 clone 26 order 19 su 262144 sc 2 2024-09-13T07:02:27.358 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.864188964, 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:02:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:02:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:02:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:02:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:02:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:02:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:02:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:02:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:02:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:02:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:02:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:02:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.3777054, 2024-09-13T07:02:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:02:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:27.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:27.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:02:27.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:02:27.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:02:27.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:02:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:02:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:02:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:02:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.377635471000001, 2024-09-13T07:02:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:02:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:02:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:02:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:02:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:02:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:02:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:02:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:02:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:02:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.377619598999999, 2024-09-13T07:02:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:02:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:02:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:02:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:02:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:02:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.377549141999999, 2024-09-13T07:02:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:02:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:02:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:02:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:02:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:02:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:02:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:02:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:02:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:02:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:02:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.374197104, 2024-09-13T07:02:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:02:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:02:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:02:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:02:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:02:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:02:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:02:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.374180803000002, 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:02:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:02:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:02:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:02:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:02:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:02:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:02:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.374161017999999, 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:02:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.374144744999999, 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:02:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:02:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:02:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:02:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:02:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:02:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.374126919999998, 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:02:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:02:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:02:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:02:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:02:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:02:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:02:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:02:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:02:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.374110826999999, 2024-09-13T07:02:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:02:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:02:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:02:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:02:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:02:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.374089641000001, 2024-09-13T07:02:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:02:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:02:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:02:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:02:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:02:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:02:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:02:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:02:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:02:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:02:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:02:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:02:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:02:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.374073608, 2024-09-13T07:02:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:02:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:02:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.374024369000001, 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:02:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:02:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:02:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:02:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.374006308999999, 2024-09-13T07:02:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:02:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:02:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.373989794, 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:02:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.373971429000001, 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:02:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:02:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:02:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.373955331000001, 2024-09-13T07:02:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:02:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:02:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:02:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:02:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:02:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.373939396000001, 2024-09-13T07:02:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:02:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:02:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:02:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:02:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:02:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:02:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.373922899, 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:02:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:02:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.445 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:27.446 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:27.447 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:02:27.447 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:27.444+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:27.547 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:27.546+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:27.550 INFO:teuthology.orchestra.run.smithi118.stdout:flattening image image_client.0-clone25 2024-09-13T07:02:27.608 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:27.608 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:27.609 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:27.619 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:27.648 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:27.644+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:27.725 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:27.725 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:27.725 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:27.725 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:27.725 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:27.735 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:27.768 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:02:27.781 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:27.780+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:27.825 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-13T07:02:27.849 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:27.848+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:27.861 INFO:teuthology.orchestra.run.smithi118.stdout:2354 trunc from 0xdac3a5a to 0x4c75d0d 2024-09-13T07:02:27.888 INFO:teuthology.orchestra.run.smithi118.stdout:2355 read 0x33b2550 thru 0x33bc0bc (0x9b6d bytes) 2024-09-13T07:02:27.888 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:27.888 INFO:teuthology.orchestra.run.smithi118.stdout:2364 punch from 0x8e7c03 to 0x8f6e3a, (0xf237 bytes) 2024-09-13T07:02:27.900 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:434 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'4, mlcod=26'2)", 2024-09-13T07:02:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.365510856, 2024-09-13T07:02:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001931945, 2024-09-13T07:02:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535695+0000", 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535709+0000", 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3577999999999999e-05 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535732+0000", 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3085e-05 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535742+0000", 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.048e-05 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535840+0000", 2024-09-13T07:02:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7159999999999998e-05 2024-09-13T07:02:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535871+0000", 2024-09-13T07:02:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1510999999999997e-05 2024-09-13T07:02:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537572+0000", 2024-09-13T07:02:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017008279999999999 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537629+0000", 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6520000000000001e-05 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:435 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'5, mlcod=26'2)", 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.364379149999998, 2024-09-13T07:02:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017126769999999999, 2024-09-13T07:02:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536835+0000", 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4939999999999997e-06 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536854+0000", 2024-09-13T07:02:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8655999999999999e-05 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536862+0000", 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6570000000000003e-06 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536926+0000", 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4037999999999999e-05 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536958+0000", 2024-09-13T07:02:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1584000000000001e-05 2024-09-13T07:02:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538493+0000", 2024-09-13T07:02:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00153513 2024-09-13T07:02:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538541+0000", 2024-09-13T07:02:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7883999999999998e-05 2024-09-13T07:02:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:483 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'24, mlcod=26'21)", 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.285900521000002, 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018371780000000001, 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615306+0000", 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615322+0000", 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5519000000000002e-05 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615335+0000", 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3854e-05 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615345+0000", 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4930000000000004e-06 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615431+0000", 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6346000000000003e-05 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615480+0000", 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8303999999999997e-05 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617102+0000", 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016225250000000001 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617144+0000", 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1936e-05 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:518 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'38, mlcod=26'36)", 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.576323782999999, 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018201179999999999, 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324883+0000", 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324891+0000", 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1499999999999999e-06 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324924+0000", 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2601999999999997e-05 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324942+0000", 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7796999999999999e-05 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325060+0000", 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011786900000000001 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325123+0000", 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3474999999999998e-05 2024-09-13T07:02:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326620+0000", 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014968830000000001 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326704+0000", 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3716999999999998e-05 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:597 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'39, mlcod=26'37)", 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.459496455, 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017959829999999999, 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441709+0000", 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441728+0000", 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9627999999999999e-05 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441765+0000", 2024-09-13T07:02:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6559e-05 2024-09-13T07:02:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441783+0000", 2024-09-13T07:02:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7691000000000001e-05 2024-09-13T07:02:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441904+0000", 2024-09-13T07:02:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121184 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441966+0000", 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2428999999999995e-05 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443448+0000", 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00148197 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443507+0000", 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8699000000000002e-05 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.844616160999998, 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:02:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:02:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:02:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:02:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1548 3.4 e27/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 27'52, mlcod=27'50)", 2024-09-13T07:02:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.758994625, 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017825779999999999, 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142211+0000", 2024-09-13T07:02:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142232+0000", 2024-09-13T07:02:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0888000000000001e-05 2024-09-13T07:02:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142268+0000", 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6261999999999999e-05 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142285+0000", 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7640000000000001e-05 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142381+0000", 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5171000000000001e-05 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142450+0000", 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9778999999999997e-05 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143953+0000", 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001503129 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143995+0000", 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1878000000000001e-05 2024-09-13T07:02:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1550 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'14, mlcod=27'12)", 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.758346914000001, 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002750452, 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142859+0000", 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142874+0000", 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5149e-05 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142911+0000", 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6439000000000001e-05 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142926+0000", 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5079000000000001e-05 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143012+0000", 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6230000000000006e-05 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143063+0000", 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1369000000000002e-05 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145539+0000", 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024753420000000002 2024-09-13T07:02:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145611+0000", 2024-09-13T07:02:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2392999999999996e-05 2024-09-13T07:02:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1668 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'17, mlcod=27'15)", 2024-09-13T07:02:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.559931452000001, 2024-09-13T07:02:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033976620000000001, 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341281+0000", 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8109999999999999e-06 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341298+0000", 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6575e-05 2024-09-13T07:02:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341308+0000", 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0369e-05 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341434+0000", 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125454 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341453+0000", 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9159000000000001e-05 2024-09-13T07:02:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344638+0000", 2024-09-13T07:02:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031851750000000002 2024-09-13T07:02:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344674+0000", 2024-09-13T07:02:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5410999999999997e-05 2024-09-13T07:02:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1666 3.4 e28/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 28'55, mlcod=27'53)", 2024-09-13T07:02:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.559877780000001, 2024-09-13T07:02:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001805809, 2024-09-13T07:02:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341329+0000", 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341349+0000", 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.999e-05 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341357+0000", 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0620000000000003e-06 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341361+0000", 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341710+0000", 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034873400000000001 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341736+0000", 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5772e-05 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343097+0000", 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013615319999999999 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343135+0000", 2024-09-13T07:02:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8111999999999997e-05 2024-09-13T07:02:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1800 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'20, mlcod=28'18)", 2024-09-13T07:02:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.357507314000003, 2024-09-13T07:02:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001874398, 2024-09-13T07:02:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543699+0000", 2024-09-13T07:02:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543713+0000", 2024-09-13T07:02:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4786e-05 2024-09-13T07:02:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543737+0000", 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3626e-05 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543752+0000", 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.489e-05 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543832+0000", 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0396999999999996e-05 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543994+0000", 2024-09-13T07:02:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016129100000000001 2024-09-13T07:02:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545471+0000", 2024-09-13T07:02:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014771960000000001 2024-09-13T07:02:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545575+0000", 2024-09-13T07:02:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103732 2024-09-13T07:02:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1882 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'23, mlcod=28'21)", 2024-09-13T07:02:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.261874333000002, 2024-09-13T07:02:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001767172, 2024-09-13T07:02:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639332+0000", 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639343+0000", 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1436e-05 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639361+0000", 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7915e-05 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639370+0000", 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3189999999999995e-06 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639425+0000", 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4917999999999998e-05 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639454+0000", 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9862999999999999e-05 2024-09-13T07:02:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641059+0000", 2024-09-13T07:02:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016048950000000001 2024-09-13T07:02:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641100+0000", 2024-09-13T07:02:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0883999999999997e-05 2024-09-13T07:02:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3069 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'33, mlcod=31'31)", 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 58.738734616999999, 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018597010000000001, 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162471+0000", 2024-09-13T07:02:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162520+0000", 2024-09-13T07:02:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8466e-05 2024-09-13T07:02:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162546+0000", 2024-09-13T07:02:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6089999999999999e-05 2024-09-13T07:02:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162562+0000", 2024-09-13T07:02:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5962000000000001e-05 2024-09-13T07:02:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162646+0000", 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3985000000000003e-05 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162691+0000", 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5785999999999998e-05 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164265+0000", 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015740890000000001 2024-09-13T07:02:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164333+0000", 2024-09-13T07:02:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7014999999999994e-05 2024-09-13T07:02:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3121 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'36, mlcod=31'34)", 2024-09-13T07:02:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 58.62478823, 2024-09-13T07:02:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019657780000000001, 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276417+0000", 2024-09-13T07:02:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276497+0000", 2024-09-13T07:02:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9729999999999997e-05 2024-09-13T07:02:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276537+0000", 2024-09-13T07:02:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0102999999999999e-05 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276553+0000", 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6518999999999999e-05 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276662+0000", 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109252 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276729+0000", 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7056000000000003e-05 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278270+0000", 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015408819999999999 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278385+0000", 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114613 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3390 3.0 e32/22 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:head v 32'45, mlcod=31'42)", 2024-09-13T07:02:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.405699882999997, 2024-09-13T07:02:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002246587, 2024-09-13T07:02:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495507+0000", 2024-09-13T07:02:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495530+0000", 2024-09-13T07:02:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2592999999999999e-05 2024-09-13T07:02:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495550+0000", 2024-09-13T07:02:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0152999999999998e-05 2024-09-13T07:02:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495562+0000", 2024-09-13T07:02:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2336e-05 2024-09-13T07:02:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495662+0000", 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100317 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495717+0000", 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4304e-05 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497707+0000", 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019906849999999999 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497754+0000", 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6715999999999998e-05 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3399 3.4 e32/22 3:22863a52:::rbd_data.11945d91e99c.0000000000000013:head v 32'65, mlcod=31'62)", 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.401763211000002, 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028830779999999999, 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499442+0000", 2024-09-13T07:02:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499459+0000", 2024-09-13T07:02:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7042e-05 2024-09-13T07:02:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499493+0000", 2024-09-13T07:02:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3788000000000002e-05 2024-09-13T07:02:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499512+0000", 2024-09-13T07:02:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9055999999999998e-05 2024-09-13T07:02:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499608+0000", 2024-09-13T07:02:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6168000000000005e-05 2024-09-13T07:02:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499658+0000", 2024-09-13T07:02:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9469999999999999e-05 2024-09-13T07:02:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502277+0000", 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026188639999999998 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502327+0000", 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0578999999999998e-05 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3395 3.0 e32/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 32'47, mlcod=31'43)", 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.401665125999997, 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027505870000000001, 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499560+0000", 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8416000000000001e-05 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499576+0000", 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5602000000000001e-05 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499587+0000", 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1112e-05 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499654+0000", 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7164999999999998e-05 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499674+0000", 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9638000000000001e-05 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502210+0000", 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025367100000000002 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502293+0000", 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2607000000000004e-05 2024-09-13T07:02:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:4534 3.0 e32/22 3:00392e9e:::rbd_data.11944cdc0656.000000000000000c:head v 32'51, mlcod=32'49)", 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.989963940000003, 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017597330000000001, 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911242+0000", 2024-09-13T07:02:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911306+0000", 2024-09-13T07:02:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4480999999999995e-05 2024-09-13T07:02:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911338+0000", 2024-09-13T07:02:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1733000000000003e-05 2024-09-13T07:02:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911351+0000", 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3241000000000001e-05 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911455+0000", 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103723 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911495+0000", 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0683000000000003e-05 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.912933+0000", 2024-09-13T07:02:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001437219 2024-09-13T07:02:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.913003+0000", 2024-09-13T07:02:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0629000000000004e-05 2024-09-13T07:02:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11205 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'50, mlcod=38'47)", 2024-09-13T07:02:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.276623454999999, 2024-09-13T07:02:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031049490000000001, 2024-09-13T07:02:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624583+0000", 2024-09-13T07:02:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625339+0000", 2024-09-13T07:02:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00075563200000000001 2024-09-13T07:02:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625358+0000", 2024-09-13T07:02:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9916999999999999e-05 2024-09-13T07:02:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625366+0000", 2024-09-13T07:02:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8050000000000007e-06 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625415+0000", 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8231e-05 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625447+0000", 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2703999999999997e-05 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627652+0000", 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022050659999999999 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627689+0000", 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6612000000000001e-05 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11206 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'52, mlcod=38'47)", 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.275352789000003, 2024-09-13T07:02:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022238470000000002, 2024-09-13T07:02:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625854+0000", 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625863+0000", 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5380000000000008e-06 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625883+0000", 2024-09-13T07:02:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9247000000000001e-05 2024-09-13T07:02:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625893+0000", 2024-09-13T07:02:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0176e-05 2024-09-13T07:02:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626411+0000", 2024-09-13T07:02:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051847600000000003 2024-09-13T07:02:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626431+0000", 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0199000000000001e-05 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628028+0000", 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015968110000000001 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628078+0000", 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0170000000000002e-05 2024-09-13T07:02:27.990 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.991 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.991 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.991 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:27.991 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:27.991 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:27.992 INFO:teuthology.orchestra.run.smithi118.stdout:2366 write 0x9c43998 thru 0x9c4e89a (0xaf03 bytes) 2024-09-13T07:02:27.993 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:27.993 INFO:teuthology.orchestra.run.smithi118.stdout:2367 write 0x2991739 thru 0x299e04d (0xc915 bytes) 2024-09-13T07:02:27.993 INFO:teuthology.orchestra.run.smithi118.stdout:2368 trunc from 0x9c4e89b to 0x6a07272 2024-09-13T07:02:27.993 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:27.994 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:27.994 INFO:teuthology.orchestra.run.smithi118.stdout:2373 read 0x2df5364 thru 0x2dfe06b (0x8d08 bytes) 2024-09-13T07:02:27.994 INFO:teuthology.orchestra.run.smithi118.stdout:2374 writesame 0x387dba8 thru 0x3880dcb data_size 0x2(0x3224 bytes) 2024-09-13T07:02:27.994 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:27.995 INFO:teuthology.orchestra.run.smithi118.stdout:2375 clone 27 order 21 su 262144 sc 2 2024-09-13T07:02:27.995 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:27.995 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:27.995 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:27.988+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:28.006 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:28.029 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:28.052 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:28.080 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:28.082 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:28.082 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:28.082 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:28.082 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:28.082 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:28.082 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:28.082 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:28.082 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:28.093 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:28.090+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:28.111 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:28.111 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:28.121 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:28.193 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:28.192+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:28.275 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:28.275 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:28.285 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:28.294 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:28.290+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:28.391 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:28.391 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:28.395 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:28.394+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:28.401 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:28.496 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:28.492+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:28.516 INFO:teuthology.orchestra.run.smithi118.stdout:leaving image image_client.0-clone26 intact 2024-09-13T07:02:28.519 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:28.519 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:28.529 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:28.597 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:28.594+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:28.637 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:28.637 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:28.649 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:28.674 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-13T07:02:28.699 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:28.698+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:28.755 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:28.755 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:28.765 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:28.799 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:28.798+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:28.875 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:28.875 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:28.884 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:28.900 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:28.898+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:28.987 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:28.987 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:28.997 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:29.001 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:29.000+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:29.091 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:29.091 INFO:teuthology.orchestra.run.smithi118.stdout:2377 read 0x278152d thru 0x2786e27 (0x58fb bytes) 2024-09-13T07:02:29.101 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:29.098+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:29.118 INFO:teuthology.orchestra.run.smithi118.stdout:2378 write 0xa175187 thru 0xa17ff03 (0xad7d bytes) 2024-09-13T07:02:29.124 INFO:teuthology.orchestra.run.smithi118.stdout:2379 read 0x20f4684 thru 0x20f77fe (0x317b bytes) 2024-09-13T07:02:29.125 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:29.127 INFO:teuthology.orchestra.run.smithi118.stdout:2381 trunc from 0xa17ff04 to 0x8fd84f8 2024-09-13T07:02:29.148 INFO:teuthology.orchestra.run.smithi118.stdout:2382 punch from 0x1e8b1e9 to 0x1e8ea84, (0x389b bytes) 2024-09-13T07:02:29.158 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:29.158 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:29.165 INFO:teuthology.orchestra.run.smithi118.stdout:2383 trunc from 0x8fd84f8 to 0x1c8b91 2024-09-13T07:02:29.169 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:29.196 INFO:teuthology.orchestra.run.smithi118.stdout:2384 read 0x42414 thru 0x4fb38 (0xd725 bytes) 2024-09-13T07:02:29.197 INFO:teuthology.orchestra.run.smithi118.stdout:2385 punch from 0x41e15 to 0x427a2, (0x98d bytes) 2024-09-13T07:02:29.197 INFO:teuthology.orchestra.run.smithi118.stdout:2386 punch from 0x109b2d to 0x1197e4, (0xfcb7 bytes) 2024-09-13T07:02:29.197 INFO:teuthology.orchestra.run.smithi118.stdout:2387 read 0x648f1 thru 0x6819e (0x38ae bytes) 2024-09-13T07:02:29.212 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:29.210+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:29.232 INFO:teuthology.orchestra.run.smithi118.stdout:2388 write 0xa0c6357 thru 0xa0c6868 (0x512 bytes) 2024-09-13T07:02:29.237 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:29.237 INFO:teuthology.orchestra.run.smithi118.stdout:2390 read 0x9d2e536 thru 0x9d37c9b (0x9766 bytes) 2024-09-13T07:02:29.238 INFO:teuthology.orchestra.run.smithi118.stdout:2391 read 0x5f3683f thru 0x5f41da3 (0xb565 bytes) 2024-09-13T07:02:29.238 INFO:teuthology.orchestra.run.smithi118.stdout:truncating to largest ever: 0xebe5425 2024-09-13T07:02:29.249 INFO:teuthology.orchestra.run.smithi118.stdout:2392 trunc from 0xa0c6869 to 0xebe5425 2024-09-13T07:02:29.263 INFO:teuthology.orchestra.run.smithi118.stdout:2393 trunc from 0xebe5425 to 0xdc11520 2024-09-13T07:02:29.276 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:29.276 INFO:teuthology.orchestra.run.smithi118.stdout:2395 punch from 0x8a6844f to 0x8a6b975, (0x3526 bytes) 2024-09-13T07:02:29.276 INFO:teuthology.orchestra.run.smithi118.stdout:2397 writesame 0xa1a5db5 thru 0xa1b4acf data_size 0x1(0xed1b bytes) 2024-09-13T07:02:29.304 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:29.302+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:29.322 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:29.322 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:29.332 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:29.349 INFO:teuthology.orchestra.run.smithi118.stdout:2398 trunc from 0xdc11520 to 0xea565 2024-09-13T07:02:29.401 INFO:teuthology.orchestra.run.smithi118.stdout:2399 write 0xbe15bef thru 0xbe25437 (0xf849 bytes) 2024-09-13T07:02:29.405 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:29.404+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:29.407 INFO:teuthology.orchestra.run.smithi118.stdout:2400 read 0x1f62beb thru 0x1f68396 (0x57ac bytes) 2024-09-13T07:02:29.407 INFO:teuthology.orchestra.run.smithi118.stdout:2401 read 0x33197e7 thru 0x331d293 (0x3aad bytes) 2024-09-13T07:02:29.408 INFO:teuthology.orchestra.run.smithi118.stdout:2403 trunc from 0xbe25438 to 0xc663503 2024-09-13T07:02:29.417 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:29.421 INFO:teuthology.orchestra.run.smithi118.stdout:2405 trunc from 0xc663503 to 0xe2d021e 2024-09-13T07:02:29.432 INFO:teuthology.orchestra.run.smithi118.stdout:2406 writesame 0x53efa78 thru 0x53f8697 data_size 0x26(0x8c20 bytes) 2024-09-13T07:02:29.438 INFO:teuthology.orchestra.run.smithi118.stdout:2407 punch from 0x91e11d1 to 0x91ee433, (0xd262 bytes) 2024-09-13T07:02:29.438 INFO:teuthology.orchestra.run.smithi118.stdout:2409 punch from 0x84a3caf to 0x84aacd3, (0x7024 bytes) 2024-09-13T07:02:29.438 INFO:teuthology.orchestra.run.smithi118.stdout:2410 write 0x8b23b43 thru 0x8b32106 (0xe5c4 bytes) 2024-09-13T07:02:29.444 INFO:teuthology.orchestra.run.smithi118.stdout:2411 write 0xcfacc8d thru 0xcfad770 (0xae4 bytes) 2024-09-13T07:02:29.448 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:02:29.449 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:29.450 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:29.451 INFO:teuthology.orchestra.run.smithi118.stdout:2412 read 0x85310c1 thru 0x853e9b5 (0xd8f5 bytes) 2024-09-13T07:02:29.460 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:29.474 INFO:teuthology.orchestra.run.smithi118.stdout:2413 trunc from 0xe2d021e to 0x239846e 2024-09-13T07:02:29.493 INFO:teuthology.orchestra.run.smithi118.stdout:2414 read 0x1c1dbb9 thru 0x1c21bcb (0x4013 bytes) 2024-09-13T07:02:29.494 INFO:teuthology.orchestra.run.smithi118.stdout:2416 write 0x185ee40 thru 0x186a391 (0xb552 bytes) 2024-09-13T07:02:29.499 INFO:teuthology.orchestra.run.smithi118.stdout:2417 punch from 0x1e2e026 to 0x1e398aa, (0xb884 bytes) 2024-09-13T07:02:29.499 INFO:teuthology.orchestra.run.smithi118.stdout:2418 read 0x12b8f52 thru 0x12beaf3 (0x5ba2 bytes) 2024-09-13T07:02:29.499 INFO:teuthology.orchestra.run.smithi118.stdout:2420 writesame 0x1ad5682 thru 0x1adccbb data_size 0x2(0x763a bytes) 2024-09-13T07:02:29.507 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:29.504+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:29.520 INFO:teuthology.orchestra.run.smithi118.stdout:2422 trunc from 0x239846e to 0x5a09fe6 2024-09-13T07:02:29.546 INFO:teuthology.orchestra.run.smithi118.stdout:2423 write 0x7e64202 thru 0x7e6b587 (0x7386 bytes) 2024-09-13T07:02:29.553 INFO:teuthology.orchestra.run.smithi118.stdout:2424 write 0x5890af5 thru 0x589bb89 (0xb095 bytes) 2024-09-13T07:02:29.570 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:29.570 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:29.581 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:29.581 INFO:teuthology.orchestra.run.smithi118.stdout:2425 writesame 0xd7ebd24 thru 0xd7f7c1c data_size 0x1(0xbef9 bytes) 2024-09-13T07:02:29.582 INFO:teuthology.orchestra.run.smithi118.stdout:2426 write 0x509220e thru 0x5098ff9 (0x6dec bytes) 2024-09-13T07:02:29.587 INFO:teuthology.orchestra.run.smithi118.stdout:2427 writesame 0x539973e thru 0x5399df3 data_size 0x1(0x6b6 bytes) 2024-09-13T07:02:29.591 INFO:teuthology.orchestra.run.smithi118.stdout:2428 read 0x772be49 thru 0x772e5c0 (0x2778 bytes) 2024-09-13T07:02:29.591 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:29.591 INFO:teuthology.orchestra.run.smithi118.stdout:2430 writesame 0x4acc344 thru 0x4acc62f data_size 0x11(0x2ec bytes) 2024-09-13T07:02:29.596 INFO:teuthology.orchestra.run.smithi118.stdout:2431 write 0x581adf8 thru 0x5821dbd (0x6fc6 bytes) 2024-09-13T07:02:29.600 INFO:teuthology.orchestra.run.smithi118.stdout:2432 read 0x1058d35 thru 0x105fde7 (0x70b3 bytes) 2024-09-13T07:02:29.600 INFO:teuthology.orchestra.run.smithi118.stdout:2434 read 0xad3d24e thru 0xad3f9e3 (0x2796 bytes) 2024-09-13T07:02:29.600 INFO:teuthology.orchestra.run.smithi118.stdout:2435 write 0x1fb4918 thru 0x1fc3381 (0xea6a bytes) 2024-09-13T07:02:29.607 INFO:teuthology.orchestra.run.smithi118.stdout:2436 punch from 0x7354443 to 0x735988a, (0x5447 bytes) 2024-09-13T07:02:29.607 INFO:teuthology.orchestra.run.smithi118.stdout:2438 read 0x85a43f9 thru 0x85b3d21 (0xf929 bytes) 2024-09-13T07:02:29.617 INFO:teuthology.orchestra.run.smithi118.stdout:2440 write 0xe22e129 thru 0xe2385dd (0xa4b5 bytes) 2024-09-13T07:02:29.625 INFO:teuthology.orchestra.run.smithi118.stdout:2441 writesame 0x266e6dc thru 0x2674fde data_size 0x57(0x6903 bytes) 2024-09-13T07:02:29.627 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:29.626+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:29.630 INFO:teuthology.orchestra.run.smithi118.stdout:2442 writesame 0x9f6fe48 thru 0x9f77385 data_size 0x2b(0x753e bytes) 2024-09-13T07:02:29.636 INFO:teuthology.orchestra.run.smithi118.stdout:2443 writesame 0x9e08c55 thru 0x9e0b19a data_size 0x1(0x2546 bytes) 2024-09-13T07:02:29.649 INFO:teuthology.orchestra.run.smithi118.stdout:2444 writesame 0x7dfe6a4 thru 0x7e07a7b data_size 0x53(0x93d8 bytes) 2024-09-13T07:02:29.654 INFO:teuthology.orchestra.run.smithi118.stdout:2447 punch from 0x1bf32ec to 0x1bf9d7e, (0x6a92 bytes) 2024-09-13T07:02:29.654 INFO:teuthology.orchestra.run.smithi118.stdout:2448 write 0x27f2229 thru 0x27f62a1 (0x4079 bytes) 2024-09-13T07:02:29.656 INFO:teuthology.orchestra.run.smithi118.stdout:2449 write 0x7d801b9 thru 0x7d8ab7c (0xa9c4 bytes) 2024-09-13T07:02:29.658 INFO:teuthology.orchestra.run.smithi118.stdout:2450 read 0x5244949 thru 0x5248705 (0x3dbd bytes) 2024-09-13T07:02:29.659 INFO:teuthology.orchestra.run.smithi118.stdout:2451 read 0x878f643 thru 0x879d003 (0xd9c1 bytes) 2024-09-13T07:02:29.659 INFO:teuthology.orchestra.run.smithi118.stdout:2456 read 0xab58d7d thru 0xab5adcc (0x2050 bytes) 2024-09-13T07:02:29.659 INFO:teuthology.orchestra.run.smithi118.stdout:2459 punch from 0x82cf873 to 0x82cfcf0, (0x47d bytes) 2024-09-13T07:02:29.659 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:29.659 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:29.659 INFO:teuthology.orchestra.run.smithi118.stdout:2462 writesame 0x6ad98e0 thru 0x6ade997 data_size 0x3f(0x50b8 bytes) 2024-09-13T07:02:29.687 INFO:teuthology.orchestra.run.smithi118.stdout:2463 trunc from 0xe2385de to 0xac2fb6 2024-09-13T07:02:29.688 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:29.688 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:29.699 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:29.707 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:29.706+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:29.708 INFO:teuthology.orchestra.run.smithi118.stdout:2465 punch from 0x13751 to 0x1b51e, (0x7dcd bytes) 2024-09-13T07:02:29.727 INFO:teuthology.orchestra.run.smithi118.stdout:2466 trunc from 0xac2fb6 to 0xc999a70 2024-09-13T07:02:29.752 INFO:teuthology.orchestra.run.smithi118.stdout:2467 write 0xe115877 thru 0xe117c0f (0x2399 bytes) 2024-09-13T07:02:29.757 INFO:teuthology.orchestra.run.smithi118.stdout:2468 writesame 0x4d5d357 thru 0x4d69e6f data_size 0x9(0xcb19 bytes) 2024-09-13T07:02:29.766 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:29.767 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:29.767 INFO:teuthology.orchestra.run.smithi118.stdout:2472 read 0xab9e95c thru 0xaba5f4a (0x75ef bytes) 2024-09-13T07:02:29.767 INFO:teuthology.orchestra.run.smithi118.stdout:2473 punch from 0xcd20189 to 0xcd2baa4, (0xb91b bytes) 2024-09-13T07:02:29.779 INFO:teuthology.orchestra.run.smithi118.stdout:2474 write 0xeddd3e0 thru 0xede7f32 (0xab53 bytes) 2024-09-13T07:02:29.785 INFO:teuthology.orchestra.run.smithi118.stdout:2475 punch from 0x1b8c173 to 0x1b93cbd, (0x7b4a bytes) 2024-09-13T07:02:29.785 INFO:teuthology.orchestra.run.smithi118.stdout:2477 write 0x911b39a thru 0x911dc05 (0x286c bytes) 2024-09-13T07:02:29.790 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:29.790 INFO:teuthology.orchestra.run.smithi118.stdout:2479 writesame 0xd724225 thru 0xd72b280 data_size 0x66(0x705c bytes) 2024-09-13T07:02:29.796 INFO:teuthology.orchestra.run.smithi118.stdout:2481 read 0xe9e5115 thru 0xe9eac7f (0x5b6b bytes) 2024-09-13T07:02:29.796 INFO:teuthology.orchestra.run.smithi118.stdout:2482 read 0xc111792 thru 0xc11b353 (0x9bc2 bytes) 2024-09-13T07:02:29.799 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:29.799 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:29.804 INFO:teuthology.orchestra.run.smithi118.stdout:2483 trunc from 0xede7f33 to 0xafa6d72 2024-09-13T07:02:29.808 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:29.806+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:29.810 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:29.825 INFO:teuthology.orchestra.run.smithi118.stdout:2484 read 0x75fe79a thru 0x760361b (0x4e82 bytes) 2024-09-13T07:02:29.837 INFO:teuthology.orchestra.run.smithi118.stdout:2485 trunc from 0xafa6d72 to 0x59b2715 2024-09-13T07:02:29.859 INFO:teuthology.orchestra.run.smithi118.stdout:2486 read 0x191b627 thru 0x192aefe (0xf8d8 bytes) 2024-09-13T07:02:29.859 INFO:teuthology.orchestra.run.smithi118.stdout:2489 read 0x474b3ac thru 0x47573f8 (0xc04d bytes) 2024-09-13T07:02:29.859 INFO:teuthology.orchestra.run.smithi118.stdout:2490 read 0x17d05fd thru 0x17d6e6f (0x6873 bytes) 2024-09-13T07:02:29.860 INFO:teuthology.orchestra.run.smithi118.stdout:2492 punch from 0x3abeba9 to 0x3ac14df, (0x2936 bytes) 2024-09-13T07:02:29.860 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:29.860 INFO:teuthology.orchestra.run.smithi118.stdout:2494 read 0x2e53597 thru 0x2e5f364 (0xbdce bytes) 2024-09-13T07:02:29.878 INFO:teuthology.orchestra.run.smithi118.stdout:2495 writesame 0x7af03fd thru 0x7afc67e data_size 0x56(0xc282 bytes) 2024-09-13T07:02:29.878 INFO:teuthology.orchestra.run.smithi118.stdout:2496 read 0x5930b93 thru 0x5937d0c (0x717a bytes) 2024-09-13T07:02:29.878 INFO:teuthology.orchestra.run.smithi118.stdout:2497 write 0x423430f thru 0x424280c (0xe4fe bytes) 2024-09-13T07:02:29.886 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:29.886 INFO:teuthology.orchestra.run.smithi118.stdout:2500 write 0x2e40649 thru 0x2e44ac5 (0x447d bytes) 2024-09-13T07:02:29.892 INFO:teuthology.orchestra.run.smithi118.stdout:2501 writesame 0x238c61c thru 0x239a5da data_size 0x3d(0xdfbf bytes) 2024-09-13T07:02:29.899 INFO:teuthology.orchestra.run.smithi118.stdout:2502 writesame 0x251429e thru 0x2521bfd data_size 0x25(0xd960 bytes) 2024-09-13T07:02:29.906 INFO:teuthology.orchestra.run.smithi118.stdout:2503 write 0x302c35e thru 0x3037f4a (0xbbed bytes) 2024-09-13T07:02:29.909 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:29.906+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:29.942 INFO:teuthology.orchestra.run.smithi118.stdout:2504 writesame 0xe0c6d99 thru 0xe0d3b1d data_size 0x1(0xcd85 bytes) 2024-09-13T07:02:29.942 INFO:teuthology.orchestra.run.smithi118.stdout:2505 writesame 0x95bbc65 thru 0x95bead3 data_size 0x1(0x2e6f bytes) 2024-09-13T07:02:29.957 INFO:teuthology.orchestra.run.smithi118.stdout:2507 punch from 0x4f71fa3 to 0x4f7d38a, (0xb3e7 bytes) 2024-09-13T07:02:29.957 INFO:teuthology.orchestra.run.smithi118.stdout:2508 write 0x500f099 thru 0x5010125 (0x108d bytes) 2024-09-13T07:02:29.967 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:29.967 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:29.977 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:29.978 INFO:teuthology.orchestra.run.smithi118.stdout:2509 trunc from 0xe0d3b1e to 0x721e7d4 2024-09-13T07:02:30.000 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:30.000 INFO:teuthology.orchestra.run.smithi118.stdout:2511 read 0x42c6c82 thru 0x42d6a0f (0xfd8e bytes) 2024-09-13T07:02:30.007 INFO:teuthology.orchestra.run.smithi118.stdout:2512 trunc from 0x721e7d4 to 0x9df08f7 2024-09-13T07:02:30.010 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:30.006+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:30.018 INFO:teuthology.orchestra.run.smithi118.stdout:2513 write 0x4c74d35 thru 0x4c7a4b4 (0x5780 bytes) 2024-09-13T07:02:30.021 INFO:teuthology.orchestra.run.smithi118.stdout:2515 punch from 0x5e92120 to 0x5e9e210, (0xc0f0 bytes) 2024-09-13T07:02:30.022 INFO:teuthology.orchestra.run.smithi118.stdout:2516 write 0x5bf1e92 thru 0x5bf5487 (0x35f6 bytes) 2024-09-13T07:02:30.028 INFO:teuthology.orchestra.run.smithi118.stdout:2517 clone 28 order 18 su 1024 sc 7 2024-09-13T07:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:30.093 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:02:30.102 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:30.145 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:30.144+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:30.213 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:30.210+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:30.248 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:30.254 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:30.254 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:30.254 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:30.254 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:30.254 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:30.264 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:30.270 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:30.288 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:30.306 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:30.314 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:30.310+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:30.329 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:30.351 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:30.369 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:02:30.370 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:02:30.370 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-13T07:02:30.370 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T07:02:30.370 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T07:02:30.370 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-13T07:02:30.372 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:30.372 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:30.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:30.372 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:30.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.372 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:30.372 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:30.372 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.864494879000006, 2024-09-13T07:02:30.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:02:30.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:02:30.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:30.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:02:30.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:02:30.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:30.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:02:30.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:02:30.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:30.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:02:30.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:02:30.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:02:30.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:02:30.378 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.378 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.378 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.378 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:30.378 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:30.378 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.839667168999995, 2024-09-13T07:02:30.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:02:30.378 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:02:30.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:02:30.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:02:30.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:02:30.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:02:30.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:02:30.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:02:30.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:02:30.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.300995345000004, 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.384 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:02:30.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.385 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:30.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:30.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:02:30.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:30.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:02:30.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:02:30.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:02:30.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.389 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.271370152000003, 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:02:30.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.391 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:30.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:30.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:02:30.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:30.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:02:30.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:02:30.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:02:30.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:02:30.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:02:30.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:02:30.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:02:30.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:02:30.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:02:30.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:02:30.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:30.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:02:30.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:02:30.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:30.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:02:30.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.738656521000003, 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.396 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:02:30.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:02:30.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:30.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.738574065000002, 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:30.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:02:30.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:30.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:30.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:02:30.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:02:30.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:02:30.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:02:30.407 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.407 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.407 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.407 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.738542254999999, 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:02:30.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:02:30.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:02:30.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:02:30.413 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.413 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.413 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.413 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:30.413 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:30.413 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.738487096999997, 2024-09-13T07:02:30.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:02:30.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:02:30.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:02:30.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:02:30.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:02:30.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:02:30.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:02:30.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:02:30.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:02:30.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:30.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:02:30.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:02:30.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:02:30.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:02:30.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:02:30.419 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.419 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.419 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.419 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:30.419 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:30.420 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.738441817999998, 2024-09-13T07:02:30.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:02:30.420 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.420 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:30.420 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.420 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.420 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.420 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:02:30.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.420 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:30.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:30.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:02:30.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:30.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:02:30.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:30.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:30.425 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.738403284, 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:30.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:02:30.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:30.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:02:30.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:02:30.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:02:30.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:02:30.431 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.431 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.431 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.431 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:30.431 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:30.431 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.738341665, 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:02:30.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:02:30.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:02:30.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:02:30.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:02:30.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:30.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:02:30.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:02:30.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:30.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:02:30.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:02:30.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:30.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:02:30.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:02:30.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 63.724118982, 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:02:30.437 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.438 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:30.438 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.438 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.438 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.438 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:02:30.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.438 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:30.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:02:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:02:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:02:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:02:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:02:30.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:30.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.712240461999997, 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.443 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.445 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.445 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.445 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:02:30.445 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.445 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.445 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.445 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:02:30.445 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.447 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.711683401000002, 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.448 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:30.449 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:30.449 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.450 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:02:30.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:02:30.452 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.452 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.452 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.452 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:30.452 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:30.452 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.711661245999998, 2024-09-13T07:02:30.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:02:30.452 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.452 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.453 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.454 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:02:30.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:02:30.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.455 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:02:30.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:02:30.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.455 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:02:30.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:02:30.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.456 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:02:30.456 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:02:30.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.456 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:02:30.456 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:02:30.456 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.456 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.457 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.457 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.457 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.457 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:30.457 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:30.457 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.711641528999998, 2024-09-13T07:02:30.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:02:30.457 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.457 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:30.457 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.457 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:02:30.459 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:02:30.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.460 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:02:30.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:02:30.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.460 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:02:30.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:02:30.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.711624198999999, 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.462 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.463 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:02:30.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:02:30.465 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.466 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.466 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.466 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.466 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.466 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:30.466 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:30.466 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.711328537, 2024-09-13T07:02:30.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:02:30.466 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.467 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:30.468 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.468 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.468 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.468 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.468 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:30.468 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.468 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.468 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.468 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.468 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:02:30.468 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:02:30.468 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.468 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.469 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:02:30.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:02:30.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.469 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:02:30.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:02:30.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.469 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:02:30.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.150125717999998, 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.471 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.472 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:30.472 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.472 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.472 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.472 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.472 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:30.472 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.472 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.472 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.472 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.472 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:02:30.472 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:30.472 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:02:30.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:02:30.473 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:02:30.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:02:30.473 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.474 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.475 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.476 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:30.476 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:02:30.476 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:02:30.476 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.476 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.476 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:02:30.476 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:02:30.476 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.476 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.476 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.476 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.477 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:02:30.477 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:30.477 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.118046815, 2024-09-13T07:02:30.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:02:30.477 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.477 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:30.477 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.477 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.477 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.477 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:30.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.477 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.477 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.478 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:30.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.478 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:02:30.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:30.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.478 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:02:30.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:02:30.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.480 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.481 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.481 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:30.482 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:30.483 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:30.483 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:30.484 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:30.484 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:30.484 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:30.484 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:30.484 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:30.484 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:30.484 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:30.484 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:30.484 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:02:30.489 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:30.480+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:30.546 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone27 from 0x9df08f7 (overlap 0xea565) to 0x0 2024-09-13T07:02:30.583 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:30.580+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:30.602 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:30.602 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T07:02:30.602 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:30.637 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:30.637 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:30.638 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:30.638 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:30.648 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:30.684 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:30.682+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:30.685 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-13T07:02:30.758 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:30.758 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:30.758 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:30.758 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:30.759 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:30.768 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:30.813 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:30.812+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:30.886 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:30.882+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:30.922 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:02:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.429566479000002, 2024-09-13T07:02:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:02:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.923 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.924 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:02:30.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.924 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:30.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:30.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:02:30.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:30.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:02:30.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:02:30.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:02:30.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:02:30.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:02:30.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:02:30.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:02:30.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:02:30.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:02:30.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:02:30.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:02:30.928 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.928 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.928 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.928 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.928 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:30.929 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.943082914999998, 2024-09-13T07:02:30.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:02:30.929 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.929 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:30.930 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.930 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.930 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.930 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:02:30.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.930 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:02:30.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:02:30.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:02:30.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:02:30.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:02:30.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:02:30.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:02:30.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:02:30.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:02:30.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:02:30.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:02:30.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:02:30.934 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.934 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.934 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.934 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.934 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:30.934 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.943012985999999, 2024-09-13T07:02:30.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:02:30.934 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.934 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:30.934 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:30.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:02:30.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:02:30.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:02:30.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:02:30.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:02:30.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:02:30.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:02:30.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:02:30.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:02:30.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:02:30.938 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.938 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.938 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.939 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.939 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:30.939 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.942997114000001, 2024-09-13T07:02:30.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:02:30.939 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.939 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:30.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.939 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:02:30.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.939 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:02:30.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:02:30.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:02:30.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:02:30.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.942926657000001, 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:30.943 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.944 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.944 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.944 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:02:30.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.944 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:30.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:02:30.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:02:30.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:02:30.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:02:30.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:02:30.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:02:30.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:30.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:02:30.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:02:30.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:02:30.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:02:30.948 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.948 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.948 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.948 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.948 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:30.948 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939574618999998, 2024-09-13T07:02:30.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:02:30.948 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.948 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:30.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.949 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.949 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:02:30.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.949 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:30.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:30.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:02:30.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:02:30.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:02:30.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:02:30.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:02:30.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:02:30.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:02:30.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:02:30.952 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.952 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.952 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.952 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.952 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:30.952 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939558318000003, 2024-09-13T07:02:30.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:02:30.952 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.953 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:30.953 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.953 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.953 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.953 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:02:30.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.953 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:30.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:02:30.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:02:30.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:02:30.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:02:30.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:02:30.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:02:30.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:02:30.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:02:30.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:02:30.956 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.956 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.956 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.956 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.956 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:30.957 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939538532999997, 2024-09-13T07:02:30.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:02:30.957 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.957 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:30.957 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.957 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.957 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.957 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:02:30.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.957 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.960 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.961 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:30.961 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939522259999997, 2024-09-13T07:02:30.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:02:30.961 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.961 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:30.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.961 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.961 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:02:30.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.961 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:02:30.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:02:30.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:02:30.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:02:30.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:02:30.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939504435000003, 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:02:30.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:02:30.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:02:30.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:02:30.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:02:30.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:02:30.968 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.968 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.968 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.969 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.969 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:30.969 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939488341999997, 2024-09-13T07:02:30.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:02:30.969 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.969 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:30.969 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.969 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.969 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.969 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:02:30.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:02:30.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:02:30.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:02:30.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:02:30.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:02:30.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:02:30.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:02:30.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:02:30.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:02:30.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:02:30.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:02:30.973 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.973 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.973 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.973 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.973 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:30.973 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939467155999999, 2024-09-13T07:02:30.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:02:30.973 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.973 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:30.973 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.973 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.974 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.974 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:02:30.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.974 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:30.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:30.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:02:30.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:02:30.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:02:30.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:02:30.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:02:30.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:02:30.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:02:30.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:02:30.977 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.977 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.977 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.977 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.977 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:30.977 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939451122999998, 2024-09-13T07:02:30.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:02:30.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:02:30.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:02:30.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:02:30.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:02:30.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.981 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939401883999999, 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:30.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:02:30.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:02:30.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:02:30.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:02:30.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:02:30.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:02:30.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.985 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939383823999997, 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:02:30.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:02:30.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:02:30.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:02:30.989 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.989 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.989 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939367308999998, 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:30.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:30.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:02:30.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:02:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939348944000002, 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:02:30.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:30.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:02:30.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:02:30.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:30.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:02:30.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:02:30.997 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.997 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:30.997 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:30.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939332845999999, 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:30.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:30.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:30.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:30.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:30.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:30.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:30.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:30.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:30.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.000 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.001 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.002 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.002 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:31.002 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:31.002 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939316910999999, 2024-09-13T07:02:31.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:02:31.002 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.002 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:31.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.002 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.002 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:02:31.005 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.006 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.006 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.006 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:31.006 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:31.006 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.939300414000002, 2024-09-13T07:02:31.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:02:31.006 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.006 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:31.006 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.006 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.006 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:31.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:02:31.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:02:31.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.008 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.008 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:02:31.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:02:31.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:02:31.010 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.010 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.010 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.010 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.010 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.010 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:31.011 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:31.012 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:31.010+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:31.113 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:31.108+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:31.121 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:31.121 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:31.121 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:31.121 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:31.131 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:31.150 INFO:teuthology.orchestra.run.smithi118.stdout:2519 trunc from 0x9df08f7 to 0x7f47b5d 2024-09-13T07:02:31.202 INFO:teuthology.orchestra.run.smithi118.stdout:2520 write 0x92b36fc thru 0x92bcbf9 (0x94fe bytes) 2024-09-13T07:02:31.213 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:31.212+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:31.222 INFO:teuthology.orchestra.run.smithi118.stdout:2521 read 0x18ba94 thru 0x19ad02 (0xf26f bytes) 2024-09-13T07:02:31.235 INFO:teuthology.orchestra.run.smithi118.stdout:2523 write 0x6e9cfc thru 0x6f5f9f (0xc2a4 bytes) 2024-09-13T07:02:31.241 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:31.241 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:31.241 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:31.241 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:31.241 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:31.252 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:31.263 INFO:teuthology.orchestra.run.smithi118.stdout:2524 trunc from 0x92bcbfa to 0x502af3c 2024-09-13T07:02:31.314 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:31.310+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:31.376 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:31.377 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:31.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:31.377 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:31.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.377 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:434 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'4, mlcod=26'2)", 2024-09-13T07:02:31.377 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:31.377 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 66.840034622000005, 2024-09-13T07:02:31.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001931945, 2024-09-13T07:02:31.377 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.377 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.377 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:31.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:31.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535695+0000", 2024-09-13T07:02:31.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:31.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535709+0000", 2024-09-13T07:02:31.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3577999999999999e-05 2024-09-13T07:02:31.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535732+0000", 2024-09-13T07:02:31.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3085e-05 2024-09-13T07:02:31.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535742+0000", 2024-09-13T07:02:31.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.048e-05 2024-09-13T07:02:31.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535840+0000", 2024-09-13T07:02:31.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7159999999999998e-05 2024-09-13T07:02:31.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535871+0000", 2024-09-13T07:02:31.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1510999999999997e-05 2024-09-13T07:02:31.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537572+0000", 2024-09-13T07:02:31.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017008279999999999 2024-09-13T07:02:31.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537629+0000", 2024-09-13T07:02:31.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6520000000000001e-05 2024-09-13T07:02:31.382 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.382 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.382 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.383 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:435 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'5, mlcod=26'2)", 2024-09-13T07:02:31.383 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:31.383 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 66.838902915999995, 2024-09-13T07:02:31.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017126769999999999, 2024-09-13T07:02:31.383 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.383 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.383 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:31.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:31.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:31.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:31.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536835+0000", 2024-09-13T07:02:31.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4939999999999997e-06 2024-09-13T07:02:31.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536854+0000", 2024-09-13T07:02:31.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8655999999999999e-05 2024-09-13T07:02:31.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536862+0000", 2024-09-13T07:02:31.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6570000000000003e-06 2024-09-13T07:02:31.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536926+0000", 2024-09-13T07:02:31.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4037999999999999e-05 2024-09-13T07:02:31.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536958+0000", 2024-09-13T07:02:31.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1584000000000001e-05 2024-09-13T07:02:31.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538493+0000", 2024-09-13T07:02:31.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00153513 2024-09-13T07:02:31.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538541+0000", 2024-09-13T07:02:31.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7883999999999998e-05 2024-09-13T07:02:31.387 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.387 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.388 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.388 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:483 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'24, mlcod=26'21)", 2024-09-13T07:02:31.388 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:31.388 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 66.760424287000006, 2024-09-13T07:02:31.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018371780000000001, 2024-09-13T07:02:31.388 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.388 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.388 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:31.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:31.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615306+0000", 2024-09-13T07:02:31.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:31.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615322+0000", 2024-09-13T07:02:31.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5519000000000002e-05 2024-09-13T07:02:31.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615335+0000", 2024-09-13T07:02:31.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3854e-05 2024-09-13T07:02:31.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615345+0000", 2024-09-13T07:02:31.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4930000000000004e-06 2024-09-13T07:02:31.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615431+0000", 2024-09-13T07:02:31.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6346000000000003e-05 2024-09-13T07:02:31.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615480+0000", 2024-09-13T07:02:31.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8303999999999997e-05 2024-09-13T07:02:31.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617102+0000", 2024-09-13T07:02:31.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016225250000000001 2024-09-13T07:02:31.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617144+0000", 2024-09-13T07:02:31.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1936e-05 2024-09-13T07:02:31.393 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.393 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.393 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.394 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:518 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'38, mlcod=26'36)", 2024-09-13T07:02:31.394 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:31.394 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 66.050847548999997, 2024-09-13T07:02:31.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018201179999999999, 2024-09-13T07:02:31.394 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.394 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.394 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:31.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:31.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324883+0000", 2024-09-13T07:02:31.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324891+0000", 2024-09-13T07:02:31.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1499999999999999e-06 2024-09-13T07:02:31.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324924+0000", 2024-09-13T07:02:31.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2601999999999997e-05 2024-09-13T07:02:31.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324942+0000", 2024-09-13T07:02:31.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7796999999999999e-05 2024-09-13T07:02:31.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325060+0000", 2024-09-13T07:02:31.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011786900000000001 2024-09-13T07:02:31.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325123+0000", 2024-09-13T07:02:31.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3474999999999998e-05 2024-09-13T07:02:31.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326620+0000", 2024-09-13T07:02:31.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014968830000000001 2024-09-13T07:02:31.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326704+0000", 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3716999999999998e-05 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:597 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'39, mlcod=26'37)", 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.934020220999997, 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017959829999999999, 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.399 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:31.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:31.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441709+0000", 2024-09-13T07:02:31.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:31.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441728+0000", 2024-09-13T07:02:31.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9627999999999999e-05 2024-09-13T07:02:31.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441765+0000", 2024-09-13T07:02:31.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6559e-05 2024-09-13T07:02:31.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441783+0000", 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7691000000000001e-05 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441904+0000", 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121184 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441966+0000", 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2428999999999995e-05 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443448+0000", 2024-09-13T07:02:31.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00148197 2024-09-13T07:02:31.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443507+0000", 2024-09-13T07:02:31.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8699000000000002e-05 2024-09-13T07:02:31.404 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.404 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.404 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.405 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:02:31.405 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:31.405 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.319139926999995, 2024-09-13T07:02:31.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:02:31.405 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.405 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.405 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:31.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:02:31.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:02:31.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:02:31.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:02:31.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:02:31.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:02:31.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:02:31.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:02:31.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:02:31.409 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.409 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.409 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.409 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1548 3.4 e27/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 27'52, mlcod=27'50)", 2024-09-13T07:02:31.409 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:31.410 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.233518391000004, 2024-09-13T07:02:31.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017825779999999999, 2024-09-13T07:02:31.410 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.410 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.410 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:31.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:31.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142211+0000", 2024-09-13T07:02:31.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:31.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142232+0000", 2024-09-13T07:02:31.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0888000000000001e-05 2024-09-13T07:02:31.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142268+0000", 2024-09-13T07:02:31.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6261999999999999e-05 2024-09-13T07:02:31.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142285+0000", 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7640000000000001e-05 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142381+0000", 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5171000000000001e-05 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142450+0000", 2024-09-13T07:02:31.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9778999999999997e-05 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143953+0000", 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001503129 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143995+0000", 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1878000000000001e-05 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.416 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1550 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'14, mlcod=27'12)", 2024-09-13T07:02:31.416 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:31.416 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.232870680000005, 2024-09-13T07:02:31.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002750452, 2024-09-13T07:02:31.416 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.416 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.416 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:31.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:31.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142859+0000", 2024-09-13T07:02:31.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:31.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142874+0000", 2024-09-13T07:02:31.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5149e-05 2024-09-13T07:02:31.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142911+0000", 2024-09-13T07:02:31.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6439000000000001e-05 2024-09-13T07:02:31.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142926+0000", 2024-09-13T07:02:31.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5079000000000001e-05 2024-09-13T07:02:31.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143012+0000", 2024-09-13T07:02:31.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6230000000000006e-05 2024-09-13T07:02:31.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143063+0000", 2024-09-13T07:02:31.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1369000000000002e-05 2024-09-13T07:02:31.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145539+0000", 2024-09-13T07:02:31.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024753420000000002 2024-09-13T07:02:31.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145611+0000", 2024-09-13T07:02:31.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2392999999999996e-05 2024-09-13T07:02:31.422 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.422 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.422 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.422 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1668 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'17, mlcod=27'15)", 2024-09-13T07:02:31.422 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:31.422 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.034455218000005, 2024-09-13T07:02:31.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033976620000000001, 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:31.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341281+0000", 2024-09-13T07:02:31.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8109999999999999e-06 2024-09-13T07:02:31.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341298+0000", 2024-09-13T07:02:31.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6575e-05 2024-09-13T07:02:31.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341308+0000", 2024-09-13T07:02:31.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0369e-05 2024-09-13T07:02:31.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341434+0000", 2024-09-13T07:02:31.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125454 2024-09-13T07:02:31.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341453+0000", 2024-09-13T07:02:31.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9159000000000001e-05 2024-09-13T07:02:31.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344638+0000", 2024-09-13T07:02:31.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031851750000000002 2024-09-13T07:02:31.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344674+0000", 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5410999999999997e-05 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1666 3.4 e28/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 28'55, mlcod=27'53)", 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.034401545999998, 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001805809, 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341329+0000", 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341349+0000", 2024-09-13T07:02:31.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.999e-05 2024-09-13T07:02:31.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341357+0000", 2024-09-13T07:02:31.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0620000000000003e-06 2024-09-13T07:02:31.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341361+0000", 2024-09-13T07:02:31.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:02:31.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341710+0000", 2024-09-13T07:02:31.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034873400000000001 2024-09-13T07:02:31.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341736+0000", 2024-09-13T07:02:31.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5772e-05 2024-09-13T07:02:31.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343097+0000", 2024-09-13T07:02:31.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013615319999999999 2024-09-13T07:02:31.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343135+0000", 2024-09-13T07:02:31.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8111999999999997e-05 2024-09-13T07:02:31.433 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.433 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.434 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.434 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1800 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'20, mlcod=28'18)", 2024-09-13T07:02:31.434 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:31.434 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.832031079999993, 2024-09-13T07:02:31.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001874398, 2024-09-13T07:02:31.434 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.435 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.435 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:31.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:31.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543699+0000", 2024-09-13T07:02:31.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:31.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543713+0000", 2024-09-13T07:02:31.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4786e-05 2024-09-13T07:02:31.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543737+0000", 2024-09-13T07:02:31.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3626e-05 2024-09-13T07:02:31.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543752+0000", 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.489e-05 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543832+0000", 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0396999999999996e-05 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543994+0000", 2024-09-13T07:02:31.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016129100000000001 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545471+0000", 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014771960000000001 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545575+0000", 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103732 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.440 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1882 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'23, mlcod=28'21)", 2024-09-13T07:02:31.440 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:31.440 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.736398098999999, 2024-09-13T07:02:31.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001767172, 2024-09-13T07:02:31.440 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.440 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.440 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:31.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:31.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639332+0000", 2024-09-13T07:02:31.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:31.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.443 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.443 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639343+0000", 2024-09-13T07:02:31.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1436e-05 2024-09-13T07:02:31.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639361+0000", 2024-09-13T07:02:31.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7915e-05 2024-09-13T07:02:31.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639370+0000", 2024-09-13T07:02:31.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3189999999999995e-06 2024-09-13T07:02:31.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.445 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639425+0000", 2024-09-13T07:02:31.445 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4917999999999998e-05 2024-09-13T07:02:31.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639454+0000", 2024-09-13T07:02:31.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9862999999999999e-05 2024-09-13T07:02:31.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641059+0000", 2024-09-13T07:02:31.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016048950000000001 2024-09-13T07:02:31.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.447 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.447 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641100+0000", 2024-09-13T07:02:31.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0883999999999997e-05 2024-09-13T07:02:31.447 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.447 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.447 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.447 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.447 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3069 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'33, mlcod=31'31)", 2024-09-13T07:02:31.447 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:31.447 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.213258383000003, 2024-09-13T07:02:31.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018597010000000001, 2024-09-13T07:02:31.447 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162471+0000", 2024-09-13T07:02:31.449 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:31.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.450 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.450 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162520+0000", 2024-09-13T07:02:31.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8466e-05 2024-09-13T07:02:31.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.450 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.450 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162546+0000", 2024-09-13T07:02:31.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6089999999999999e-05 2024-09-13T07:02:31.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162562+0000", 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5962000000000001e-05 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162646+0000", 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3985000000000003e-05 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162691+0000", 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5785999999999998e-05 2024-09-13T07:02:31.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164265+0000", 2024-09-13T07:02:31.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015740890000000001 2024-09-13T07:02:31.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164333+0000", 2024-09-13T07:02:31.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7014999999999994e-05 2024-09-13T07:02:31.452 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.452 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.452 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3121 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'36, mlcod=31'34)", 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.099311995999997, 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019657780000000001, 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276417+0000", 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276497+0000", 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9729999999999997e-05 2024-09-13T07:02:31.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276537+0000", 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0102999999999999e-05 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276553+0000", 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6518999999999999e-05 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276662+0000", 2024-09-13T07:02:31.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109252 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276729+0000", 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7056000000000003e-05 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278270+0000", 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015408819999999999 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.456 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278385+0000", 2024-09-13T07:02:31.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114613 2024-09-13T07:02:31.457 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.457 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.457 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.457 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.457 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.457 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3390 3.0 e32/22 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:head v 32'45, mlcod=31'42)", 2024-09-13T07:02:31.457 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:31.457 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.880223649000001, 2024-09-13T07:02:31.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002246587, 2024-09-13T07:02:31.457 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.458 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.458 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.458 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:31.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.458 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:31.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.459 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.459 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495507+0000", 2024-09-13T07:02:31.459 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.459 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.459 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495530+0000", 2024-09-13T07:02:31.459 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2592999999999999e-05 2024-09-13T07:02:31.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.460 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495550+0000", 2024-09-13T07:02:31.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0152999999999998e-05 2024-09-13T07:02:31.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.460 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495562+0000", 2024-09-13T07:02:31.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2336e-05 2024-09-13T07:02:31.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495662+0000", 2024-09-13T07:02:31.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100317 2024-09-13T07:02:31.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495717+0000", 2024-09-13T07:02:31.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4304e-05 2024-09-13T07:02:31.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.462 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.462 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.462 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497707+0000", 2024-09-13T07:02:31.462 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019906849999999999 2024-09-13T07:02:31.462 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.462 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.462 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.462 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497754+0000", 2024-09-13T07:02:31.462 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6715999999999998e-05 2024-09-13T07:02:31.462 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.462 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.463 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.463 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.463 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.463 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3399 3.4 e32/22 3:22863a52:::rbd_data.11945d91e99c.0000000000000013:head v 32'65, mlcod=31'62)", 2024-09-13T07:02:31.463 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:31.463 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.876286976999999, 2024-09-13T07:02:31.463 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028830779999999999, 2024-09-13T07:02:31.463 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.463 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.463 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.463 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.463 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499442+0000", 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499459+0000", 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7042e-05 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499493+0000", 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3788000000000002e-05 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499512+0000", 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9055999999999998e-05 2024-09-13T07:02:31.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499608+0000", 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6168000000000005e-05 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499658+0000", 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9469999999999999e-05 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502277+0000", 2024-09-13T07:02:31.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026188639999999998 2024-09-13T07:02:31.467 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.467 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.467 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.467 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502327+0000", 2024-09-13T07:02:31.467 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0578999999999998e-05 2024-09-13T07:02:31.467 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.467 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.467 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.467 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.467 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.467 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3395 3.0 e32/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 32'47, mlcod=31'43)", 2024-09-13T07:02:31.468 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:31.468 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.876188892000002, 2024-09-13T07:02:31.468 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027505870000000001, 2024-09-13T07:02:31.468 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.468 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.468 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.468 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.468 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.468 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:31.468 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.468 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.468 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.468 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499560+0000", 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8416000000000001e-05 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499576+0000", 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5602000000000001e-05 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499587+0000", 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1112e-05 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499654+0000", 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7164999999999998e-05 2024-09-13T07:02:31.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.471 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.471 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.471 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499674+0000", 2024-09-13T07:02:31.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9638000000000001e-05 2024-09-13T07:02:31.471 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.471 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.471 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.471 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502210+0000", 2024-09-13T07:02:31.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025367100000000002 2024-09-13T07:02:31.471 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.472 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.472 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.472 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502293+0000", 2024-09-13T07:02:31.472 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2607000000000004e-05 2024-09-13T07:02:31.472 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.472 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.472 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.472 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.472 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.472 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:4534 3.0 e32/22 3:00392e9e:::rbd_data.11944cdc0656.000000000000000c:head v 32'51, mlcod=32'49)", 2024-09-13T07:02:31.473 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:31.473 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.464487706, 2024-09-13T07:02:31.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017597330000000001, 2024-09-13T07:02:31.473 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.473 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.473 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:31.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.473 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.474 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.474 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:31.474 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.474 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.474 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.474 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911242+0000", 2024-09-13T07:02:31.474 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:31.474 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.474 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.475 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911306+0000", 2024-09-13T07:02:31.475 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4480999999999995e-05 2024-09-13T07:02:31.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.475 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.475 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.475 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911338+0000", 2024-09-13T07:02:31.475 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1733000000000003e-05 2024-09-13T07:02:31.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.475 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.475 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911351+0000", 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3241000000000001e-05 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911455+0000", 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103723 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911495+0000", 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0683000000000003e-05 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.477 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.477 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.912933+0000", 2024-09-13T07:02:31.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001437219 2024-09-13T07:02:31.477 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.477 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.477 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.477 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.913003+0000", 2024-09-13T07:02:31.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0629000000000004e-05 2024-09-13T07:02:31.478 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.478 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.478 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.478 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11205 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'50, mlcod=38'47)", 2024-09-13T07:02:31.478 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:31.478 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.751147220999997, 2024-09-13T07:02:31.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031049490000000001, 2024-09-13T07:02:31.479 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.479 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.479 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:31.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:31.480 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.480 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.480 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.480 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.480 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624583+0000", 2024-09-13T07:02:31.480 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:31.480 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.480 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.480 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.480 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625339+0000", 2024-09-13T07:02:31.480 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00075563200000000001 2024-09-13T07:02:31.480 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.480 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625358+0000", 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9916999999999999e-05 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625366+0000", 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8050000000000007e-06 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625415+0000", 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8231e-05 2024-09-13T07:02:31.481 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625447+0000", 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2703999999999997e-05 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627652+0000", 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022050659999999999 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627689+0000", 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6612000000000001e-05 2024-09-13T07:02:31.482 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.483 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.483 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.483 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.483 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.483 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11206 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'52, mlcod=38'47)", 2024-09-13T07:02:31.483 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:31.483 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 52.749876555, 2024-09-13T07:02:31.483 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022238470000000002, 2024-09-13T07:02:31.483 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.483 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:31.483 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.484 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.484 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.484 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:31.484 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.484 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.484 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.484 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.484 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:31.484 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.484 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.484 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.484 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.484 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625854+0000", 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625863+0000", 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5380000000000008e-06 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625883+0000", 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9247000000000001e-05 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.485 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625893+0000", 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0176e-05 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626411+0000", 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051847600000000003 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626431+0000", 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0199000000000001e-05 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.486 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:31.487 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628028+0000", 2024-09-13T07:02:31.487 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015968110000000001 2024-09-13T07:02:31.487 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.487 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.487 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.487 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628078+0000", 2024-09-13T07:02:31.487 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0170000000000002e-05 2024-09-13T07:02:31.487 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.487 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.487 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.487 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.487 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.488 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:31.490 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:31.490 INFO:teuthology.orchestra.run.smithi118.stdout:2525 write 0x80713e8 thru 0x8073de8 (0x2a01 bytes) 2024-09-13T07:02:31.491 INFO:teuthology.orchestra.run.smithi118.stdout:2526 write 0x155925 thru 0x160ae1 (0xb1bd bytes) 2024-09-13T07:02:31.491 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:31.486+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:31.591 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:31.588+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:31.609 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:31.609 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:31.609 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:31.609 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:31.620 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:31.671 INFO:teuthology.orchestra.run.smithi118.stdout:2527 punch from 0x7e937b6 to 0x7ea2525, (0xed6f bytes) 2024-09-13T07:02:31.677 INFO:teuthology.orchestra.run.smithi118.stdout:2528 trunc from 0x8073de9 to 0x575ee3a 2024-09-13T07:02:31.691 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:31.688+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:31.705 INFO:teuthology.orchestra.run.smithi118.stdout:2529 trunc from 0x575ee3a to 0xb4d10e5 2024-09-13T07:02:31.717 INFO:teuthology.orchestra.run.smithi118.stdout:2531 read 0x7dfe26e thru 0x7e092b1 (0xb044 bytes) 2024-09-13T07:02:31.717 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:31.717 INFO:teuthology.orchestra.run.smithi118.stdout:2534 read 0x63ceafd thru 0x63d4058 (0x555c bytes) 2024-09-13T07:02:31.717 INFO:teuthology.orchestra.run.smithi118.stdout:2535 punch from 0xa27cdcf to 0xa285ddb, (0x900c bytes) 2024-09-13T07:02:31.717 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:31.718 INFO:teuthology.orchestra.run.smithi118.stdout:truncating to largest ever: 0xedc4a46 2024-09-13T07:02:31.723 INFO:teuthology.orchestra.run.smithi118.stdout:2537 trunc from 0xb4d10e5 to 0xedc4a46 2024-09-13T07:02:31.733 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:31.733 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:31.733 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:31.733 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:31.733 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:31.734 INFO:teuthology.orchestra.run.smithi118.stdout:2539 write 0xb22aab4 thru 0xb22c31b (0x1868 bytes) 2024-09-13T07:02:31.743 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:31.754 INFO:teuthology.orchestra.run.smithi118.stdout:2540 read 0xbaaa0b3 thru 0xbab6735 (0xc683 bytes) 2024-09-13T07:02:31.754 INFO:teuthology.orchestra.run.smithi118.stdout:2541 write 0xd5b040c thru 0xd5b286b (0x2460 bytes) 2024-09-13T07:02:31.769 INFO:teuthology.orchestra.run.smithi118.stdout:2542 read 0xccc05a thru 0xcdadaa (0xed51 bytes) 2024-09-13T07:02:31.770 INFO:teuthology.orchestra.run.smithi118.stdout:2543 writesame 0x6e91b75 thru 0x6ea07d8 data_size 0x52(0xec64 bytes) 2024-09-13T07:02:31.785 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:31.787 INFO:teuthology.orchestra.run.smithi118.stdout:2545 trunc from 0xedc4a46 to 0xdc56fa5 2024-09-13T07:02:31.792 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:31.790+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:31.795 INFO:teuthology.orchestra.run.smithi118.stdout:2546 punch from 0x930018f to 0x930c3a1, (0xc212 bytes) 2024-09-13T07:02:31.795 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:31.796 INFO:teuthology.orchestra.run.smithi118.stdout:2549 writesame 0x161f6d thru 0x162e8f data_size 0x19(0xf23 bytes) 2024-09-13T07:02:31.804 INFO:teuthology.orchestra.run.smithi118.stdout:2550 read 0x3b69fcd thru 0x3b71fcf (0x8003 bytes) 2024-09-13T07:02:31.823 INFO:teuthology.orchestra.run.smithi118.stdout:2551 trunc from 0xdc56fa5 to 0x2e5a286 2024-09-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7348904730000001, 2024-09-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7348699960000005, 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87036 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 6963200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7281859300000004, 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042468831999999998, 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87036 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127964+0000", 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7799999999999999e-06 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127967+0000", 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9730000000000002e-06 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127971+0000", 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4349999999999999e-06 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170202+0000", 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042230940000000002 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170293+0000", 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0858000000000003e-05 2024-09-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170431+0000", 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013821699999999999 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7281733930000005, 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7281613240000002, 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7281488809999992, 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7281312829999997, 2024-09-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:02:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:02:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7281205800000006, 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:02:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:02:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.728076647, 2024-09-13T07:02:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:02:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:02:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7280568610000007, 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:02:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:02:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:02:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:02:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:02:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:02:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7280411109999996, 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:02:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7280298589999994, 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:02:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7280192599999999, 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:02:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:02:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:02:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:02:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:02:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7280097669999996, 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:02:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:02:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:02:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:02:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:02:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:02:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:02:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:02:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:02:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:02:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:02:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7280002230000004, 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:02:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:02:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:02:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:02:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:02:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:02:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:02:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:02:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:02:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:02:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:02:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:02:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7279676199999994, 2024-09-13T07:02:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:02:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:02:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7279545970000001, 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:02:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:02:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:02:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:02:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:02:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7246036119999992, 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:02:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7245833350000002, 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:02:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7245631340000003, 2024-09-13T07:02:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:02:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:31.947 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:31.949 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:31.949 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:31.948+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:31.990 INFO:teuthology.orchestra.run.smithi118.stdout:2552 read 0x2df684c thru 0x2e01ef1 (0xb6a6 bytes) 2024-09-13T07:02:32.004 INFO:teuthology.orchestra.run.smithi118.stdout:2553 trunc from 0x2e5a286 to 0x78844e1 2024-09-13T07:02:32.017 INFO:teuthology.orchestra.run.smithi118.stdout:2554 punch from 0x68c34ac to 0x68ccc5b, (0x97af bytes) 2024-09-13T07:02:32.031 INFO:teuthology.orchestra.run.smithi118.stdout:2555 write 0x81e7735 thru 0x81f6a19 (0xf2e5 bytes) 2024-09-13T07:02:32.049 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:32.048+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:32.054 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:32.054 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:32.054 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:32.054 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:32.058 INFO:teuthology.orchestra.run.smithi118.stdout:2556 trunc from 0x81f6a1a to 0xc8a49a3 2024-09-13T07:02:32.065 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:32.068 INFO:teuthology.orchestra.run.smithi118.stdout:2557 punch from 0x5458a59 to 0x545a068, (0x160f bytes) 2024-09-13T07:02:32.068 INFO:teuthology.orchestra.run.smithi118.stdout:2558 write 0x76b1f39 thru 0x76b29d4 (0xa9c bytes) 2024-09-13T07:02:32.080 INFO:teuthology.orchestra.run.smithi118.stdout:2559 write 0x73f3826 thru 0x73fb721 (0x7efc bytes) 2024-09-13T07:02:32.096 INFO:teuthology.orchestra.run.smithi118.stdout:2560 writesame 0xa7abc3d thru 0xa7ac2f6 data_size 0x15(0x6ba bytes) 2024-09-13T07:02:32.101 INFO:teuthology.orchestra.run.smithi118.stdout:2561 write 0xb02e839 thru 0xb036699 (0x7e61 bytes) 2024-09-13T07:02:32.117 INFO:teuthology.orchestra.run.smithi118.stdout:2562 write 0xae5f076 thru 0xae6e424 (0xf3af bytes) 2024-09-13T07:02:32.136 INFO:teuthology.orchestra.run.smithi118.stdout:2563 write 0x400be13 thru 0x400fc55 (0x3e43 bytes) 2024-09-13T07:02:32.150 INFO:teuthology.orchestra.run.smithi118.stdout:2564 read 0xbf3bf88 thru 0xbf427e0 (0x6859 bytes) 2024-09-13T07:02:32.150 INFO:teuthology.orchestra.run.smithi118.stdout:2565 write 0x882a8d1 thru 0x882d5c4 (0x2cf4 bytes) 2024-09-13T07:02:32.150 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:32.148+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:32.164 INFO:teuthology.orchestra.run.smithi118.stdout:2566 write 0x7c23ee0 thru 0x7c25994 (0x1ab5 bytes) 2024-09-13T07:02:32.177 INFO:teuthology.orchestra.run.smithi118.stdout:2567 writesame 0xb4a684d thru 0xb4aa6af data_size 0x1(0x3e63 bytes) 2024-09-13T07:02:32.189 INFO:teuthology.orchestra.run.smithi118.stdout:2569 punch from 0x22235a2 to 0x2228c56, (0x56b4 bytes) 2024-09-13T07:02:32.189 INFO:teuthology.orchestra.run.smithi118.stdout:2571 write 0x75e64ba thru 0x75f13f0 (0xaf37 bytes) 2024-09-13T07:02:32.189 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:32.190 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:32.190 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:32.190 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:32.190 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:32.196 INFO:teuthology.orchestra.run.smithi118.stdout:2572 read 0x45d550b thru 0x45e0be3 (0xb6d9 bytes) 2024-09-13T07:02:32.197 INFO:teuthology.orchestra.run.smithi118.stdout:2573 write 0x99ff3e1 thru 0x9a02c7a (0x389a bytes) 2024-09-13T07:02:32.201 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:32.218 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:32.218 INFO:teuthology.orchestra.run.smithi118.stdout:2575 write 0xb97ebe thru 0xba2db9 (0xaefc bytes) 2024-09-13T07:02:32.236 INFO:teuthology.orchestra.run.smithi118.stdout:2576 write 0x19e3a65 thru 0x19ee4fa (0xaa96 bytes) 2024-09-13T07:02:32.251 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:32.248+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:32.257 INFO:teuthology.orchestra.run.smithi118.stdout:2578 write 0x84807c6 thru 0x848c886 (0xc0c1 bytes) 2024-09-13T07:02:32.273 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:32.283 INFO:teuthology.orchestra.run.smithi118.stdout:2580 write 0xca72c23 thru 0xca76e3e (0x421c bytes) 2024-09-13T07:02:32.300 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:32.302 INFO:teuthology.orchestra.run.smithi118.stdout:2582 trunc from 0xca76e3f to 0xd65cd97 2024-09-13T07:02:32.318 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:32.318 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:32.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:32.318 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:32.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.318 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1524 3.14 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:32.318 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:32.318 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.271798321999995, 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061190880000000003, 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1524 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:32.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044767+0000", 2024-09-13T07:02:32.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:32.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052670+0000", 2024-09-13T07:02:32.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079022450000000004 2024-09-13T07:02:32.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052682+0000", 2024-09-13T07:02:32.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2336e-05 2024-09-13T07:02:32.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052696+0000", 2024-09-13T07:02:32.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-13T07:02:32.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052755+0000", 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9298e-05 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052947+0000", 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019153900000000001 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.053694+0000", 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074663099999999999 2024-09-13T07:02:32.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.102148+0000", 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048454329999999997 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105816+0000", 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036675950000000001 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105921+0000", 2024-09-13T07:02:32.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105874 2024-09-13T07:02:32.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105959+0000", 2024-09-13T07:02:32.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7991000000000003e-05 2024-09-13T07:02:32.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.324 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.324 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:48369 3.f e58/22 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head v 58'78, mlcod=57'75)", 2024-09-13T07:02:32.324 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:32.324 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.076100457000003, 2024-09-13T07:02:32.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.057439072000000001, 2024-09-13T07:02:32.325 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.325 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:32.330 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:32.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:32.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240464+0000", 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245716+0000", 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005252329 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245745+0000", 2024-09-13T07:02:32.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8844e-05 2024-09-13T07:02:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245758+0000", 2024-09-13T07:02:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2242000000000001e-05 2024-09-13T07:02:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245849+0000", 2024-09-13T07:02:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1187999999999995e-05 2024-09-13T07:02:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245898+0000", 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8881000000000002e-05 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297837+0000", 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051938947999999999 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297906+0000", 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8955999999999995e-05 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.334 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94699 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.334 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674309+0000", 2024-09-13T07:02:32.334 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6422580990000002, 2024-09-13T07:02:32.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051428639999999998, 2024-09-13T07:02:32.334 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.334 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.334 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.334 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.334 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.334 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94699 2024-09-13T07:02:32.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674309+0000", 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674309+0000", 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674308+0000", 2024-09-13T07:02:32.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674311+0000", 2024-09-13T07:02:32.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4200000000000001e-06 2024-09-13T07:02:32.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674317+0000", 2024-09-13T07:02:32.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9429999999999997e-06 2024-09-13T07:02:32.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674321+0000", 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6460000000000001e-06 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725637+0000", 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051315686999999999 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725687+0000", 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0096000000000003e-05 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725737+0000", 2024-09-13T07:02:32.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0232999999999998e-05 2024-09-13T07:02:32.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.338 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.338 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94700 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.338 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674326+0000", 2024-09-13T07:02:32.338 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6422411690000001, 2024-09-13T07:02:32.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051537695000000001, 2024-09-13T07:02:32.338 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94700 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674326+0000", 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674326+0000", 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674325+0000", 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674328+0000", 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4399999999999999e-06 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674333+0000", 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4569999999999996e-06 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674338+0000", 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6659999999999999e-06 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725762+0000", 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051423979000000002 2024-09-13T07:02:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725816+0000", 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4509999999999998e-05 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725863+0000", 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7021999999999998e-05 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94701 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2056192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674342+0000", 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6422249390000001, 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051643316000000002, 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94701 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674342+0000", 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674342+0000", 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674342+0000", 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674344+0000", 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674349+0000", 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1880000000000001e-06 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674354+0000", 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3800000000000004e-06 2024-09-13T07:02:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725889+0000", 2024-09-13T07:02:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051535091999999998 2024-09-13T07:02:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725937+0000", 2024-09-13T07:02:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8680000000000001e-05 2024-09-13T07:02:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725985+0000", 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7871000000000003e-05 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94702 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2121728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674358+0000", 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.642209051, 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051754434000000002, 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94702 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674358+0000", 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674358+0000", 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674358+0000", 2024-09-13T07:02:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674360+0000", 2024-09-13T07:02:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4099999999999998e-06 2024-09-13T07:02:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674365+0000", 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0860000000000001e-06 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674370+0000", 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726009+0000", 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051639184999999997 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726061+0000", 2024-09-13T07:02:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1613000000000002e-05 2024-09-13T07:02:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726112+0000", 2024-09-13T07:02:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1684999999999997e-05 2024-09-13T07:02:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94703 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2236416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674374+0000", 2024-09-13T07:02:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6421929779999997, 2024-09-13T07:02:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051856764999999999, 2024-09-13T07:02:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94703 2024-09-13T07:02:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674374+0000", 2024-09-13T07:02:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674374+0000", 2024-09-13T07:02:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674373+0000", 2024-09-13T07:02:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674385+0000", 2024-09-13T07:02:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1894e-05 2024-09-13T07:02:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674391+0000", 2024-09-13T07:02:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2280000000000003e-06 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674397+0000", 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.164e-06 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726136+0000", 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051739072999999997 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726181+0000", 2024-09-13T07:02:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4914999999999999e-05 2024-09-13T07:02:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726231+0000", 2024-09-13T07:02:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0120000000000001e-05 2024-09-13T07:02:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94704 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2351104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674401+0000", 2024-09-13T07:02:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6421655790000003, 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051952104999999998, 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94704 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674401+0000", 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674401+0000", 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674401+0000", 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674403+0000", 2024-09-13T07:02:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.655e-06 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674439+0000", 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5126999999999998e-05 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674444+0000", 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3970000000000002e-06 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726254+0000", 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051810387999999999 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726305+0000", 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1057000000000003e-05 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726353+0000", 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8013999999999998e-05 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94705 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2465792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674452+0000", 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6421152479999996, 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.052023257000000003, 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94705 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674452+0000", 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674452+0000", 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674451+0000", 2024-09-13T07:02:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674455+0000", 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.506e-06 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674461+0000", 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.455e-06 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674466+0000", 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7999999999999998e-06 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726382+0000", 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051916074999999999 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726428+0000", 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5914000000000001e-05 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726475+0000", 2024-09-13T07:02:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6993999999999998e-05 2024-09-13T07:02:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94706 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674678+0000", 2024-09-13T07:02:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6418891000000002, 2024-09-13T07:02:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051920368000000001, 2024-09-13T07:02:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94706 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674678+0000", 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674678+0000", 2024-09-13T07:02:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674470+0000", 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9997926 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674682+0000", 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000211695 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674781+0000", 2024-09-13T07:02:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9296000000000006e-05 2024-09-13T07:02:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674981+0000", 2024-09-13T07:02:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019990000000000001 2024-09-13T07:02:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726497+0000", 2024-09-13T07:02:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051516144 2024-09-13T07:02:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726549+0000", 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1721000000000002e-05 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726598+0000", 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9039999999999998e-05 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94707 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2646016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674991+0000", 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6415760989999999, 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051730683, 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94707 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674991+0000", 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674991+0000", 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674990+0000", 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674999+0000", 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675007+0000", 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0870000000000007e-06 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675013+0000", 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726625+0000", 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051612492000000003 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726674+0000", 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.884e-05 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726721+0000", 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7348999999999997e-05 2024-09-13T07:02:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94708 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2760704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675018+0000", 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.641548985, 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051826217000000001, 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94708 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675018+0000", 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675018+0000", 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675018+0000", 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675021+0000", 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1269999999999999e-06 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675026+0000", 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7610000000000004e-06 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675031+0000", 2024-09-13T07:02:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6160000000000004e-06 2024-09-13T07:02:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726747+0000", 2024-09-13T07:02:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051716170999999998 2024-09-13T07:02:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726796+0000", 2024-09-13T07:02:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8718999999999999e-05 2024-09-13T07:02:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726844+0000", 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8214000000000003e-05 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94709 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675035+0000", 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6415314470000002, 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051926446000000001, 2024-09-13T07:02:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94709 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675035+0000", 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675035+0000", 2024-09-13T07:02:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675035+0000", 2024-09-13T07:02:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675038+0000", 2024-09-13T07:02:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6910000000000002e-06 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675044+0000", 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3879999999999997e-06 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675049+0000", 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726868+0000", 2024-09-13T07:02:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051818636000000001 2024-09-13T07:02:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726917+0000", 2024-09-13T07:02:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9586999999999998e-05 2024-09-13T07:02:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726962+0000", 2024-09-13T07:02:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4807e-05 2024-09-13T07:02:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94710 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675054+0000", 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6415130619999996, 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.052041476000000003, 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94710 2024-09-13T07:02:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675054+0000", 2024-09-13T07:02:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675054+0000", 2024-09-13T07:02:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675053+0000", 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675056+0000", 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8619999999999999e-06 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675062+0000", 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5709999999999997e-06 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675370+0000", 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030795900000000001 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726989+0000", 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051619842999999999 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727038+0000", 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8792000000000003e-05 2024-09-13T07:02:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727095+0000", 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7095000000000001e-05 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94711 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3104768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675377+0000", 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6411902869999997, 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051839969, 2024-09-13T07:02:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94711 2024-09-13T07:02:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675377+0000", 2024-09-13T07:02:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675377+0000", 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675376+0000", 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675380+0000", 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7749999999999999e-06 2024-09-13T07:02:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675385+0000", 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6559999999999997e-06 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675388+0000", 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.022e-06 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727124+0000", 2024-09-13T07:02:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051735712000000003 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727174+0000", 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9716999999999998e-05 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727217+0000", 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2710999999999998e-05 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94712 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3170304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675391+0000", 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6411759339999996, 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051946194000000001, 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94712 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675391+0000", 2024-09-13T07:02:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675391+0000", 2024-09-13T07:02:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675391+0000", 2024-09-13T07:02:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675392+0000", 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5340000000000001e-06 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675395+0000", 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8140000000000001e-06 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675398+0000", 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5809999999999999e-06 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727240+0000", 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051842436999999998 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727288+0000", 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7901000000000001e-05 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727337+0000", 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9177e-05 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94714 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.676292+0000", 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6402747479999995, 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051320047000000001, 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94714 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676292+0000", 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676292+0000", 2024-09-13T07:02:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676291+0000", 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676295+0000", 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8689999999999997e-06 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676302+0000", 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4230000000000002e-06 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676307+0000", 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9320000000000004e-06 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727488+0000", 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051180941000000001 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727552+0000", 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4376999999999997e-05 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727612+0000", 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0198999999999997e-05 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94715 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.676311+0000", 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6402556910000001, 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051421327000000003, 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94715 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676311+0000", 2024-09-13T07:02:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676311+0000", 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676311+0000", 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676320+0000", 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3989999999999993e-06 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676326+0000", 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9939999999999997e-06 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676331+0000", 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.028e-06 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727641+0000", 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051309886999999998 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727688+0000", 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6755999999999998e-05 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727733+0000", 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4668e-05 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94716 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3629056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.676336+0000", 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6402311919999999, 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051505305000000001, 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94716 2024-09-13T07:02:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676336+0000", 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676336+0000", 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676335+0000", 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676338+0000", 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3329999999999999e-06 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676343+0000", 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3569999999999997e-06 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676347+0000", 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2769999999999999e-06 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727752+0000", 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051404367999999999 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727796+0000", 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4057999999999997e-05 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727841+0000", 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5256000000000002e-05 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94717 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3694592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.676351+0000", 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.640215489, 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051655690999999997, 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94717 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676351+0000", 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676351+0000", 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676351+0000", 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676358+0000", 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6549999999999998e-06 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676363+0000", 2024-09-13T07:02:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-13T07:02:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676368+0000", 2024-09-13T07:02:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:02:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727869+0000", 2024-09-13T07:02:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051501208 2024-09-13T07:02:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727943+0000", 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4120000000000002e-05 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.728007+0000", 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4096000000000004e-05 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:32.413 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:32.415 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:32.416 INFO:teuthology.orchestra.run.smithi118.stdout:2583 writesame 0xd8b28ca thru 0xd8b6478 data_size 0xb(0x3baf bytes) 2024-09-13T07:02:32.417 INFO:teuthology.orchestra.run.smithi118.stdout:2585 read 0xa889645 thru 0xa898cc3 (0xf67f bytes) 2024-09-13T07:02:32.417 INFO:teuthology.orchestra.run.smithi118.stdout:2586 read 0x66cdd7e thru 0x66ce11f (0x3a2 bytes) 2024-09-13T07:02:32.417 INFO:teuthology.orchestra.run.smithi118.stdout:2587 trunc from 0xd8b6479 to 0x7c9db3f 2024-09-13T07:02:32.417 INFO:teuthology.orchestra.run.smithi118.stdout:2588 write 0x943053 thru 0x94bc0f (0x8bbd bytes) 2024-09-13T07:02:32.417 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:32.417 INFO:teuthology.orchestra.run.smithi118.stdout:2590 read 0x22c4a3c thru 0x22c9675 (0x4c3a bytes) 2024-09-13T07:02:32.417 INFO:teuthology.orchestra.run.smithi118.stdout:2591 punch from 0x13c07b7 to 0x13c8d9e, (0x85e7 bytes) 2024-09-13T07:02:32.417 INFO:teuthology.orchestra.run.smithi118.stdout:2592 read 0x20116b0 thru 0x201d85f (0xc1b0 bytes) 2024-09-13T07:02:32.417 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:32.417 INFO:teuthology.orchestra.run.smithi118.stdout:2594 read 0x68c121 thru 0x694df1 (0x8cd1 bytes) 2024-09-13T07:02:32.417 INFO:teuthology.orchestra.run.smithi118.stdout:2595 clone 29 order 18 su 16384 sc 13 2024-09-13T07:02:32.418 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:32.412+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:32.489 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:02:32.490 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:02:32.515 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:32.514+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:32.538 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:32.538 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:32.538 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:32.538 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:32.549 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:32.561 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone28 from 0x7c9db3f (overlap 0x2e5a286) to 0x0 2024-09-13T07:02:32.616 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:32.614+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:32.633 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:32.659 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:32.676 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:32.676 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:32.676 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:32.676 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:32.677 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:32.688 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:32.689 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:32.711 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:32.717 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:32.714+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:32.733 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:32.757 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:32.781 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:32.807 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:32.809 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:32.809 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:32.809 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:32.809 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:32.809 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:32.809 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:32.809 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:32.810 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:32.814 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:32.814 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:32.818 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:32.816+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:32.827 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:32.919 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:32.916+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:32.993 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:32.994 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:33.004 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:33.020 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:33.016+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:33.121 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:33.120+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:33.125 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:33.125 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:33.137 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:33.222 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:33.218+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:33.256 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:33.256 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:33.266 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:33.323 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:33.320+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:33.384 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:33.384 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:33.395 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:33.420 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:02:33.423 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:33.420+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:33.524 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:33.522+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:33.526 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:33.526 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:33.538 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:33.626 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:33.622+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:33.655 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:33.656 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:33.670 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:33.727 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:33.724+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:33.785 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:33.786 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:33.796 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:33.827 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:33.826+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:33.902 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:33.902 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:33.912 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:33.928 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:33.924+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:33.962 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-13T07:02:33.965 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:33.965 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:33.965 INFO:teuthology.orchestra.run.smithi118.stdout:2599 read 0x606bb33 thru 0x607924a (0xd718 bytes) 2024-09-13T07:02:33.992 INFO:teuthology.orchestra.run.smithi118.stdout:2600 write 0xa33493d thru 0xa343ff4 (0xf6b8 bytes) 2024-09-13T07:02:34.004 INFO:teuthology.orchestra.run.smithi118.stdout:2603 write 0x59a5f57 thru 0x59b434c (0xe3f6 bytes) 2024-09-13T07:02:34.017 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:34.017 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:34.019 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:34.019 INFO:teuthology.orchestra.run.smithi118.stdout:2606 write 0x32adb2b thru 0x32af7fc (0x1cd2 bytes) 2024-09-13T07:02:34.024 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:34.024 INFO:teuthology.orchestra.run.smithi118.stdout:2608 read 0x75c440d thru 0x75cf482 (0xb076 bytes) 2024-09-13T07:02:34.027 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:34.028 INFO:teuthology.orchestra.run.smithi118.stdout:2609 punch from 0x76477a3 to 0x764af00, (0x375d bytes) 2024-09-13T07:02:34.028 INFO:teuthology.orchestra.run.smithi118.stdout:2610 read 0x872d28e thru 0x872dd61 (0xad4 bytes) 2024-09-13T07:02:34.029 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:34.026+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:34.043 INFO:teuthology.orchestra.run.smithi118.stdout:2611 writesame 0xd2f1b9b thru 0xd2fabfd data_size 0x25(0x9063 bytes) 2024-09-13T07:02:34.044 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:34.058 INFO:teuthology.orchestra.run.smithi118.stdout:2615 writesame 0xe97c714 thru 0xe98280a data_size 0x1(0x60f7 bytes) 2024-09-13T07:02:34.058 INFO:teuthology.orchestra.run.smithi118.stdout:2616 writesame 0xa5df84e thru 0xa5e98f9 data_size 0xe(0xa0ac bytes) 2024-09-13T07:02:34.092 INFO:teuthology.orchestra.run.smithi118.stdout:2618 trunc from 0xe98280b to 0x1d385f4 2024-09-13T07:02:34.123 INFO:teuthology.orchestra.run.smithi118.stdout:2621 read 0x98ece3 thru 0x99a9c5 (0xbce3 bytes) 2024-09-13T07:02:34.130 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:34.128+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:34.140 INFO:teuthology.orchestra.run.smithi118.stdout:2622 write 0x52d00bb thru 0x52d6fe9 (0x6f2f bytes) 2024-09-13T07:02:34.146 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:34.147 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:34.168 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:34.176 INFO:teuthology.orchestra.run.smithi118.stdout:2623 write 0xd01f852 thru 0xd020920 (0x10cf bytes) 2024-09-13T07:02:34.182 INFO:teuthology.orchestra.run.smithi118.stdout:2624 write 0xb7ca2f0 thru 0xb7d7725 (0xd436 bytes) 2024-09-13T07:02:34.193 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:34.193 INFO:teuthology.orchestra.run.smithi118.stdout:2626 punch from 0x9cc5511 to 0x9ccfa1f, (0xa50e bytes) 2024-09-13T07:02:34.193 INFO:teuthology.orchestra.run.smithi118.stdout:2628 write 0xa371bd7 thru 0xa375a58 (0x3e82 bytes) 2024-09-13T07:02:34.199 INFO:teuthology.orchestra.run.smithi118.stdout:2630 write 0xa4ff354 thru 0xa5048ec (0x5599 bytes) 2024-09-13T07:02:34.205 INFO:teuthology.orchestra.run.smithi118.stdout:2631 read 0x5637da0 thru 0x563c07d (0x42de bytes) 2024-09-13T07:02:34.215 INFO:teuthology.orchestra.run.smithi118.stdout:2632 writesame 0xd61e97a thru 0xd62c2a4 data_size 0x1(0xd92b bytes) 2024-09-13T07:02:34.225 INFO:teuthology.orchestra.run.smithi118.stdout:2633 trunc from 0xd62c2a5 to 0x9556542 2024-09-13T07:02:34.231 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:34.230+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:34.245 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:34.245 INFO:teuthology.orchestra.run.smithi118.stdout:2639 read 0x7a336a2 thru 0x7a381fc (0x4b5b bytes) 2024-09-13T07:02:34.257 INFO:teuthology.orchestra.run.smithi118.stdout:2640 trunc from 0x9556542 to 0x181eccd 2024-09-13T07:02:34.291 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:34.291 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:34.303 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:34.332 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:34.330+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:34.359 INFO:teuthology.orchestra.run.smithi118.stdout:2641 write 0x9a4676d thru 0x9a51ca9 (0xb53d bytes) 2024-09-13T07:02:34.371 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:34.372 INFO:teuthology.orchestra.run.smithi118.stdout:2643 punch from 0x64e8920 to 0x64ee039, (0x5719 bytes) 2024-09-13T07:02:34.372 INFO:teuthology.orchestra.run.smithi118.stdout:2644 write 0x494c2 thru 0x4b188 (0x1cc7 bytes) 2024-09-13T07:02:34.399 INFO:teuthology.orchestra.run.smithi118.stdout:2645 punch from 0x81c6001 to 0x81cac66, (0x4c65 bytes) 2024-09-13T07:02:34.400 INFO:teuthology.orchestra.run.smithi118.stdout:2646 read 0x3c72f5e thru 0x3c7306e (0x111 bytes) 2024-09-13T07:02:34.400 INFO:teuthology.orchestra.run.smithi118.stdout:2648 read 0x596c1fe thru 0x596e3c9 (0x21cc bytes) 2024-09-13T07:02:34.400 INFO:teuthology.orchestra.run.smithi118.stdout:2649 read 0x5e83cde thru 0x5e88d02 (0x5025 bytes) 2024-09-13T07:02:34.414 INFO:teuthology.orchestra.run.smithi118.stdout:2650 trunc from 0x9a51caa to 0x27421a4 2024-09-13T07:02:34.433 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:34.430+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:34.435 INFO:teuthology.orchestra.run.smithi118.stdout:2651 punch from 0x103d096 to 0x103d28f, (0x1f9 bytes) 2024-09-13T07:02:34.451 INFO:teuthology.orchestra.run.smithi118.stdout:2652 trunc from 0x27421a4 to 0xbfd8405 2024-09-13T07:02:34.460 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:34.460 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:34.464 INFO:teuthology.orchestra.run.smithi118.stdout:2653 punch from 0x477856 to 0x477e80, (0x62a bytes) 2024-09-13T07:02:34.464 INFO:teuthology.orchestra.run.smithi118.stdout:2654 write 0xb0876cc thru 0xb08c5d3 (0x4f08 bytes) 2024-09-13T07:02:34.470 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:34.477 INFO:teuthology.orchestra.run.smithi118.stdout:2655 write 0x6bd49c6 thru 0x6bd6593 (0x1bce bytes) 2024-09-13T07:02:34.499 INFO:teuthology.orchestra.run.smithi118.stdout:2656 trunc from 0xbfd8405 to 0x3ae5725 2024-09-13T07:02:34.534 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:34.532+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:34.547 INFO:teuthology.orchestra.run.smithi118.stdout:2657 writesame 0xd304204 thru 0xd3128f4 data_size 0x3(0xe6f1 bytes) 2024-09-13T07:02:34.547 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:34.547 INFO:teuthology.orchestra.run.smithi118.stdout:2659 write 0x3670a48 thru 0x3670ec8 (0x481 bytes) 2024-09-13T07:02:34.573 INFO:teuthology.orchestra.run.smithi118.stdout:2660 trunc from 0xd3128f5 to 0x4c9e35 2024-09-13T07:02:34.579 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:34.580 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:34.590 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:34.635 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:34.632+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:34.736 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:34.734+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:34.743 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:34.743 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:34.754 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:34.760 INFO:teuthology.orchestra.run.smithi118.stdout:2661 write 0x9fafc30 thru 0x9fb3073 (0x3444 bytes) 2024-09-13T07:02:34.768 INFO:teuthology.orchestra.run.smithi118.stdout:2662 read 0xb85531 thru 0xb95337 (0xfe07 bytes) 2024-09-13T07:02:34.768 INFO:teuthology.orchestra.run.smithi118.stdout:2663 write 0x7e8bc5a thru 0x7e9ab5c (0xef03 bytes) 2024-09-13T07:02:34.780 INFO:teuthology.orchestra.run.smithi118.stdout:2664 writesame 0x1e9220a thru 0x1e9e7a0 data_size 0xd(0xc597 bytes) 2024-09-13T07:02:34.790 INFO:teuthology.orchestra.run.smithi118.stdout:2665 read 0x1bac4b6 thru 0x1bb8e57 (0xc9a2 bytes) 2024-09-13T07:02:34.790 INFO:teuthology.orchestra.run.smithi118.stdout:2666 write 0x632e708 thru 0x6334373 (0x5c6c bytes) 2024-09-13T07:02:34.810 INFO:teuthology.orchestra.run.smithi118.stdout:2667 trunc from 0x9fb3074 to 0xdb1ef59 2024-09-13T07:02:34.820 INFO:teuthology.orchestra.run.smithi118.stdout:2668 read 0x6cf63ad thru 0x6d00f24 (0xab78 bytes) 2024-09-13T07:02:34.820 INFO:teuthology.orchestra.run.smithi118.stdout:2669 write 0xa5ab256 thru 0xa5ae426 (0x31d1 bytes) 2024-09-13T07:02:34.825 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:02:34.828 INFO:teuthology.orchestra.run.smithi118.stdout:2670 write 0xa89793c thru 0xa89a3f4 (0x2ab9 bytes) 2024-09-13T07:02:34.837 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:34.832+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:34.837 INFO:teuthology.orchestra.run.smithi118.stdout:2671 writesame 0xa724b00 thru 0xa72d50d data_size 0x29(0x8a0e bytes) 2024-09-13T07:02:34.843 INFO:teuthology.orchestra.run.smithi118.stdout:2672 writesame 0x8c3ff85 thru 0x8c46a6e data_size 0x46(0x6aea bytes) 2024-09-13T07:02:34.852 INFO:teuthology.orchestra.run.smithi118.stdout:2673 write 0x5c829aa thru 0x5c8454b (0x1ba2 bytes) 2024-09-13T07:02:34.858 INFO:teuthology.orchestra.run.smithi118.stdout:2675 write 0x25373f5 thru 0x25468fc (0xf508 bytes) 2024-09-13T07:02:34.863 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:34.863 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:34.870 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:34.870 INFO:teuthology.orchestra.run.smithi118.stdout:2678 writesame 0x19ce62a thru 0x19d1a91 data_size 0x27(0x3468 bytes) 2024-09-13T07:02:34.874 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:34.882 INFO:teuthology.orchestra.run.smithi118.stdout:2679 trunc from 0xdb1ef59 to 0x996b2c8 2024-09-13T07:02:34.902 INFO:teuthology.orchestra.run.smithi118.stdout:2680 punch from 0x31c15fb to 0x31cf8b0, (0xe2b5 bytes) 2024-09-13T07:02:34.902 INFO:teuthology.orchestra.run.smithi118.stdout:2681 read 0x758a331 thru 0x758d0c2 (0x2d92 bytes) 2024-09-13T07:02:34.902 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:34.902 INFO:teuthology.orchestra.run.smithi118.stdout:2683 writesame 0x5d765ef thru 0x5d7839b data_size 0x1(0x1dad bytes) 2024-09-13T07:02:34.913 INFO:teuthology.orchestra.run.smithi118.stdout:2684 flatten 2024-09-13T07:02:34.938 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:34.936+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:34.978 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:35.001 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:35.001 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:35.001 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:35.012 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:35.021 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:35.043 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:35.044 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:35.040+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:35.065 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:35.087 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:35.106 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:35.108 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:35.108 INFO:teuthology.orchestra.run.smithi118.stdout:2687 punch from 0x6c39181 to 0x6c39239, (0xb8 bytes) 2024-09-13T07:02:35.108 INFO:teuthology.orchestra.run.smithi118.stdout:2688 read 0x3416c52 thru 0x3422a2c (0xbddb bytes) 2024-09-13T07:02:35.108 INFO:teuthology.orchestra.run.smithi118.stdout:2689 writesame 0x910ab10 thru 0x910b040 data_size 0x1(0x531 bytes) 2024-09-13T07:02:35.113 INFO:teuthology.orchestra.run.smithi118.stdout:2690 read 0x96482d3 thru 0x964b225 (0x2f53 bytes) 2024-09-13T07:02:35.114 INFO:teuthology.orchestra.run.smithi118.stdout:2693 write 0x8c6062a thru 0x8c6670c (0x60e3 bytes) 2024-09-13T07:02:35.120 INFO:teuthology.orchestra.run.smithi118.stdout:2694 write 0x20acec2 thru 0x20ad989 (0xac8 bytes) 2024-09-13T07:02:35.128 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:35.129 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:35.129 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:35.129 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:35.129 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:35.135 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:35.137 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:35.137 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:35.137 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:35.137 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:35.137 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:35.137 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:35.138 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:35.138 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:35.139 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:35.140 INFO:teuthology.orchestra.run.smithi118.stdout:2695 write 0xbf425e5 thru 0xbf4c3b4 (0x9dd0 bytes) 2024-09-13T07:02:35.140 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:35.138+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:35.153 INFO:teuthology.orchestra.run.smithi118.stdout:2697 read 0x2537011 thru 0x2542199 (0xb189 bytes) 2024-09-13T07:02:35.155 INFO:teuthology.orchestra.run.smithi118.stdout:2698 writesame 0x7c5c206 thru 0x7c6bdcc data_size 0x5(0xfbc7 bytes) 2024-09-13T07:02:35.168 INFO:teuthology.orchestra.run.smithi118.stdout:2700 write 0xd82c6b thru 0xd88695 (0x5a2b bytes) 2024-09-13T07:02:35.177 INFO:teuthology.orchestra.run.smithi118.stdout:2701 write 0xab5c4f6 thru 0xab68b74 (0xc67f bytes) 2024-09-13T07:02:35.187 INFO:teuthology.orchestra.run.smithi118.stdout:2703 write 0x6cec769 thru 0x6cf92db (0xcb73 bytes) 2024-09-13T07:02:35.198 INFO:teuthology.orchestra.run.smithi118.stdout:2704 writesame 0xa675a4e thru 0xa676e55 data_size 0x4(0x1408 bytes) 2024-09-13T07:02:35.203 INFO:teuthology.orchestra.run.smithi118.stdout:2705 read 0x6f62ada thru 0x6f66e3f (0x4366 bytes) 2024-09-13T07:02:35.203 INFO:teuthology.orchestra.run.smithi118.stdout:2706 read 0x44a176f thru 0x44a425f (0x2af1 bytes) 2024-09-13T07:02:35.203 INFO:teuthology.orchestra.run.smithi118.stdout:2707 write 0xa6489d7 thru 0xa648f0f (0x539 bytes) 2024-09-13T07:02:35.209 INFO:teuthology.orchestra.run.smithi118.stdout:2708 punch from 0xa115c08 to 0xa11d7aa, (0x7ba2 bytes) 2024-09-13T07:02:35.209 INFO:teuthology.orchestra.run.smithi118.stdout:2709 read 0x88113b1 thru 0x881276f (0x13bf bytes) 2024-09-13T07:02:35.209 INFO:teuthology.orchestra.run.smithi118.stdout:2710 writesame 0x68a9395 thru 0x68b89b2 data_size 0x3(0xf61e bytes) 2024-09-13T07:02:35.233 INFO:teuthology.orchestra.run.smithi118.stdout:2711 read 0x9c941c4 thru 0x9c9a0cf (0x5f0c bytes) 2024-09-13T07:02:35.233 INFO:teuthology.orchestra.run.smithi118.stdout:2712 read 0x4edf734 thru 0x4edf97c (0x249 bytes) 2024-09-13T07:02:35.233 INFO:teuthology.orchestra.run.smithi118.stdout:2713 punch from 0x4c50bc2 to 0x4c5fb4b, (0xef89 bytes) 2024-09-13T07:02:35.233 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:35.233 INFO:teuthology.orchestra.run.smithi118.stdout:2717 punch from 0x45fb20e to 0x45ff4e0, (0x42d2 bytes) 2024-09-13T07:02:35.233 INFO:teuthology.orchestra.run.smithi118.stdout:2719 write 0xba9c233 thru 0xbaa5918 (0x96e6 bytes) 2024-09-13T07:02:35.241 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:35.240+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:35.242 INFO:teuthology.orchestra.run.smithi118.stdout:2722 read 0x6b1811f thru 0x6b2495e (0xc840 bytes) 2024-09-13T07:02:35.243 INFO:teuthology.orchestra.run.smithi118.stdout:2723 write 0xb4f01f0 thru 0xb4f36e2 (0x34f3 bytes) 2024-09-13T07:02:35.247 INFO:teuthology.orchestra.run.smithi118.stdout:2724 read 0x100f22f thru 0x101dcf5 (0xeac7 bytes) 2024-09-13T07:02:35.254 INFO:teuthology.orchestra.run.smithi118.stdout:2725 trunc from 0xbf4c3b5 to 0x8309b32 2024-09-13T07:02:35.261 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.141032315, 2024-09-13T07:02:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:02:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:02:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:02:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:02:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:02:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:02:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:02:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.141011838000001, 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:02:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:02:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:02:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:02:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:02:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87036 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 6963200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134327772000001, 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042468831999999998, 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87036 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127964+0000", 2024-09-13T07:02:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7799999999999999e-06 2024-09-13T07:02:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127967+0000", 2024-09-13T07:02:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9730000000000002e-06 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127971+0000", 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4349999999999999e-06 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170202+0000", 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042230940000000002 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170293+0000", 2024-09-13T07:02:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0858000000000003e-05 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170431+0000", 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013821699999999999 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134315235000001, 2024-09-13T07:02:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:02:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:02:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:02:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:02:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:02:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:02:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:02:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:02:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134303166, 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:02:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:02:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134290722999999, 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:02:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:02:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:02:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:02:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:02:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134273125, 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:02:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:02:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:02:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:02:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:02:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:02:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134262422000001, 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:02:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:02:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:02:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134218489, 2024-09-13T07:02:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:02:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:02:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:02:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134198702999999, 2024-09-13T07:02:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:02:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:02:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134182953, 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:02:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:02:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:02:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:02:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:02:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134171701, 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.315 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134161102, 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:02:35.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:02:35.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134151609, 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:35.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:02:35.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:02:35.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134142065000001, 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:02:35.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:02:35.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134109462, 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:02:35.328 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:02:35.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.134096439, 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:02:35.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:02:35.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:02:35.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:02:35.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.130745453999999, 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.337 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:35.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:02:35.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:02:35.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:02:35.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:02:35.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:02:35.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:02:35.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:02:35.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:02:35.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:02:35.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:02:35.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:02:35.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.130725177, 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:02:35.342 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:02:35.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.347 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:35.347 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:35.347 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.130704976000001, 2024-09-13T07:02:35.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:02:35.347 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:35.347 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:35.347 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:35.347 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:35.347 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.347 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:02:35.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:35.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:35.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:02:35.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:02:35.351 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.351 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.351 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.351 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.351 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.351 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:35.353 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:35.353 INFO:teuthology.orchestra.run.smithi118.stdout:2726 read 0x24b4c84 thru 0x24b9a01 (0x4d7e bytes) 2024-09-13T07:02:35.353 INFO:teuthology.orchestra.run.smithi118.stdout:2729 trunc from 0x8309b32 to 0x1171d46 2024-09-13T07:02:35.353 INFO:teuthology.orchestra.run.smithi118.stdout:2730 write 0x35cd747 thru 0x35dc4d4 (0xed8e bytes) 2024-09-13T07:02:35.354 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:35.350+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:35.383 INFO:teuthology.orchestra.run.smithi118.stdout:2731 write 0xcfb53bb thru 0xcfc375e (0xe3a4 bytes) 2024-09-13T07:02:35.399 INFO:teuthology.orchestra.run.smithi118.stdout:2732 punch from 0xac9b0fb to 0xacaa76b, (0xf670 bytes) 2024-09-13T07:02:35.399 INFO:teuthology.orchestra.run.smithi118.stdout:2733 read 0x37a1733 thru 0x37ae143 (0xca11 bytes) 2024-09-13T07:02:35.399 INFO:teuthology.orchestra.run.smithi118.stdout:2734 punch from 0x4796daa to 0x47a513f, (0xe395 bytes) 2024-09-13T07:02:35.400 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:35.400 INFO:teuthology.orchestra.run.smithi118.stdout:2736 clone 30 order 23 su 524288 sc 7 2024-09-13T07:02:35.434 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:02:35.453 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:35.450+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:35.475 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:35.475 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:35.475 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:35.475 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:35.485 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:35.554 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:35.550+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:35.589 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:35.590 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:35.590 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:35.590 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:35.590 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:35.599 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:35.613 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:02:35.613 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:02:35.613 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T07:02:35.613 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T07:02:35.655 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:35.652+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:35.709 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-13T07:02:35.712 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1524 3.14 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.666921177000006, 2024-09-13T07:02:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061190880000000003, 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1524 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044767+0000", 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052670+0000", 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079022450000000004 2024-09-13T07:02:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052682+0000", 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2336e-05 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052696+0000", 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052755+0000", 2024-09-13T07:02:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9298e-05 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052947+0000", 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019153900000000001 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.053694+0000", 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074663099999999999 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.102148+0000", 2024-09-13T07:02:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048454329999999997 2024-09-13T07:02:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105816+0000", 2024-09-13T07:02:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036675950000000001 2024-09-13T07:02:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105921+0000", 2024-09-13T07:02:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105874 2024-09-13T07:02:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105959+0000", 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7991000000000003e-05 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:48369 3.f e58/22 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head v 58'78, mlcod=57'75)", 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.471223311999999, 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.057439072000000001, 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240464+0000", 2024-09-13T07:02:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245716+0000", 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005252329 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245745+0000", 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8844e-05 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245758+0000", 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2242000000000001e-05 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245849+0000", 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1187999999999995e-05 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245898+0000", 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8881000000000002e-05 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297837+0000", 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051938947999999999 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297906+0000", 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8955999999999995e-05 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94699 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674309+0000", 2024-09-13T07:02:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0373809539999996, 2024-09-13T07:02:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051428639999999998, 2024-09-13T07:02:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94699 2024-09-13T07:02:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674309+0000", 2024-09-13T07:02:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674309+0000", 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674308+0000", 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674311+0000", 2024-09-13T07:02:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4200000000000001e-06 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674317+0000", 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9429999999999997e-06 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674321+0000", 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6460000000000001e-06 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725637+0000", 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051315686999999999 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725687+0000", 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0096000000000003e-05 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725737+0000", 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0232999999999998e-05 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94700 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674326+0000", 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0373640240000004, 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051537695000000001, 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94700 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674326+0000", 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674326+0000", 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674325+0000", 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674328+0000", 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4399999999999999e-06 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674333+0000", 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4569999999999996e-06 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674338+0000", 2024-09-13T07:02:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6659999999999999e-06 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725762+0000", 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051423979000000002 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725816+0000", 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4509999999999998e-05 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725863+0000", 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7021999999999998e-05 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94701 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2056192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674342+0000", 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0373477940000004, 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051643316000000002, 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94701 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674342+0000", 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674342+0000", 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674342+0000", 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674344+0000", 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674349+0000", 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1880000000000001e-06 2024-09-13T07:02:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674354+0000", 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3800000000000004e-06 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725889+0000", 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051535091999999998 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725937+0000", 2024-09-13T07:02:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8680000000000001e-05 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725985+0000", 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7871000000000003e-05 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94702 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2121728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674358+0000", 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0373319060000004, 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051754434000000002, 2024-09-13T07:02:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94702 2024-09-13T07:02:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674358+0000", 2024-09-13T07:02:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674358+0000", 2024-09-13T07:02:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674358+0000", 2024-09-13T07:02:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674360+0000", 2024-09-13T07:02:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4099999999999998e-06 2024-09-13T07:02:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674365+0000", 2024-09-13T07:02:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0860000000000001e-06 2024-09-13T07:02:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674370+0000", 2024-09-13T07:02:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:02:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726009+0000", 2024-09-13T07:02:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051639184999999997 2024-09-13T07:02:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726061+0000", 2024-09-13T07:02:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1613000000000002e-05 2024-09-13T07:02:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726112+0000", 2024-09-13T07:02:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1684999999999997e-05 2024-09-13T07:02:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94703 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2236416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674374+0000", 2024-09-13T07:02:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0373158329999992, 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051856764999999999, 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94703 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674374+0000", 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674374+0000", 2024-09-13T07:02:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674373+0000", 2024-09-13T07:02:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674385+0000", 2024-09-13T07:02:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1894e-05 2024-09-13T07:02:35.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674391+0000", 2024-09-13T07:02:35.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2280000000000003e-06 2024-09-13T07:02:35.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674397+0000", 2024-09-13T07:02:35.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.164e-06 2024-09-13T07:02:35.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726136+0000", 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051739072999999997 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726181+0000", 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4914999999999999e-05 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726231+0000", 2024-09-13T07:02:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0120000000000001e-05 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94704 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2351104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674401+0000", 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0372884340000006, 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051952104999999998, 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94704 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674401+0000", 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674401+0000", 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674401+0000", 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674403+0000", 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.655e-06 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674439+0000", 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5126999999999998e-05 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674444+0000", 2024-09-13T07:02:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3970000000000002e-06 2024-09-13T07:02:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726254+0000", 2024-09-13T07:02:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051810387999999999 2024-09-13T07:02:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726305+0000", 2024-09-13T07:02:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1057000000000003e-05 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726353+0000", 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8013999999999998e-05 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94705 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2465792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674452+0000", 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.037238103, 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.052023257000000003, 2024-09-13T07:02:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94705 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674452+0000", 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674452+0000", 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674451+0000", 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674455+0000", 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.506e-06 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674461+0000", 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.455e-06 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674466+0000", 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7999999999999998e-06 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726382+0000", 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051916074999999999 2024-09-13T07:02:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726428+0000", 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5914000000000001e-05 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726475+0000", 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6993999999999998e-05 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94706 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674678+0000", 2024-09-13T07:02:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0370119550000005, 2024-09-13T07:02:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051920368000000001, 2024-09-13T07:02:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94706 2024-09-13T07:02:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674678+0000", 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674678+0000", 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674470+0000", 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9997926 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674682+0000", 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000211695 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674781+0000", 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9296000000000006e-05 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674981+0000", 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019990000000000001 2024-09-13T07:02:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726497+0000", 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051516144 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726549+0000", 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1721000000000002e-05 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726598+0000", 2024-09-13T07:02:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9039999999999998e-05 2024-09-13T07:02:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94707 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2646016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674991+0000", 2024-09-13T07:02:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0366989540000002, 2024-09-13T07:02:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051730683, 2024-09-13T07:02:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94707 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674991+0000", 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674991+0000", 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674990+0000", 2024-09-13T07:02:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674999+0000", 2024-09-13T07:02:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:02:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675007+0000", 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0870000000000007e-06 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675013+0000", 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726625+0000", 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051612492000000003 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726674+0000", 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.884e-05 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726721+0000", 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7348999999999997e-05 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94708 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2760704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675018+0000", 2024-09-13T07:02:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0366718400000003, 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051826217000000001, 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94708 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675018+0000", 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675018+0000", 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675018+0000", 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675021+0000", 2024-09-13T07:02:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1269999999999999e-06 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675026+0000", 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7610000000000004e-06 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675031+0000", 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6160000000000004e-06 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726747+0000", 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051716170999999998 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726796+0000", 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8718999999999999e-05 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726844+0000", 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8214000000000003e-05 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94709 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675035+0000", 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0366543020000005, 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051926446000000001, 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94709 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675035+0000", 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675035+0000", 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675035+0000", 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675038+0000", 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6910000000000002e-06 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675044+0000", 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3879999999999997e-06 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675049+0000", 2024-09-13T07:02:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726868+0000", 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051818636000000001 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726917+0000", 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9586999999999998e-05 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726962+0000", 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4807e-05 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94710 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675054+0000", 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0366359169999999, 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.052041476000000003, 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94710 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675054+0000", 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675054+0000", 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675053+0000", 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675056+0000", 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8619999999999999e-06 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675062+0000", 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5709999999999997e-06 2024-09-13T07:02:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675370+0000", 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030795900000000001 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726989+0000", 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051619842999999999 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727038+0000", 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8792000000000003e-05 2024-09-13T07:02:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727095+0000", 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7095000000000001e-05 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94711 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3104768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675377+0000", 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0363131419999991, 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051839969, 2024-09-13T07:02:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94711 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675377+0000", 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675377+0000", 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675376+0000", 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675380+0000", 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7749999999999999e-06 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675385+0000", 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6559999999999997e-06 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675388+0000", 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.022e-06 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727124+0000", 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051735712000000003 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727174+0000", 2024-09-13T07:02:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9716999999999998e-05 2024-09-13T07:02:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727217+0000", 2024-09-13T07:02:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2710999999999998e-05 2024-09-13T07:02:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94712 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3170304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675391+0000", 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0362987889999999, 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051946194000000001, 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94712 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.780 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675391+0000", 2024-09-13T07:02:35.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675391+0000", 2024-09-13T07:02:35.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675391+0000", 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675392+0000", 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5340000000000001e-06 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675395+0000", 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8140000000000001e-06 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675398+0000", 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5809999999999999e-06 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727240+0000", 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051842436999999998 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727288+0000", 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7901000000000001e-05 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727337+0000", 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9177e-05 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94714 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.676292+0000", 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0353976029999998, 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051320047000000001, 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.784 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94714 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676292+0000", 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676292+0000", 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676291+0000", 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676295+0000", 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8689999999999997e-06 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676302+0000", 2024-09-13T07:02:35.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4230000000000002e-06 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676307+0000", 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9320000000000004e-06 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727488+0000", 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051180941000000001 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727552+0000", 2024-09-13T07:02:35.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4376999999999997e-05 2024-09-13T07:02:35.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727612+0000", 2024-09-13T07:02:35.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0198999999999997e-05 2024-09-13T07:02:35.788 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.788 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.788 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.788 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94715 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.788 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.676311+0000", 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0353785460000005, 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051421327000000003, 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94715 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676311+0000", 2024-09-13T07:02:35.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676311+0000", 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676311+0000", 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676320+0000", 2024-09-13T07:02:35.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3989999999999993e-06 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676326+0000", 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9939999999999997e-06 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676331+0000", 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.028e-06 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727641+0000", 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051309886999999998 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727688+0000", 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6755999999999998e-05 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727733+0000", 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4668e-05 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94716 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3629056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.676336+0000", 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0353540470000002, 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051505305000000001, 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94716 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.793 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676336+0000", 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676336+0000", 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676335+0000", 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676338+0000", 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3329999999999999e-06 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676343+0000", 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3569999999999997e-06 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676347+0000", 2024-09-13T07:02:35.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2769999999999999e-06 2024-09-13T07:02:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727752+0000", 2024-09-13T07:02:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051404367999999999 2024-09-13T07:02:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727796+0000", 2024-09-13T07:02:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4057999999999997e-05 2024-09-13T07:02:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727841+0000", 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5256000000000002e-05 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94717 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3694592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.676351+0000", 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0353383439999995, 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051655690999999997, 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:35.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94717 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676351+0000", 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676351+0000", 2024-09-13T07:02:35.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676351+0000", 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676358+0000", 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6549999999999998e-06 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:35.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676363+0000", 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676368+0000", 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727869+0000", 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051501208 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:35.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727943+0000", 2024-09-13T07:02:35.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4120000000000002e-05 2024-09-13T07:02:35.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:35.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:35.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:35.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.728007+0000", 2024-09-13T07:02:35.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4096000000000004e-05 2024-09-13T07:02:35.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.801 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:35.801 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:35.801 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:35.803 INFO:teuthology.orchestra.run.smithi118.stdout:2737 trunc from 0xcfc375f to 0xcbc16b9 2024-09-13T07:02:35.803 INFO:teuthology.orchestra.run.smithi118.stdout:2739 write 0x8e9f7b5 thru 0x8eab795 (0xbfe1 bytes) 2024-09-13T07:02:35.803 INFO:teuthology.orchestra.run.smithi118.stdout:2740 punch from 0x5e1c598 to 0x5e1f141, (0x2ba9 bytes) 2024-09-13T07:02:35.804 INFO:teuthology.orchestra.run.smithi118.stdout:2741 trunc from 0xcbc16b9 to 0x4dec583 2024-09-13T07:02:35.804 INFO:teuthology.orchestra.run.smithi118.stdout:2743 punch from 0x5406bb to 0x54fc8a, (0xf5cf bytes) 2024-09-13T07:02:35.804 INFO:teuthology.orchestra.run.smithi118.stdout:2744 write 0x6376250 thru 0x6382c68 (0xca19 bytes) 2024-09-13T07:02:35.804 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:35.804 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:35.802+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:35.818 INFO:teuthology.orchestra.run.smithi118.stdout:2745 trunc from 0x6382c69 to 0xd695e98 2024-09-13T07:02:35.829 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:35.829 INFO:teuthology.orchestra.run.smithi118.stdout:2748 writesame 0x852a492 thru 0x852fa6e data_size 0x11(0x55dd bytes) 2024-09-13T07:02:35.836 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:35.836 INFO:teuthology.orchestra.run.smithi118.stdout:2750 read 0x34a279c thru 0x34b1c7b (0xf4e0 bytes) 2024-09-13T07:02:35.837 INFO:teuthology.orchestra.run.smithi118.stdout:2751 punch from 0x2ea4110 to 0x2eab90f, (0x77ff bytes) 2024-09-13T07:02:35.837 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:35.837 INFO:teuthology.orchestra.run.smithi118.stdout:2753 punch from 0xd48b9f3 to 0xd490729, (0x4d36 bytes) 2024-09-13T07:02:35.837 INFO:teuthology.orchestra.run.smithi118.stdout:2754 write 0x2b8a18f thru 0x2b95543 (0xb3b5 bytes) 2024-09-13T07:02:35.847 INFO:teuthology.orchestra.run.smithi118.stdout:2755 read 0x6a894a6 thru 0x6a8d7d5 (0x4330 bytes) 2024-09-13T07:02:35.848 INFO:teuthology.orchestra.run.smithi118.stdout:2756 read 0x99677bf thru 0x9970d9b (0x95dd bytes) 2024-09-13T07:02:35.848 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:35.848 INFO:teuthology.orchestra.run.smithi118.stdout:2758 punch from 0x54b7792 to 0x54c26c8, (0xaf36 bytes) 2024-09-13T07:02:35.867 INFO:teuthology.orchestra.run.smithi118.stdout:2759 trunc from 0xd695e98 to 0x1f16089 2024-09-13T07:02:35.904 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:35.900+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:35.915 INFO:teuthology.orchestra.run.smithi118.stdout:2762 trunc from 0x1f16089 to 0x55a1fdc 2024-09-13T07:02:35.925 INFO:teuthology.orchestra.run.smithi118.stdout:2764 trunc from 0x55a1fdc to 0x4c133f7 2024-09-13T07:02:35.928 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:02:35.928 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:35.928 INFO:teuthology.orchestra.run.smithi118.stdout: "epoch": 91, 2024-09-13T07:02:35.928 INFO:teuthology.orchestra.run.smithi118.stdout: "fsid": "25bb30d1-9541-4e40-8659-18d83bb1bd8a", 2024-09-13T07:02:35.928 INFO:teuthology.orchestra.run.smithi118.stdout: "created": "2024-09-13T07:00:49.323883+0000", 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: "modified": "2024-09-13T07:02:35.413324+0000", 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: "last_up_change": "2024-09-13T07:01:00.146832+0000", 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: "last_in_change": "2024-09-13T07:01:46.465676+0000", 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_num": 5806080, 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_set": [ 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: "nodeep-scrub", 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: "noscrub", 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: "pglog_hardlimit", 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: "purged_snapdirs", 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_deletes", 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: "sortbitwise" 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:35.929 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_version": 4, 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_snapshot": "", 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_max": 3, 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: "max_osd": 8, 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: "require_min_compat_client": "luminous", 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: "min_compat_client": "jewel", 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: "require_osd_release": "reef", 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: "allow_crimson": false, 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: "pools": [ 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 1, 2024-09-13T07:02:35.930 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": ".mgr", 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:00.675365+0000", 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 1, 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool", 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:02:35.931 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 1, 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 1, 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 1, 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 1, 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 1, 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "16", 2024-09-13T07:02:35.932 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 0, 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 0, 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:02:35.933 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:02:35.934 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "options": { 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_max": 32, 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_min": 1 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:02:35.935 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": {} 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 8, 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 8, 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.25, 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 2, 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 2, 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 0.92424774169921875, 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 0.92000001668930054, 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 0.92000001668930054 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.936 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 2, 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": "rbd", 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:07.404263+0000", 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 8193, 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:02:35.937 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 8, 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 8, 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 8, 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 8, 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 8, 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.938 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "20", 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 3, 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 20, 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:02:35.939 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:02:35.940 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:02:35.940 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:02:35.940 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:02:35.940 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:02:35.940 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:02:35.940 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:02:35.940 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:02:35.940 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:02:35.940 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:02:35.940 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:02:35.940 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:02:35.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.940 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: "options": {}, 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd": {} 2024-09-13T07:02:35.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 3, 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 3, 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.75, 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 2.25, 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 2.25, 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 0.92424774169921875, 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 0.92000001668930054, 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 0.92000001668930054 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 3, 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": "pool_client.0", 2024-09-13T07:02:35.942 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:17.321011+0000", 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 8193, 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 32, 2024-09-13T07:02:35.943 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 32, 2024-09-13T07:02:35.944 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 32, 2024-09-13T07:02:35.944 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 32, 2024-09-13T07:02:35.944 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 32, 2024-09-13T07:02:35.944 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:02:35.944 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:02:35.944 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:02:35.944 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:02:35.944 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:02:35.944 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:02:35.944 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:02:35.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.945 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "91", 2024-09-13T07:02:35.945 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:02:35.945 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:02:35.945 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:02:35.945 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:02:35.945 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:02:35.945 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 33, 2024-09-13T07:02:35.945 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 91, 2024-09-13T07:02:35.945 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:02:35.945 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:02:35.945 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:02:35.945 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:02:35.946 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: "options": {}, 2024-09-13T07:02:35.947 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd": {} 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 2.4800000190734863, 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 2.4800000190734863, 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.62999999523162842, 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 1.559999942779541, 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 1.559999942779541, 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 0.92424774169921875, 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 0.92000001668930054, 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 0.92000001668930054 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.948 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osds": [ 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 0, 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "21736eee-00cd-4eef-9725-eca7776957b4", 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 0, 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0, 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 22, 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:35.949 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6817", 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6819", 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.950 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6821", 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6823", 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.951 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6829", 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6831", 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.952 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6825", 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6827", 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.953 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6819/3731386737", 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6823/3731386737", 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6831/3731386737", 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6827/3731386737", 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 1, 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "05818db2-841d-406d-be17-8fbdab4fff69", 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:35.954 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.1844329833984375, 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 84, 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6816", 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.956 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.956 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6818", 2024-09-13T07:02:35.956 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:35.956 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.956 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.956 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:35.956 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.956 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.956 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6820", 2024-09-13T07:02:35.956 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:35.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6822", 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6828", 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6830", 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6824", 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6826", 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6818/876310814", 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6822/876310814", 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6830/876310814", 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6826/876310814", 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:35.959 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 2, 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "2387761c-42d2-481b-b5b5-b16d3a943b22", 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 0, 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0, 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 17, 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:35.960 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6808", 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6809", 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.961 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6810", 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6811", 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.962 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6814", 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6815", 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.963 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6812", 2024-09-13T07:02:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6813", 2024-09-13T07:02:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6809/690188762", 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6811/690188762", 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6815/690188762", 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6813/690188762", 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 3, 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "56500e90-f76b-47c8-a649-893400359c53", 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:35.965 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.8596343994140625, 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 0.39398193359375, 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 81, 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6800", 2024-09-13T07:02:35.966 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6801", 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6802", 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:35.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6803", 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6806", 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:35.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6807", 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6804", 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:35.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6805", 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6801/2198175481", 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6803/2198175481", 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6807/2198175481", 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6805/2198175481", 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:35.970 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 4, 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "f594a612-aa07-4565-a09e-08368cf397b6", 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.9017486572265625, 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 84, 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:35.971 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6810", 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6813", 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6817", 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6819", 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6827", 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6830", 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6822", 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6825", 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.975 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6813/1835601651", 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6819/1835601651", 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6830/1835601651", 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6825/1835601651", 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 5, 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "1013b81f-e6b4-430b-938d-88759d43b881", 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:35.976 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.95001220703125, 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 84, 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6800", 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:35.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6801", 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6802", 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:35.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6803", 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6806", 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:35.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6807", 2024-09-13T07:02:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6804", 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6805", 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6801/2853696331", 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6803/2853696331", 2024-09-13T07:02:35.981 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6807/2853696331", 2024-09-13T07:02:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6805/2853696331", 2024-09-13T07:02:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 6, 2024-09-13T07:02:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "367f5f5b-5919-4534-b88c-229f327e4950", 2024-09-13T07:02:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:02:35.982 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 84, 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6808", 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6812", 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6815", 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6818", 2024-09-13T07:02:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6828", 2024-09-13T07:02:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:35.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6831", 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6821", 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:35.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6824", 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6812/131309660", 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6818/131309660", 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6831/131309660", 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6824/131309660", 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:35.987 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:35.988 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.988 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 7, 2024-09-13T07:02:35.988 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "90d9e486-e1ae-4305-80a6-6f483e534643", 2024-09-13T07:02:35.988 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:35.988 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:35.988 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:02:35.988 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:35.988 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:35.988 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:35.988 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:35.988 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 84, 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6809", 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6811", 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.989 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6814", 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6816", 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.990 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6826", 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6829", 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.991 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6820", 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6823", 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.992 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6811/652173699", 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6816/652173699", 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6829/652173699", 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6823/652173699", 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_xinfo": [ 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.993 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 0, 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 54166, 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.059233+0000", 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 1, 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:35.994 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.083130+0000", 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 2, 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 65536, 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.848896+0000", 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:35.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 3, 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.847456+0000", 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 4, 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:35.996 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.914135+0000", 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 5, 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:35.997 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.834284+0000", 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 6, 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.003715+0000", 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 7, 2024-09-13T07:02:35.998 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:35.999 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:35.999 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:35.999 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:35.999 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:35.999 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.033514+0000", 2024-09-13T07:02:35.999 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:35.999 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:35.999 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:35.999 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap": [], 2024-09-13T07:02:35.999 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap_items": [], 2024-09-13T07:02:35.999 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap_primaries": [], 2024-09-13T07:02:35.999 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_temp": [], 2024-09-13T07:02:35.999 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_temp": [], 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: "blocklist": {}, 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: "range_blocklist": {}, 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profiles": { 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: "default": { 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: "crush-failure-domain": "osd", 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: "k": "2", 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: "m": "1", 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: "plugin": "jerasure", 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: "technique": "reed_sol_van" 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps_queue": [], 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: "new_removed_snaps": [], 2024-09-13T07:02:36.000 INFO:teuthology.orchestra.run.smithi118.stdout: "new_purged_snaps": [], 2024-09-13T07:02:36.001 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_node_flags": {}, 2024-09-13T07:02:36.001 INFO:teuthology.orchestra.run.smithi118.stdout: "device_class_flags": {}, 2024-09-13T07:02:36.001 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode": { 2024-09-13T07:02:36.001 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode_enabled": false, 2024-09-13T07:02:36.001 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_bucket_count": 0, 2024-09-13T07:02:36.001 INFO:teuthology.orchestra.run.smithi118.stdout: "degraded_stretch_mode": 0, 2024-09-13T07:02:36.001 INFO:teuthology.orchestra.run.smithi118.stdout: "recovering_stretch_mode": 0, 2024-09-13T07:02:36.001 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode_bucket": 0 2024-09-13T07:02:36.001 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:36.001 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:36.002 INFO:teuthology.orchestra.run.smithi118.stdout:2765 read 0x2a68765 thru 0x2a6f552 (0x6dee bytes) 2024-09-13T07:02:36.002 INFO:teuthology.orchestra.run.smithi118.stdout:2766 write 0x16b149c thru 0x16b773f (0x62a4 bytes) 2024-09-13T07:02:36.003 INFO:teuthology.orchestra.run.smithi118.stdout:2768 write 0x4bd846a thru 0x4be6aa3 (0xe63a bytes) 2024-09-13T07:02:36.003 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:36.003 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:36.003 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:36.003 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:36.004 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:36.005 INFO:tasks.thrashosds.thrasher:j is {'epoch': 91, 'fsid': '25bb30d1-9541-4e40-8659-18d83bb1bd8a', 'created': '2024-09-13T07:00:49.323883+0000', 'modified': '2024-09-13T07:02:35.413324+0000', 'last_up_change': '2024-09-13T07:01:00.146832+0000', 'last_in_change': '2024-09-13T07:01:46.465676+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T07:01:00.675365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9242477416992188, 'average_primary_affinity': 0.9200000166893005, 'average_primary_affinity_weighted': 0.9200000166893005}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T07:01:07.404263+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9242477416992188, 'average_primary_affinity': 0.9200000166893005, 'average_primary_affinity_weighted': 0.9200000166893005}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T07:01:17.321011+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 91, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.4800000190734863, 'score_stable': 2.4800000190734863, 'optimal_score': 0.6299999952316284, 'raw_score_acting': 1.559999942779541, 'raw_score_stable': 1.559999942779541, 'primary_affinity_weighted': 0.9242477416992188, 'average_primary_affinity': 0.9200000166893005, 'average_primary_affinity_weighted': 0.9200000166893005}}], 'osds': [{'osd': 0, 'uuid': '21736eee-00cd-4eef-9725-eca7776957b4', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6817', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6819', 'nonce': 3731386737}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6821', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6823', 'nonce': 3731386737}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6829', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6831', 'nonce': 3731386737}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6825', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6827', 'nonce': 3731386737}]}, 'public_addr': '172.21.15.118:6819/3731386737', 'cluster_addr': '172.21.15.118:6823/3731386737', 'heartbeat_back_addr': '172.21.15.118:6831/3731386737', 'heartbeat_front_addr': '172.21.15.118:6827/3731386737', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '05818db2-841d-406d-be17-8fbdab4fff69', 'up': 1, 'in': 1, 'weight': 0.1844329833984375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6816', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6818', 'nonce': 876310814}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6820', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6822', 'nonce': 876310814}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6828', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6830', 'nonce': 876310814}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6824', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6826', 'nonce': 876310814}]}, 'public_addr': '172.21.15.118:6818/876310814', 'cluster_addr': '172.21.15.118:6822/876310814', 'heartbeat_back_addr': '172.21.15.118:6830/876310814', 'heartbeat_front_addr': '172.21.15.118:6826/876310814', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2387761c-42d2-481b-b5b5-b16d3a943b22', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6808', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6809', 'nonce': 690188762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6810', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6811', 'nonce': 690188762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6814', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6815', 'nonce': 690188762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6812', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6813', 'nonce': 690188762}]}, 'public_addr': '172.21.15.118:6809/690188762', 'cluster_addr': '172.21.15.118:6811/690188762', 'heartbeat_back_addr': '172.21.15.118:6815/690188762', 'heartbeat_front_addr': '172.21.15.118:6813/690188762', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '56500e90-f76b-47c8-a649-893400359c53', 'up': 1, 'in': 1, 'weight': 0.8596343994140625, 'primary_affinity': 0.39398193359375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6800', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6801', 'nonce': 2198175481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6802', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6803', 'nonce': 2198175481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6806', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6807', 'nonce': 2198175481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6804', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6805', 'nonce': 2198175481}]}, 'public_addr': '172.21.15.118:6801/2198175481', 'cluster_addr': '172.21.15.118:6803/2198175481', 'heartbeat_back_addr': '172.21.15.118:6807/2198175481', 'heartbeat_front_addr': '172.21.15.118:6805/2198175481', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f594a612-aa07-4565-a09e-08368cf397b6', 'up': 1, 'in': 1, 'weight': 0.9017486572265625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 1835601651}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6817', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 1835601651}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6827', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6830', 'nonce': 1835601651}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 1835601651}]}, 'public_addr': '172.21.15.164:6813/1835601651', 'cluster_addr': '172.21.15.164:6819/1835601651', 'heartbeat_back_addr': '172.21.15.164:6830/1835601651', 'heartbeat_front_addr': '172.21.15.164:6825/1835601651', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1013b81f-e6b4-430b-938d-88759d43b881', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 2853696331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 2853696331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 2853696331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 2853696331}]}, 'public_addr': '172.21.15.164:6801/2853696331', 'cluster_addr': '172.21.15.164:6803/2853696331', 'heartbeat_back_addr': '172.21.15.164:6807/2853696331', 'heartbeat_front_addr': '172.21.15.164:6805/2853696331', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '367f5f5b-5919-4534-b88c-229f327e4950', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6812', 'nonce': 131309660}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6815', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6818', 'nonce': 131309660}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 131309660}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6821', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6824', 'nonce': 131309660}]}, 'public_addr': '172.21.15.164:6812/131309660', 'cluster_addr': '172.21.15.164:6818/131309660', 'heartbeat_back_addr': '172.21.15.164:6831/131309660', 'heartbeat_front_addr': '172.21.15.164:6824/131309660', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '90d9e486-e1ae-4305-80a6-6f483e534643', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6809', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 652173699}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6816', 'nonce': 652173699}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 652173699}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 652173699}]}, 'public_addr': '172.21.15.164:6811/652173699', 'cluster_addr': '172.21.15.164:6816/652173699', 'heartbeat_back_addr': '172.21.15.164:6829/652173699', 'heartbeat_front_addr': '172.21.15.164:6823/652173699', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 54166, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.059233+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.083130+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.848896+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.847456+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.914135+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.834284+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.003715+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.033514+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T07:02:36.005 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:02:36.007 INFO:teuthology.orchestra.run.smithi118.stdout:2769 trunc from 0x4c133f7 to 0xe40899b 2024-09-13T07:02:36.007 INFO:teuthology.orchestra.run.smithi118.stdout:2770 read 0x6627900 thru 0x662896c (0x106d bytes) 2024-09-13T07:02:36.007 INFO:teuthology.orchestra.run.smithi118.stdout:2771 write 0x874d485 thru 0x875164b (0x41c7 bytes) 2024-09-13T07:02:36.007 INFO:teuthology.orchestra.run.smithi118.stdout:2773 punch from 0xd7d1a13 to 0xd7e0b0c, (0xf0f9 bytes) 2024-09-13T07:02:36.007 INFO:teuthology.orchestra.run.smithi118.stdout:2775 read 0x4bfd2f1 thru 0x4c05b94 (0x88a4 bytes) 2024-09-13T07:02:36.007 INFO:teuthology.orchestra.run.smithi118.stdout:2776 writesame 0xce5b96e thru 0xce62964 data_size 0x1(0x6ff7 bytes) 2024-09-13T07:02:36.007 INFO:teuthology.orchestra.run.smithi118.stdout:2777 write 0xbc84c6b thru 0xbc8f577 (0xa90d bytes) 2024-09-13T07:02:36.008 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.008 INFO:teuthology.orchestra.run.smithi118.stdout:2779 read 0x8cc686b thru 0x8ccc676 (0x5e0c bytes) 2024-09-13T07:02:36.008 INFO:teuthology.orchestra.run.smithi118.stdout:2780 punch from 0xae497b to 0xaf21e1, (0xd866 bytes) 2024-09-13T07:02:36.008 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.008 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:36.006+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:36.015 INFO:teuthology.orchestra.run.smithi118.stdout:2783 trunc from 0xe40899b to 0x9712cf0 2024-09-13T07:02:36.037 INFO:teuthology.orchestra.run.smithi118.stdout:2784 punch from 0x3de4651 to 0x3df2d4c, (0xe6fb bytes) 2024-09-13T07:02:36.037 INFO:teuthology.orchestra.run.smithi118.stdout:2785 writesame 0x2508e3c thru 0x250ae1c data_size 0x1(0x1fe1 bytes) 2024-09-13T07:02:36.045 INFO:teuthology.orchestra.run.smithi118.stdout:2786 read 0x2a11540 thru 0x2a14d4f (0x3810 bytes) 2024-09-13T07:02:36.046 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.046 INFO:teuthology.orchestra.run.smithi118.stdout:2788 read 0x4e646fc thru 0x4e6b409 (0x6d0e bytes) 2024-09-13T07:02:36.046 INFO:teuthology.orchestra.run.smithi118.stdout:2791 write 0x4bd3379 thru 0x4bd5b1c (0x27a4 bytes) 2024-09-13T07:02:36.049 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.050 INFO:teuthology.orchestra.run.smithi118.stdout:2793 writesame 0x9106db7 thru 0x91101cd data_size 0x1(0x9417 bytes) 2024-09-13T07:02:36.081 INFO:teuthology.orchestra.run.smithi118.stdout:2794 trunc from 0x9712cf0 to 0x9fa65d6 2024-09-13T07:02:36.092 INFO:teuthology.orchestra.run.smithi118.stdout:2795 read 0x78b8b57 thru 0x78c7e2a (0xf2d4 bytes) 2024-09-13T07:02:36.093 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.093 INFO:teuthology.orchestra.run.smithi118.stdout:2798 write 0x6d4dd23 thru 0x6d5339f (0x567d bytes) 2024-09-13T07:02:36.100 INFO:teuthology.orchestra.run.smithi118.stdout:2799 read 0x944d11e thru 0x9451d19 (0x4bfc bytes) 2024-09-13T07:02:36.100 INFO:teuthology.orchestra.run.smithi118.stdout:2801 write 0x7a3bbd1 thru 0x7a455ba (0x99ea bytes) 2024-09-13T07:02:36.106 INFO:teuthology.orchestra.run.smithi118.stdout:2802 punch from 0x904d205 to 0x9052738, (0x5533 bytes) 2024-09-13T07:02:36.108 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:36.104+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:36.120 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:36.121 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:36.122 INFO:teuthology.orchestra.run.smithi118.stdout:2803 trunc from 0x9fa65d6 to 0x1b2882d 2024-09-13T07:02:36.132 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:36.166 INFO:teuthology.orchestra.run.smithi118.stdout:2804 writesame 0xb0f83ba thru 0xb0f9d20 data_size 0x1(0x1967 bytes) 2024-09-13T07:02:36.166 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.178 INFO:teuthology.orchestra.run.smithi118.stdout:2809 writesame 0xbf0ef7f thru 0xbf14dfb data_size 0x3(0x5e7d bytes) 2024-09-13T07:02:36.178 INFO:teuthology.orchestra.run.smithi118.stdout:2810 read 0xa45343e thru 0xa45799c (0x455f bytes) 2024-09-13T07:02:36.178 INFO:teuthology.orchestra.run.smithi118.stdout:2812 read 0x48061ef thru 0x480b1e1 (0x4ff3 bytes) 2024-09-13T07:02:36.181 INFO:teuthology.orchestra.run.smithi118.stdout:2814 trunc from 0xbf14dfc to 0xe295bbd 2024-09-13T07:02:36.194 INFO:teuthology.orchestra.run.smithi118.stdout:2817 writesame 0x1f5c4eb thru 0x1f5c729 data_size 0x5(0x23f bytes) 2024-09-13T07:02:36.208 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:36.206+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:36.220 INFO:teuthology.orchestra.run.smithi118.stdout:2818 trunc from 0xe295bbd to 0x99ab98 2024-09-13T07:02:36.234 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:36.234 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:36.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:36.235 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:36.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.235 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1531 3.c 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:36.235 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:36.235 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.183917128999994, 2024-09-13T07:02:36.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063168486999999995, 2024-09-13T07:02:36.235 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.235 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:36.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.235 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1531 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049511+0000", 2024-09-13T07:02:36.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:36.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069234+0000", 2024-09-13T07:02:36.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019723920999999998 2024-09-13T07:02:36.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069269+0000", 2024-09-13T07:02:36.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4446999999999997e-05 2024-09-13T07:02:36.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069286+0000", 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7405999999999999e-05 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069370+0000", 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-05 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069511+0000", 2024-09-13T07:02:36.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014134299999999999 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070463+0000", 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095167899999999998 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105076+0000", 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034613139000000001 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:36.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112562+0000", 2024-09-13T07:02:36.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074861429999999998 2024-09-13T07:02:36.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:36.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112631+0000", 2024-09-13T07:02:36.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:02:36.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112680+0000", 2024-09-13T07:02:36.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9607000000000001e-05 2024-09-13T07:02:36.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.240 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.241 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7628 3.1f 3:fdf2e50b:::rbd_data.1194c5936dae.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1032192~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:02:36.241 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:36.241 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.470990972000003, 2024-09-13T07:02:36.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.630407088, 2024-09-13T07:02:36.241 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.241 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:36.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.241 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7628 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762435+0000", 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762543+0000", 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010850500000000001 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762597+0000", 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4262000000000003e-05 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762626+0000", 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9176e-05 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762761+0000", 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013405599999999999 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762776+0000", 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5679999999999999e-05 2024-09-13T07:02:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388492+0000", 2024-09-13T07:02:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.62571606300000004 2024-09-13T07:02:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for missing object", 2024-09-13T07:02:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388614+0000", 2024-09-13T07:02:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012126 2024-09-13T07:02:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390837+0000", 2024-09-13T07:02:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002223806 2024-09-13T07:02:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390963+0000", 2024-09-13T07:02:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012550300000000001 2024-09-13T07:02:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-13T07:02:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391127+0000", 2024-09-13T07:02:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163995 2024-09-13T07:02:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391989+0000", 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086230499999999995 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392662+0000", 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067260799999999999 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392728+0000", 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6501000000000006e-05 2024-09-13T07:02:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392845+0000", 2024-09-13T07:02:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116665 2024-09-13T07:02:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:02:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.114840663000003, 2024-09-13T07:02:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:02:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:02:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:02:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:02:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:02:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:02:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:02:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:02:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:02:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:02:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:02:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:02:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:02:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:02:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:02:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:02:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:02:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:02:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:02:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:02:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:02:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:02:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48376 3.c 3:350dc8a4:::rbd_data.1194fc792839.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.004219820000003, 2024-09-13T07:02:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.068448320000000007, 2024-09-13T07:02:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48376 2024-09-13T07:02:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229207+0000", 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.241745+0000", 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012537726000000001 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244174+0000", 2024-09-13T07:02:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024284760000000002 2024-09-13T07:02:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244198+0000", 2024-09-13T07:02:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4629000000000001e-05 2024-09-13T07:02:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244312+0000", 2024-09-13T07:02:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114282 2024-09-13T07:02:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244679+0000", 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000366399 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.246046+0000", 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013672479999999999 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.266272+0000", 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020226381000000002 2024-09-13T07:02:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297510+0000", 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031237576999999999 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297595+0000", 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5413000000000004e-05 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297657+0000", 2024-09-13T07:02:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1860999999999996e-05 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95510 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 2048000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.849258+0000", 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3841706350000003, 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053515343, 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95510 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849258+0000", 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849258+0000", 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849258+0000", 2024-09-13T07:02:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849260+0000", 2024-09-13T07:02:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.075e-06 2024-09-13T07:02:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849264+0000", 2024-09-13T07:02:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9720000000000003e-06 2024-09-13T07:02:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849268+0000", 2024-09-13T07:02:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8349999999999997e-06 2024-09-13T07:02:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.902589+0000", 2024-09-13T07:02:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053320871999999998 2024-09-13T07:02:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.902675+0000", 2024-09-13T07:02:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6732999999999998e-05 2024-09-13T07:02:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.902774+0000", 2024-09-13T07:02:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8169000000000001e-05 2024-09-13T07:02:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95511 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 2113536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.849271+0000", 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3841581569999999, 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053615055000000002, 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95511 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849271+0000", 2024-09-13T07:02:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849271+0000", 2024-09-13T07:02:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849270+0000", 2024-09-13T07:02:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849272+0000", 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1449999999999998e-06 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849277+0000", 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2069999999999997e-06 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849281+0000", 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-13T07:02:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.902792+0000", 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053511712000000003 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.902838+0000", 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.621e-05 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.902886+0000", 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7284000000000003e-05 2024-09-13T07:02:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95512 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 2228224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.849283+0000", 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3841453759999993, 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053698285999999998, 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95512 2024-09-13T07:02:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849283+0000", 2024-09-13T07:02:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849283+0000", 2024-09-13T07:02:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849283+0000", 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849285+0000", 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.024e-06 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849289+0000", 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0339999999999998e-06 2024-09-13T07:02:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849293+0000", 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8809999999999998e-06 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.902900+0000", 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053606912999999999 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.902937+0000", 2024-09-13T07:02:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6692e-05 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.902982+0000", 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5062000000000003e-05 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95513 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 2342912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.849296+0000", 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3841328320000006, 2024-09-13T07:02:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053777519000000003, 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95513 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849296+0000", 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849296+0000", 2024-09-13T07:02:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849296+0000", 2024-09-13T07:02:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849298+0000", 2024-09-13T07:02:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9709999999999998e-06 2024-09-13T07:02:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849302+0000", 2024-09-13T07:02:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0990000000000001e-06 2024-09-13T07:02:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849306+0000", 2024-09-13T07:02:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9060000000000004e-06 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.902995+0000", 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053689579000000001 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903031+0000", 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5873e-05 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903074+0000", 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2407000000000002e-05 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95514 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 2457600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.849309+0000", 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3841198909999992, 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053948680999999998, 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95514 2024-09-13T07:02:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849309+0000", 2024-09-13T07:02:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849309+0000", 2024-09-13T07:02:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849309+0000", 2024-09-13T07:02:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849311+0000", 2024-09-13T07:02:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0889999999999998e-06 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849315+0000", 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849319+0000", 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1130000000000003e-06 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903088+0000", 2024-09-13T07:02:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053768942 2024-09-13T07:02:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903159+0000", 2024-09-13T07:02:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0811999999999999e-05 2024-09-13T07:02:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903258+0000", 2024-09-13T07:02:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8994999999999997e-05 2024-09-13T07:02:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95515 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.849573+0000", 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3838559270000008, 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053873800999999999, 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95515 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849573+0000", 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849573+0000", 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849324+0000", 2024-09-13T07:02:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9997506 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849584+0000", 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026026899999999998 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849596+0000", 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2305e-05 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849605+0000", 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1230000000000006e-06 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903289+0000", 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053683467999999998 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903384+0000", 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5080000000000004e-05 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903447+0000", 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2805999999999995e-05 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95516 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 2637824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.849613+0000", 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3838156539999993, 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053933448000000002, 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95516 2024-09-13T07:02:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849613+0000", 2024-09-13T07:02:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849613+0000", 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849613+0000", 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849616+0000", 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629999999999998e-06 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849627+0000", 2024-09-13T07:02:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0908e-05 2024-09-13T07:02:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849634+0000", 2024-09-13T07:02:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8379999999999996e-06 2024-09-13T07:02:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903461+0000", 2024-09-13T07:02:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053827331999999999 2024-09-13T07:02:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903501+0000", 2024-09-13T07:02:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9351999999999999e-05 2024-09-13T07:02:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903547+0000", 2024-09-13T07:02:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5850999999999999e-05 2024-09-13T07:02:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95517 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 2752512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.849809+0000", 2024-09-13T07:02:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3836199049999998, 2024-09-13T07:02:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053830098, 2024-09-13T07:02:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95517 2024-09-13T07:02:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849809+0000", 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849809+0000", 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849639+0000", 2024-09-13T07:02:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9998302 2024-09-13T07:02:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849813+0000", 2024-09-13T07:02:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017424699999999999 2024-09-13T07:02:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849823+0000", 2024-09-13T07:02:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7389999999999994e-06 2024-09-13T07:02:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849830+0000", 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0269999999999998e-06 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903560+0000", 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053730131 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903597+0000", 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6412000000000003e-05 2024-09-13T07:02:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903639+0000", 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2347e-05 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95518 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.850011+0000", 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3834178010000002, 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053760815000000003, 2024-09-13T07:02:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95518 2024-09-13T07:02:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.850011+0000", 2024-09-13T07:02:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.850011+0000", 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.849836+0000", 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999825 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.850016+0000", 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000179801 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.850029+0000", 2024-09-13T07:02:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2887e-05 2024-09-13T07:02:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.850033+0000", 2024-09-13T07:02:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4530000000000004e-06 2024-09-13T07:02:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903653+0000", 2024-09-13T07:02:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053620022000000003 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903692+0000", 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9264999999999998e-05 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.903772+0000", 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9457999999999997e-05 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95524 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 3506176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.851147+0000", 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3822822909999992, 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053522770999999997, 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95524 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.851147+0000", 2024-09-13T07:02:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.851147+0000", 2024-09-13T07:02:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.851078+0000", 2024-09-13T07:02:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999313 2024-09-13T07:02:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.851151+0000", 2024-09-13T07:02:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3646000000000006e-05 2024-09-13T07:02:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.851160+0000", 2024-09-13T07:02:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.456e-06 2024-09-13T07:02:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.851166+0000", 2024-09-13T07:02:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3040000000000001e-06 2024-09-13T07:02:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.904502+0000", 2024-09-13T07:02:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053335670000000002 2024-09-13T07:02:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.904604+0000", 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101724 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.904669+0000", 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5778999999999995e-05 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95529 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 4030464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.852195+0000", 2024-09-13T07:02:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3812337699999997, 2024-09-13T07:02:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053646903000000003, 2024-09-13T07:02:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95529 2024-09-13T07:02:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.852195+0000", 2024-09-13T07:02:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.852195+0000", 2024-09-13T07:02:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.852068+0000", 2024-09-13T07:02:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9998732 2024-09-13T07:02:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.852198+0000", 2024-09-13T07:02:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012977 2024-09-13T07:02:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.852205+0000", 2024-09-13T07:02:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2420000000000003e-06 2024-09-13T07:02:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.852212+0000", 2024-09-13T07:02:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6959999999999999e-06 2024-09-13T07:02:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.905514+0000", 2024-09-13T07:02:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053302083 2024-09-13T07:02:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.905655+0000", 2024-09-13T07:02:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000141011 2024-09-13T07:02:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.905842+0000", 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018681599999999999 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95566 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.853251+0000", 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3801782429999996, 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053738603000000003, 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95566 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853251+0000", 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853251+0000", 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853250+0000", 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853253+0000", 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6410000000000002e-06 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853261+0000", 2024-09-13T07:02:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5760000000000006e-06 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853267+0000", 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1569999999999999e-06 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.906759+0000", 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.05349201 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.906859+0000", 2024-09-13T07:02:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9974000000000002e-05 2024-09-13T07:02:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.906989+0000", 2024-09-13T07:02:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001306 2024-09-13T07:02:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95567 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.853413+0000", 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3800158529999997, 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053695000999999999, 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95567 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853413+0000", 2024-09-13T07:02:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853413+0000", 2024-09-13T07:02:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853271+0000", 2024-09-13T07:02:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9998584 2024-09-13T07:02:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853417+0000", 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014561700000000001 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853424+0000", 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000002e-06 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853428+0000", 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2130000000000002e-06 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.906991+0000", 2024-09-13T07:02:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053563107999999998 2024-09-13T07:02:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.907050+0000", 2024-09-13T07:02:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9098999999999998e-05 2024-09-13T07:02:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.907108+0000", 2024-09-13T07:02:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7525999999999998e-05 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95569 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 663552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.853996+0000", 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3794332869999995, 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053510751000000002, 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95569 2024-09-13T07:02:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853996+0000", 2024-09-13T07:02:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853996+0000", 2024-09-13T07:02:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853995+0000", 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.853999+0000", 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1880000000000004e-06 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854005+0000", 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4930000000000003e-06 2024-09-13T07:02:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854011+0000", 2024-09-13T07:02:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1129999999999997e-06 2024-09-13T07:02:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.907391+0000", 2024-09-13T07:02:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053379159000000002 2024-09-13T07:02:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.907448+0000", 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7216000000000002e-05 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.907506+0000", 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8449000000000002e-05 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95570 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.854027+0000", 2024-09-13T07:02:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3794017889999992, 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053714619999999998, 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95570 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854027+0000", 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854027+0000", 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854027+0000", 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854030+0000", 2024-09-13T07:02:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8929999999999998e-06 2024-09-13T07:02:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854044+0000", 2024-09-13T07:02:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3509999999999999e-05 2024-09-13T07:02:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854050+0000", 2024-09-13T07:02:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0639999999999999e-06 2024-09-13T07:02:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.907533+0000", 2024-09-13T07:02:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053483112999999999 2024-09-13T07:02:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.907643+0000", 2024-09-13T07:02:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010983499999999999 2024-09-13T07:02:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.907742+0000", 2024-09-13T07:02:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8727999999999994e-05 2024-09-13T07:02:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:95571 3.c 3:3570d586:::rbd_data.1194a0700cb4.000000000000001b:1c [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.854223+0000", 2024-09-13T07:02:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3792057240000002, 2024-09-13T07:02:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053644731000000001, 2024-09-13T07:02:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.331 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 95571 2024-09-13T07:02:36.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.331 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854223+0000", 2024-09-13T07:02:36.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854223+0000", 2024-09-13T07:02:36.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854054+0000", 2024-09-13T07:02:36.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9998312 2024-09-13T07:02:36.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854239+0000", 2024-09-13T07:02:36.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018473099999999999 2024-09-13T07:02:36.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854248+0000", 2024-09-13T07:02:36.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8550000000000008e-06 2024-09-13T07:02:36.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.854255+0000", 2024-09-13T07:02:36.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9140000000000002e-06 2024-09-13T07:02:36.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.907757+0000", 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.053501924999999999 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.907810+0000", 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3294000000000002e-05 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.907868+0000", 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7626e-05 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.335 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.335 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:36.337 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:36.337 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:36.334+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:36.338 INFO:teuthology.orchestra.run.smithi118.stdout:2819 trunc from 0x99ab98 to 0x59cafc8 2024-09-13T07:02:36.338 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.338 INFO:teuthology.orchestra.run.smithi118.stdout:2822 write 0xabbe2fe thru 0xabc469b (0x639e bytes) 2024-09-13T07:02:36.338 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.338 INFO:teuthology.orchestra.run.smithi118.stdout:2825 trunc from 0xabc469c to 0x96e6d45 2024-09-13T07:02:36.338 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.338 INFO:teuthology.orchestra.run.smithi118.stdout:2827 write 0x91c0d67 thru 0x91c7b3e (0x6dd8 bytes) 2024-09-13T07:02:36.339 INFO:teuthology.orchestra.run.smithi118.stdout:2828 punch from 0x639f67e to 0x63abafe, (0xc480 bytes) 2024-09-13T07:02:36.339 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.339 INFO:teuthology.orchestra.run.smithi118.stdout:2830 writesame 0xbfc1891 thru 0xbfcc580 data_size 0x8(0xacf0 bytes) 2024-09-13T07:02:36.341 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:02:36.342 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:02:36.342 INFO:teuthology.orchestra.run.smithi118.stdout:2831 write 0xd4266b6 thru 0xd42dba6 (0x74f1 bytes) 2024-09-13T07:02:36.342 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.342 INFO:teuthology.orchestra.run.smithi118.stdout:2834 write 0x13fb636 thru 0x13fec31 (0x35fc bytes) 2024-09-13T07:02:36.343 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-13T07:02:35.422260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188304754,"num_objects":1119,"num_object_clones":266,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1119,"num_whiteouts":432,"num_read":125136,"num_read_kb":437649,"num_write":9162,"num_write_kb":281847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":693,"num_bytes_recovered":28889828,"num_keys_recovered":153,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6849,"ondisk_log_size":6849,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":476392,"kb_used_data":256920,"kb_used_omap":9,"kb_used_meta":219382,"kb_avail":565754648,"statfs":{"total":579820584960,"available":579332759552,"internally_reserved":0,"allocated":263086080,"data_stored":297959437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9556,"internal_metadata":224647852},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":72946154,"num_objects":107,"num_object_clones":22,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":14,"num_read":8917,"num_read_kb":56918,"num_write":679,"num_write_kb":14959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.021688"},"pg_stats":[{"pgid":"3.1f","version":"87'117","reported_seq":4528,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:29.356426+0000","last_change":"2024-09-13T07:01:41.516000+0000","last_active":"2024-09-13T07:02:29.356426+0000","last_peered":"2024-09-13T07:02:29.356426+0000","last_clean":"2024-09-13T07:02:29.356426+0000","last_became_active":"2024-09-13T07:01:41.437584+0000","last_became_peered":"2024-09-13T07:01:41.437584+0000","last_unstale":"2024-09-13T07:02:29.356426+0000","last_undegraded":"2024-09-13T07:02:29.356426+0000","last_fullsized":"2024-09-13T07:02:29.356426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:20:47.432372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013321120000000001,"stat_sum":{"num_bytes":47907822,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":17,"num_read":4300,"num_read_kb":17662,"num_write":203,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":834141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"87'141","reported_seq":2801,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:29.696075+0000","last_change":"2024-09-13T07:02:26.373963+0000","last_active":"2024-09-13T07:02:29.696075+0000","last_peered":"2024-09-13T07:02:29.696075+0000","last_clean":"2024-09-13T07:02:29.696075+0000","last_became_active":"2024-09-13T07:02:26.373467+0000","last_became_peered":"2024-09-13T07:02:26.373467+0000","last_unstale":"2024-09-13T07:02:29.696075+0000","last_undegraded":"2024-09-13T07:02:29.696075+0000","last_fullsized":"2024-09-13T07:02:29.696075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:30.228938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001950586,"stat_sum":{"num_bytes":73284667,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":15,"num_read":2687,"num_read_kb":28252,"num_write":170,"num_write_kb":4070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"87'233","reported_seq":1289,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:29.699895+0000","last_change":"2024-09-13T07:01:21.293288+0000","last_active":"2024-09-13T07:02:29.699895+0000","last_peered":"2024-09-13T07:02:29.699895+0000","last_clean":"2024-09-13T07:02:29.699895+0000","last_became_active":"2024-09-13T07:01:19.276445+0000","last_became_peered":"2024-09-13T07:01:19.276445+0000","last_unstale":"2024-09-13T07:02:29.699895+0000","last_undegraded":"2024-09-13T07:02:29.699895+0000","last_fullsized":"2024-09-13T07:02:29.699895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:16:37.731881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076373900000000004,"stat_sum":{"num_bytes":18092678,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":2011,"num_read_kb":7600,"num_write":304,"num_write_kb":5078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"88'139","reported_seq":4146,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.636692+0000","last_change":"2024-09-13T07:01:21.292234+0000","last_active":"2024-09-13T07:02:30.636692+0000","last_peered":"2024-09-13T07:02:30.636692+0000","last_clean":"2024-09-13T07:02:30.636692+0000","last_became_active":"2024-09-13T07:01:19.276459+0000","last_became_peered":"2024-09-13T07:01:19.276459+0000","last_unstale":"2024-09-13T07:02:30.636692+0000","last_undegraded":"2024-09-13T07:02:30.636692+0000","last_fullsized":"2024-09-13T07:02:30.636692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:50:50.173262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050866599999999996,"stat_sum":{"num_bytes":31121529,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":4634,"num_read_kb":10697,"num_write":194,"num_write_kb":5111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"88'112","reported_seq":2551,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.561173+0000","last_change":"2024-09-13T07:01:34.715025+0000","last_active":"2024-09-13T07:02:30.561173+0000","last_peered":"2024-09-13T07:02:30.561173+0000","last_clean":"2024-09-13T07:02:30.561173+0000","last_became_active":"2024-09-13T07:01:34.676996+0000","last_became_peered":"2024-09-13T07:01:34.676996+0000","last_unstale":"2024-09-13T07:02:30.561173+0000","last_undegraded":"2024-09-13T07:02:30.561173+0000","last_fullsized":"2024-09-13T07:02:30.561173+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:26:51.849086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083111000000000003,"stat_sum":{"num_bytes":47455970,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":2423,"num_read_kb":17588,"num_write":171,"num_write_kb":6068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":67840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"87'384","reported_seq":3245,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.665774+0000","last_change":"2024-09-13T07:02:23.332383+0000","last_active":"2024-09-13T07:02:30.665774+0000","last_peered":"2024-09-13T07:02:30.665774+0000","last_clean":"2024-09-13T07:02:30.665774+0000","last_became_active":"2024-09-13T07:02:23.331309+0000","last_became_peered":"2024-09-13T07:02:23.331309+0000","last_unstale":"2024-09-13T07:02:30.665774+0000","last_undegraded":"2024-09-13T07:02:30.665774+0000","last_fullsized":"2024-09-13T07:02:30.665774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:42:20.241073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001799911,"stat_sum":{"num_bytes":19676021,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":8,"num_read":5429,"num_read_kb":10250,"num_write":462,"num_write_kb":7012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":104677,"num_keys_recovered":34,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"86'206","reported_seq":5419,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:28.789216+0000","last_change":"2024-09-13T07:02:23.332539+0000","last_active":"2024-09-13T07:02:28.789216+0000","last_peered":"2024-09-13T07:02:28.789216+0000","last_clean":"2024-09-13T07:02:28.789216+0000","last_became_active":"2024-09-13T07:02:23.331645+0000","last_became_peered":"2024-09-13T07:02:23.331645+0000","last_unstale":"2024-09-13T07:02:28.789216+0000","last_undegraded":"2024-09-13T07:02:28.789216+0000","last_fullsized":"2024-09-13T07:02:28.789216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:26.278406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055727200000000004,"stat_sum":{"num_bytes":32331658,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":13,"num_read":5446,"num_read_kb":8235,"num_write":273,"num_write_kb":12204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":1288675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"87'203","reported_seq":4583,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:29.696525+0000","last_change":"2024-09-13T07:02:26.369654+0000","last_active":"2024-09-13T07:02:29.696525+0000","last_peered":"2024-09-13T07:02:29.696525+0000","last_clean":"2024-09-13T07:02:29.696525+0000","last_became_active":"2024-09-13T07:02:26.367900+0000","last_became_peered":"2024-09-13T07:02:26.367900+0000","last_unstale":"2024-09-13T07:02:29.696525+0000","last_undegraded":"2024-09-13T07:02:29.696525+0000","last_fullsized":"2024-09-13T07:02:29.696525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:26:02.617503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082396300000000004,"stat_sum":{"num_bytes":34114128,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":5824,"num_read_kb":13155,"num_write":268,"num_write_kb":8380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":4637802,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'366","reported_seq":9250,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:29.986624+0000","last_change":"2024-09-13T07:01:50.079211+0000","last_active":"2024-09-13T07:02:29.986624+0000","last_peered":"2024-09-13T07:02:29.986624+0000","last_clean":"2024-09-13T07:02:29.986624+0000","last_became_active":"2024-09-13T07:01:50.078679+0000","last_became_peered":"2024-09-13T07:01:50.078679+0000","last_unstale":"2024-09-13T07:02:29.986624+0000","last_undegraded":"2024-09-13T07:02:29.986624+0000","last_fullsized":"2024-09-13T07:02:29.986624+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:30:15.075968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001182647,"stat_sum":{"num_bytes":32285444,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":10477,"num_read_kb":21789,"num_write":511,"num_write_kb":14096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2756020,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"87'255","reported_seq":10354,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:29.811750+0000","last_change":"2024-09-13T07:01:21.293012+0000","last_active":"2024-09-13T07:02:29.811750+0000","last_peered":"2024-09-13T07:02:29.811750+0000","last_clean":"2024-09-13T07:02:29.811750+0000","last_became_active":"2024-09-13T07:01:19.277380+0000","last_became_peered":"2024-09-13T07:01:19.277380+0000","last_unstale":"2024-09-13T07:02:29.811750+0000","last_undegraded":"2024-09-13T07:02:29.811750+0000","last_fullsized":"2024-09-13T07:02:29.811750+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:05:57.870409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010463009999999999,"stat_sum":{"num_bytes":32272049,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":8,"num_read":10762,"num_read_kb":15462,"num_write":400,"num_write_kb":4019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"86'123","reported_seq":644,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:28.795868+0000","last_change":"2024-09-13T07:01:21.314353+0000","last_active":"2024-09-13T07:02:28.795868+0000","last_peered":"2024-09-13T07:02:28.795868+0000","last_clean":"2024-09-13T07:02:28.795868+0000","last_became_active":"2024-09-13T07:01:19.280203+0000","last_became_peered":"2024-09-13T07:01:19.280203+0000","last_unstale":"2024-09-13T07:02:28.795868+0000","last_undegraded":"2024-09-13T07:02:28.795868+0000","last_fullsized":"2024-09-13T07:02:28.795868+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:38.598517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024699230000000002,"stat_sum":{"num_bytes":16305609,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":1107,"num_read_kb":967,"num_write":125,"num_write_kb":477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"88'240","reported_seq":1491,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.615688+0000","last_change":"2024-09-13T07:02:09.224417+0000","last_active":"2024-09-13T07:02:30.615688+0000","last_peered":"2024-09-13T07:02:30.615688+0000","last_clean":"2024-09-13T07:02:30.615688+0000","last_became_active":"2024-09-13T07:02:09.223575+0000","last_became_peered":"2024-09-13T07:02:09.223575+0000","last_unstale":"2024-09-13T07:02:30.615688+0000","last_undegraded":"2024-09-13T07:02:30.615688+0000","last_fullsized":"2024-09-13T07:02:30.615688+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:57:31.053474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001937825,"stat_sum":{"num_bytes":42325723,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":21,"num_read":1641,"num_read_kb":12233,"num_write":296,"num_write_kb":11644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":4572463,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":83,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:28.484833+0000","last_change":"2024-09-13T07:01:11.221939+0000","last_active":"2024-09-13T07:02:28.484833+0000","last_peered":"2024-09-13T07:02:28.484833+0000","last_clean":"2024-09-13T07:02:28.484833+0000","last_became_active":"2024-09-13T07:01:09.212302+0000","last_became_peered":"2024-09-13T07:01:09.212302+0000","last_unstale":"2024-09-13T07:02:28.484833+0000","last_undegraded":"2024-09-13T07:02:28.484833+0000","last_fullsized":"2024-09-13T07:02:28.484833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:49.220240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"87'94","reported_seq":414,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:29.481663+0000","last_change":"2024-09-13T07:02:20.373122+0000","last_active":"2024-09-13T07:02:29.481663+0000","last_peered":"2024-09-13T07:02:29.481663+0000","last_clean":"2024-09-13T07:02:29.481663+0000","last_became_active":"2024-09-13T07:02:20.311230+0000","last_became_peered":"2024-09-13T07:02:20.311230+0000","last_unstale":"2024-09-13T07:02:29.481663+0000","last_undegraded":"2024-09-13T07:02:29.481663+0000","last_fullsized":"2024-09-13T07:02:29.481663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:07:45.257392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038364399999999999,"stat_sum":{"num_bytes":15874899,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":193,"num_read_kb":3922,"num_write":74,"num_write_kb":2188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":268161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:28.484826+0000","last_change":"2024-09-13T07:02:20.313654+0000","last_active":"2024-09-13T07:02:28.484826+0000","last_peered":"2024-09-13T07:02:28.484826+0000","last_clean":"2024-09-13T07:02:28.484826+0000","last_became_active":"2024-09-13T07:02:20.313362+0000","last_became_peered":"2024-09-13T07:02:20.313362+0000","last_unstale":"2024-09-13T07:02:28.484826+0000","last_undegraded":"2024-09-13T07:02:28.484826+0000","last_fullsized":"2024-09-13T07:02:28.484826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:58.845083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"87'239","reported_seq":2397,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:29.696737+0000","last_change":"2024-09-13T07:02:26.368320+0000","last_active":"2024-09-13T07:02:29.696737+0000","last_peered":"2024-09-13T07:02:29.696737+0000","last_clean":"2024-09-13T07:02:29.696737+0000","last_became_active":"2024-09-13T07:02:26.367717+0000","last_became_peered":"2024-09-13T07:02:26.367717+0000","last_unstale":"2024-09-13T07:02:29.696737+0000","last_undegraded":"2024-09-13T07:02:29.696737+0000","last_fullsized":"2024-09-13T07:02:29.696737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:36:49.852981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016908909999999999,"stat_sum":{"num_bytes":65699784,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":2598,"num_read_kb":20047,"num_write":305,"num_write_kb":4890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":92,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:28.398277+0000","last_change":"2024-09-13T07:01:47.489709+0000","last_active":"2024-09-13T07:02:28.398277+0000","last_peered":"2024-09-13T07:02:28.398277+0000","last_clean":"2024-09-13T07:02:28.398277+0000","last_became_active":"2024-09-13T07:01:47.489419+0000","last_became_peered":"2024-09-13T07:01:47.489419+0000","last_unstale":"2024-09-13T07:02:28.398277+0000","last_undegraded":"2024-09-13T07:02:28.398277+0000","last_fullsized":"2024-09-13T07:02:28.398277+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:41.389847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"87'177","reported_seq":685,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:29.812317+0000","last_change":"2024-09-13T07:02:23.331277+0000","last_active":"2024-09-13T07:02:29.812317+0000","last_peered":"2024-09-13T07:02:29.812317+0000","last_clean":"2024-09-13T07:02:29.812317+0000","last_became_active":"2024-09-13T07:02:23.330495+0000","last_became_peered":"2024-09-13T07:02:23.330495+0000","last_unstale":"2024-09-13T07:02:29.812317+0000","last_undegraded":"2024-09-13T07:02:29.812317+0000","last_fullsized":"2024-09-13T07:02:29.812317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:53.653902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085579199999999997,"stat_sum":{"num_bytes":38208885,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":22,"num_read":580,"num_read_kb":3778,"num_write":232,"num_write_kb":4058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":156677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":65,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:28.398101+0000","last_change":"2024-09-13T07:02:20.309702+0000","last_active":"2024-09-13T07:02:28.398101+0000","last_peered":"2024-09-13T07:02:28.398101+0000","last_clean":"2024-09-13T07:02:28.398101+0000","last_became_active":"2024-09-13T07:02:20.309401+0000","last_became_peered":"2024-09-13T07:02:20.309401+0000","last_unstale":"2024-09-13T07:02:28.398101+0000","last_undegraded":"2024-09-13T07:02:28.398101+0000","last_fullsized":"2024-09-13T07:02:28.398101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:36:08.303855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.5","version":"88'210","reported_seq":1337,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.403615+0000","last_change":"2024-09-13T07:01:21.293697+0000","last_active":"2024-09-13T07:02:30.403615+0000","last_peered":"2024-09-13T07:02:30.403615+0000","last_clean":"2024-09-13T07:02:30.403615+0000","last_became_active":"2024-09-13T07:01:19.279719+0000","last_became_peered":"2024-09-13T07:01:19.279719+0000","last_unstale":"2024-09-13T07:02:30.403615+0000","last_undegraded":"2024-09-13T07:02:30.403615+0000","last_fullsized":"2024-09-13T07:02:30.403615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:05:06.593984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085976499999999997,"stat_sum":{"num_bytes":39746647,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":1681,"num_read_kb":6041,"num_write":309,"num_write_kb":2563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"88'207","reported_seq":1505,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.407078+0000","last_change":"2024-09-13T07:01:47.540796+0000","last_active":"2024-09-13T07:02:30.407078+0000","last_peered":"2024-09-13T07:02:30.407078+0000","last_clean":"2024-09-13T07:02:30.407078+0000","last_became_active":"2024-09-13T07:01:47.491983+0000","last_became_peered":"2024-09-13T07:01:47.491983+0000","last_unstale":"2024-09-13T07:02:30.407078+0000","last_undegraded":"2024-09-13T07:02:30.407078+0000","last_fullsized":"2024-09-13T07:02:30.407078+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:18.382486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017265959999999999,"stat_sum":{"num_bytes":23197264,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":1826,"num_read_kb":18144,"num_write":313,"num_write_kb":8229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":3083106,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:28.387690+0000","last_change":"2024-09-13T07:02:20.315421+0000","last_active":"2024-09-13T07:02:28.387690+0000","last_peered":"2024-09-13T07:02:28.387690+0000","last_clean":"2024-09-13T07:02:28.387690+0000","last_became_active":"2024-09-13T07:02:20.312417+0000","last_became_peered":"2024-09-13T07:02:20.312417+0000","last_unstale":"2024-09-13T07:02:28.387690+0000","last_undegraded":"2024-09-13T07:02:28.387690+0000","last_fullsized":"2024-09-13T07:02:28.387690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:48.435411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"85'199","reported_seq":10935,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:29.171674+0000","last_change":"2024-09-13T07:02:23.332259+0000","last_active":"2024-09-13T07:02:29.171674+0000","last_peered":"2024-09-13T07:02:29.171674+0000","last_clean":"2024-09-13T07:02:29.171674+0000","last_became_active":"2024-09-13T07:02:23.331157+0000","last_became_peered":"2024-09-13T07:02:23.331157+0000","last_unstale":"2024-09-13T07:02:29.171674+0000","last_undegraded":"2024-09-13T07:02:29.171674+0000","last_fullsized":"2024-09-13T07:02:29.171674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:42:56.655972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065804299999999997,"stat_sum":{"num_bytes":47723028,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":17,"num_read":10806,"num_read_kb":18166,"num_write":222,"num_write_kb":14877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3447375,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T07:02:27.537506+0000","last_change":"2024-09-13T07:02:26.370215+0000","last_active":"2024-09-13T07:02:27.537506+0000","last_peered":"2024-09-13T07:02:27.537506+0000","last_clean":"2024-09-13T07:02:27.537506+0000","last_became_active":"2024-09-13T07:02:26.369577+0000","last_became_peered":"2024-09-13T07:02:26.369577+0000","last_unstale":"2024-09-13T07:02:27.537506+0000","last_undegraded":"2024-09-13T07:02:27.537506+0000","last_fullsized":"2024-09-13T07:02:27.537506+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:16:39.536510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"85'116","reported_seq":2441,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:29.094324+0000","last_change":"2024-09-13T07:02:16.282550+0000","last_active":"2024-09-13T07:02:29.094324+0000","last_peered":"2024-09-13T07:02:29.094324+0000","last_clean":"2024-09-13T07:02:29.094324+0000","last_became_active":"2024-09-13T07:02:16.282010+0000","last_became_peered":"2024-09-13T07:02:16.282010+0000","last_unstale":"2024-09-13T07:02:29.094324+0000","last_undegraded":"2024-09-13T07:02:29.094324+0000","last_fullsized":"2024-09-13T07:02:29.094324+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:07:01.767243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044811299999999998,"stat_sum":{"num_bytes":25598265,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":2757,"num_read_kb":10840,"num_write":134,"num_write_kb":1967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":240544,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:28.397714+0000","last_change":"2024-09-13T07:02:20.307961+0000","last_active":"2024-09-13T07:02:28.397714+0000","last_peered":"2024-09-13T07:02:28.397714+0000","last_clean":"2024-09-13T07:02:28.397714+0000","last_became_active":"2024-09-13T07:02:20.307607+0000","last_became_peered":"2024-09-13T07:02:20.307607+0000","last_unstale":"2024-09-13T07:02:28.397714+0000","last_undegraded":"2024-09-13T07:02:28.397714+0000","last_fullsized":"2024-09-13T07:02:28.397714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:05.765371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"85'206","reported_seq":2202,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:29.125678+0000","last_change":"2024-09-13T07:02:26.368592+0000","last_active":"2024-09-13T07:02:29.125678+0000","last_peered":"2024-09-13T07:02:29.125678+0000","last_clean":"2024-09-13T07:02:29.125678+0000","last_became_active":"2024-09-13T07:02:26.367948+0000","last_became_peered":"2024-09-13T07:02:26.367948+0000","last_unstale":"2024-09-13T07:02:29.125678+0000","last_undegraded":"2024-09-13T07:02:29.125678+0000","last_fullsized":"2024-09-13T07:02:29.125678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:37:10.553378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064294199999999997,"stat_sum":{"num_bytes":49651628,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":3780,"num_read_kb":12511,"num_write":227,"num_write_kb":14025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:28.387750+0000","last_change":"2024-09-13T07:01:41.443501+0000","last_active":"2024-09-13T07:02:28.387750+0000","last_peered":"2024-09-13T07:02:28.387750+0000","last_clean":"2024-09-13T07:02:28.387750+0000","last_became_active":"2024-09-13T07:01:41.436739+0000","last_became_peered":"2024-09-13T07:01:41.436739+0000","last_unstale":"2024-09-13T07:02:28.387750+0000","last_undegraded":"2024-09-13T07:02:28.387750+0000","last_fullsized":"2024-09-13T07:02:28.387750+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:33:04.362119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":253,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:28.397900+0000","last_change":"2024-09-13T07:01:50.080564+0000","last_active":"2024-09-13T07:02:28.397900+0000","last_peered":"2024-09-13T07:02:28.397900+0000","last_clean":"2024-09-13T07:02:28.397900+0000","last_became_active":"2024-09-13T07:01:50.080072+0000","last_became_peered":"2024-09-13T07:01:50.080072+0000","last_unstale":"2024-09-13T07:02:28.397900+0000","last_undegraded":"2024-09-13T07:02:28.397900+0000","last_fullsized":"2024-09-13T07:02:28.397900+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:14:58.662626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"88'323","reported_seq":1734,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.591037+0000","last_change":"2024-09-13T07:02:23.331997+0000","last_active":"2024-09-13T07:02:30.591037+0000","last_peered":"2024-09-13T07:02:30.591037+0000","last_clean":"2024-09-13T07:02:30.591037+0000","last_became_active":"2024-09-13T07:02:23.331206+0000","last_became_peered":"2024-09-13T07:02:23.331206+0000","last_unstale":"2024-09-13T07:02:30.591037+0000","last_undegraded":"2024-09-13T07:02:30.591037+0000","last_fullsized":"2024-09-13T07:02:30.591037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:30:37.551347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001264996,"stat_sum":{"num_bytes":19868378,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":1905,"num_read_kb":11614,"num_write":492,"num_write_kb":3451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":861185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"88'146","reported_seq":1293,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.681657+0000","last_change":"2024-09-13T07:01:41.467037+0000","last_active":"2024-09-13T07:02:30.681657+0000","last_peered":"2024-09-13T07:02:30.681657+0000","last_clean":"2024-09-13T07:02:30.681657+0000","last_became_active":"2024-09-13T07:01:41.438743+0000","last_became_peered":"2024-09-13T07:01:41.438743+0000","last_unstale":"2024-09-13T07:02:30.681657+0000","last_undegraded":"2024-09-13T07:02:30.681657+0000","last_fullsized":"2024-09-13T07:02:30.681657+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:11:01.568164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022509729999999999,"stat_sum":{"num_bytes":21996551,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":13,"num_read":1488,"num_read_kb":14735,"num_write":211,"num_write_kb":15866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":26238,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'409","reported_seq":2490,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.529262+0000","last_change":"2024-09-13T07:01:22.025365+0000","last_active":"2024-09-13T07:02:30.529262+0000","last_peered":"2024-09-13T07:02:30.529262+0000","last_clean":"2024-09-13T07:02:30.529262+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:02:30.529262+0000","last_undegraded":"2024-09-13T07:02:30.529262+0000","last_fullsized":"2024-09-13T07:02:30.529262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:11.537644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064478199999999999,"stat_sum":{"num_bytes":41981722,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":13,"num_read":3272,"num_read_kb":21064,"num_write":616,"num_write_kb":17227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"88'212","reported_seq":4508,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.667596+0000","last_change":"2024-09-13T07:02:26.369574+0000","last_active":"2024-09-13T07:02:30.667596+0000","last_peered":"2024-09-13T07:02:30.667596+0000","last_clean":"2024-09-13T07:02:30.667596+0000","last_became_active":"2024-09-13T07:02:26.368387+0000","last_became_peered":"2024-09-13T07:02:26.368387+0000","last_unstale":"2024-09-13T07:02:30.667596+0000","last_undegraded":"2024-09-13T07:02:30.667596+0000","last_fullsized":"2024-09-13T07:02:30.667596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:23:15.792514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001533932,"stat_sum":{"num_bytes":44113981,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":5623,"num_read_kb":14291,"num_write":249,"num_write_kb":6851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1474279,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"88'337","reported_seq":8775,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.570449+0000","last_change":"2024-09-13T07:01:21.292633+0000","last_active":"2024-09-13T07:02:30.570449+0000","last_peered":"2024-09-13T07:02:30.570449+0000","last_clean":"2024-09-13T07:02:30.570449+0000","last_became_active":"2024-09-13T07:01:19.279813+0000","last_became_peered":"2024-09-13T07:01:19.279813+0000","last_unstale":"2024-09-13T07:02:30.570449+0000","last_undegraded":"2024-09-13T07:02:30.570449+0000","last_fullsized":"2024-09-13T07:02:30.570449+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:39.697691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072600099999999999,"stat_sum":{"num_bytes":60079740,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":8,"num_read":10576,"num_read_kb":29105,"num_write":407,"num_write_kb":19710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"88'186","reported_seq":1697,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.403022+0000","last_change":"2024-09-13T07:01:21.313310+0000","last_active":"2024-09-13T07:02:30.403022+0000","last_peered":"2024-09-13T07:02:30.403022+0000","last_clean":"2024-09-13T07:02:30.403022+0000","last_became_active":"2024-09-13T07:01:19.282520+0000","last_became_peered":"2024-09-13T07:01:19.282520+0000","last_unstale":"2024-09-13T07:02:30.403022+0000","last_undegraded":"2024-09-13T07:02:30.403022+0000","last_fullsized":"2024-09-13T07:02:30.403022+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:41:44.801487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018248170000000001,"stat_sum":{"num_bytes":39708328,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":18,"num_read":1821,"num_read_kb":7123,"num_write":177,"num_write_kb":9563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"87'117","reported_seq":1516,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:29.811977+0000","last_change":"2024-09-13T07:01:21.313674+0000","last_active":"2024-09-13T07:02:29.811977+0000","last_peered":"2024-09-13T07:02:29.811977+0000","last_clean":"2024-09-13T07:02:29.811977+0000","last_became_active":"2024-09-13T07:01:19.281215+0000","last_became_peered":"2024-09-13T07:01:19.281215+0000","last_unstale":"2024-09-13T07:02:29.811977+0000","last_undegraded":"2024-09-13T07:02:29.811977+0000","last_fullsized":"2024-09-13T07:02:29.811977+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:22.161596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019878309999999998,"stat_sum":{"num_bytes":32259101,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":15,"num_read":1384,"num_read_kb":13752,"num_write":174,"num_write_kb":8538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"88'358","reported_seq":1584,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.656920+0000","last_change":"2024-09-13T07:01:21.314048+0000","last_active":"2024-09-13T07:02:30.656920+0000","last_peered":"2024-09-13T07:02:30.656920+0000","last_clean":"2024-09-13T07:02:30.656920+0000","last_became_active":"2024-09-13T07:01:19.281169+0000","last_became_peered":"2024-09-13T07:01:19.281169+0000","last_unstale":"2024-09-13T07:02:30.656920+0000","last_undegraded":"2024-09-13T07:02:30.656920+0000","last_fullsized":"2024-09-13T07:02:30.656920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:49:46.525500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023438180000000001,"stat_sum":{"num_bytes":36724400,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":2445,"num_read_kb":12750,"num_write":583,"num_write_kb":22781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"86'107","reported_seq":1320,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:28.796595+0000","last_change":"2024-09-13T07:02:12.324979+0000","last_active":"2024-09-13T07:02:28.796595+0000","last_peered":"2024-09-13T07:02:28.796595+0000","last_clean":"2024-09-13T07:02:28.796595+0000","last_became_active":"2024-09-13T07:02:12.247568+0000","last_became_peered":"2024-09-13T07:02:12.247568+0000","last_unstale":"2024-09-13T07:02:28.796595+0000","last_undegraded":"2024-09-13T07:02:28.796595+0000","last_fullsized":"2024-09-13T07:02:28.796595+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:04:50.472141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061928799999999996,"stat_sum":{"num_bytes":22467993,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":6,"num_read":1085,"num_read_kb":4351,"num_write":127,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":4419876,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"88'159","reported_seq":5308,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.485493+0000","last_change":"2024-09-13T07:01:21.313825+0000","last_active":"2024-09-13T07:02:30.485493+0000","last_peered":"2024-09-13T07:02:30.485493+0000","last_clean":"2024-09-13T07:02:30.485493+0000","last_became_active":"2024-09-13T07:01:19.279714+0000","last_became_peered":"2024-09-13T07:01:19.279714+0000","last_unstale":"2024-09-13T07:02:30.485493+0000","last_undegraded":"2024-09-13T07:02:30.485493+0000","last_fullsized":"2024-09-13T07:02:30.485493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:06:09.717433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002154132,"stat_sum":{"num_bytes":45943431,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":5733,"num_read_kb":9129,"num_write":200,"num_write_kb":10882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"85'203","reported_seq":6322,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-13T07:02:28.794957+0000","last_change":"2024-09-13T07:01:50.078398+0000","last_active":"2024-09-13T07:02:28.794957+0000","last_peered":"2024-09-13T07:02:28.794957+0000","last_clean":"2024-09-13T07:02:28.794957+0000","last_became_active":"2024-09-13T07:01:50.078015+0000","last_became_peered":"2024-09-13T07:01:50.078015+0000","last_unstale":"2024-09-13T07:02:28.794957+0000","last_undegraded":"2024-09-13T07:02:28.794957+0000","last_fullsized":"2024-09-13T07:02:28.794957+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:52.203434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060210699999999995,"stat_sum":{"num_bytes":54920284,"num_objects":32,"num_object_clones":10,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":9,"num_read":6497,"num_read_kb":25571,"num_write":301,"num_write_kb":7345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":60382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"88'205","reported_seq":1815,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T07:02:30.600627+0000","last_change":"2024-09-13T07:01:21.313226+0000","last_active":"2024-09-13T07:02:30.600627+0000","last_peered":"2024-09-13T07:02:30.600627+0000","last_clean":"2024-09-13T07:02:30.600627+0000","last_became_active":"2024-09-13T07:01:19.279419+0000","last_became_peered":"2024-09-13T07:01:19.279419+0000","last_unstale":"2024-09-13T07:02:30.600627+0000","last_undegraded":"2024-09-13T07:02:30.600627+0000","last_fullsized":"2024-09-13T07:02:30.600627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:46:29.817757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001803165,"stat_sum":{"num_bytes":34776760,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":2229,"num_read_kb":16725,"num_write":271,"num_write_kb":16379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1187714367,"num_objects":1115,"num_object_clones":266,"num_object_copies":2230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1115,"num_whiteouts":432,"num_read":125020,"num_read_kb":437549,"num_write":9001,"num_write_kb":279429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":689,"num_bytes_recovered":28299441,"num_keys_recovered":153,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":276389888,"data_stored":317973417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6729,"ondisk_log_size":6729,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104272,"kb_used_data":64964,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94267568,"statfs":{"total":96636764160,"available":96529989632,"internally_reserved":0,"allocated":66523136,"data_stored":76263289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.85599999999999998}]},{"osd":3,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.93200000000000005}]},{"osd":5,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.72699999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.878},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62812,"kb_used_data":31692,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94309028,"statfs":{"total":96636764160,"available":96572444672,"internally_reserved":0,"allocated":32452608,"data_stored":37888020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.88100000000000001}]},{"osd":1,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.64400000000000002}]},{"osd":2,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.02},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.0429999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.0660000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.0029999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.97099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76268,"kb_used_data":41060,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94295572,"statfs":{"total":96636764160,"available":96558665728,"internally_reserved":0,"allocated":42045440,"data_stored":46188548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.90800000000000003}]},{"osd":1,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.1449999999999996,"5min":6.1449999999999996,"15min":6.1449999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.78300000000000003}]},{"osd":2,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":6.2030000000000003,"5min":6.2030000000000003,"15min":6.2030000000000003},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":6.1100000000000003,"5min":6.1100000000000003,"15min":6.1100000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.88600000000000001}]},{"osd":7,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.872},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":6.1749999999999998,"5min":6.1749999999999998,"15min":6.1749999999999998},"last":0.71999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":110032,"kb_used_data":70720,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94261808,"statfs":{"total":96636764160,"available":96524091392,"internally_reserved":0,"allocated":72417280,"data_stored":82460131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.752}]},{"osd":2,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.81599999999999995}]},{"osd":3,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.78900000000000003}]},{"osd":5,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.82999999999999996}]},{"osd":6,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.68600000000000005}]},{"osd":7,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.877},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.72799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86788,"kb_used_data":47480,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94285052,"statfs":{"total":96636764160,"available":96547893248,"internally_reserved":0,"allocated":48619520,"data_stored":54690135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.86599999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.746}]},{"osd":4,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.0840000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.93999999999999995}]},{"osd":6,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.024},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.1359999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.998},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.1060000000000001}]}]},{"osd":2,"up_from":0,"seq":55834574869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36220,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335620,"statfs":{"total":96636764160,"available":96599674880,"internally_reserved":0,"allocated":1028096,"data_stored":469314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.625}]},{"osd":4,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.76400000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.92300000000000004}]},{"osd":7,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.046}]}]},{"osd":0,"up_from":0,"seq":55834574869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9293824,"data_stored":10365051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9109504,"data_stored":11024733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2158592,"data_stored":2536533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":47579136,"data_stored":54214678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":71385088,"data_stored":81984884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":41123840,"data_stored":45815348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":30826496,"data_stored":36828319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":64913408,"data_stored":75203871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:02:36.346 INFO:tasks.thrashosds.thrasher:Setting 2.5 to [6, 4] 2024-09-13T07:02:36.346 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.5', '6', '4'] 2024-09-13T07:02:36.346 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.5 6 4 2024-09-13T07:02:36.353 INFO:teuthology.orchestra.run.smithi118.stdout:2835 trunc from 0xd42dba7 to 0x7e15d0f 2024-09-13T07:02:36.377 INFO:teuthology.orchestra.run.smithi118.stdout:2836 read 0x2624780 thru 0x262ec4c (0xa4cd bytes) 2024-09-13T07:02:36.378 INFO:teuthology.orchestra.run.smithi118.stdout:2837 write 0x5dec18e thru 0x5df89db (0xc84e bytes) 2024-09-13T07:02:36.399 INFO:teuthology.orchestra.run.smithi118.stdout:2838 writesame 0x91181e8 thru 0x9119003 data_size 0x2a(0xe1c bytes) 2024-09-13T07:02:36.399 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.411 INFO:teuthology.orchestra.run.smithi118.stdout:2840 trunc from 0x9119004 to 0xecc42e4 2024-09-13T07:02:36.432 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:02:36.437 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:36.436+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:36.438 INFO:teuthology.orchestra.run.smithi118.stdout:2841 trunc from 0xecc42e4 to 0x9a36e3e 2024-09-13T07:02:36.457 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.457 INFO:teuthology.orchestra.run.smithi118.stdout:2843 punch from 0x394bcdb to 0x394f83a, (0x3b5f bytes) 2024-09-13T07:02:36.457 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.457 INFO:teuthology.orchestra.run.smithi118.stdout:2846 read 0x8b1c836 thru 0x8b2304a (0x6815 bytes) 2024-09-13T07:02:36.457 INFO:teuthology.orchestra.run.smithi118.stdout:2847 trunc from 0x9a36e3e to 0xa03c544 2024-09-13T07:02:36.461 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:36.461 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:36.461 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:36.461 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:36.467 INFO:teuthology.orchestra.run.smithi118.stdout:2849 writesame 0x504f54 thru 0x50bb3b data_size 0xc(0x6be8 bytes) 2024-09-13T07:02:36.470 INFO:teuthology.orchestra.run.smithi118.stdout:2850 read 0x815e475 thru 0x816cb33 (0xe6bf bytes) 2024-09-13T07:02:36.473 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:36.474 INFO:teuthology.orchestra.run.smithi118.stdout:2852 trunc from 0xa03c544 to 0x7ea3ab8 2024-09-13T07:02:36.487 INFO:teuthology.orchestra.run.smithi118.stdout:2853 writesame 0x436dd9d thru 0x4371ec1 data_size 0x33(0x4125 bytes) 2024-09-13T07:02:36.493 INFO:teuthology.orchestra.run.smithi118.stdout:2854 punch from 0x2b90a4d to 0x2b988a4, (0x7e57 bytes) 2024-09-13T07:02:36.493 INFO:teuthology.orchestra.run.smithi118.stdout:2855 read 0x45af21e thru 0x45b65eb (0x73ce bytes) 2024-09-13T07:02:36.514 INFO:teuthology.orchestra.run.smithi118.stdout:2856 write 0xd4223f9 thru 0xd423085 (0xc8d bytes) 2024-09-13T07:02:36.520 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.520 INFO:teuthology.orchestra.run.smithi118.stdout:2858 writesame 0x95124ee thru 0x9519792 data_size 0x9(0x72a5 bytes) 2024-09-13T07:02:36.525 INFO:teuthology.orchestra.run.smithi118.stdout:2859 writesame 0x1f5f334 thru 0x1f65100 data_size 0x25(0x5dcd bytes) 2024-09-13T07:02:36.528 INFO:teuthology.orchestra.run.smithi118.stdout:2860 writesame 0x549a0a thru 0x54fdc8 data_size 0x1(0x63bf bytes) 2024-09-13T07:02:36.538 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:36.536+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:36.551 INFO:teuthology.orchestra.run.smithi118.stdout:2864 read 0x7bbe797 thru 0x7bcc7a0 (0xe00a bytes) 2024-09-13T07:02:36.552 INFO:teuthology.orchestra.run.smithi118.stdout:2865 trunc from 0xd423086 to 0xe09aafa 2024-09-13T07:02:36.562 INFO:teuthology.orchestra.run.smithi118.stdout:2866 read 0xce822c3 thru 0xce853f2 (0x3130 bytes) 2024-09-13T07:02:36.562 INFO:teuthology.orchestra.run.smithi118.stdout:2867 write 0x6a51d9b thru 0x6a5f83b (0xdaa1 bytes) 2024-09-13T07:02:36.567 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.567 INFO:teuthology.orchestra.run.smithi118.stdout:2869 read 0xdb2e81 thru 0xdbb6f8 (0x8878 bytes) 2024-09-13T07:02:36.568 INFO:teuthology.orchestra.run.smithi118.stdout:2870 write 0x5786b00 thru 0x578efda (0x84db bytes) 2024-09-13T07:02:36.574 INFO:teuthology.orchestra.run.smithi118.stdout:2871 read 0xc47a689 thru 0xc47d0cf (0x2a47 bytes) 2024-09-13T07:02:36.574 INFO:teuthology.orchestra.run.smithi118.stdout:2872 read 0xb9ff96c thru 0xba0439d (0x4a32 bytes) 2024-09-13T07:02:36.574 INFO:teuthology.orchestra.run.smithi118.stdout:2873 read 0x6fffe thru 0x74f61 (0x4f64 bytes) 2024-09-13T07:02:36.574 INFO:teuthology.orchestra.run.smithi118.stdout:2874 punch from 0x4d92fd3 to 0x4da0bf2, (0xdc1f bytes) 2024-09-13T07:02:36.575 INFO:teuthology.orchestra.run.smithi118.stdout:2876 writesame 0x2912d72 thru 0x291bd2a data_size 0x1(0x8fb9 bytes) 2024-09-13T07:02:36.581 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:36.582 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:36.591 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:36.618 INFO:teuthology.orchestra.run.smithi118.stdout:2877 trunc from 0xe09aafa to 0x51d6f01 2024-09-13T07:02:36.639 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:36.638+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:36.640 INFO:teuthology.orchestra.run.smithi118.stdout:2881 write 0x1f5599f thru 0x1f63411 (0xda73 bytes) 2024-09-13T07:02:36.650 INFO:teuthology.orchestra.run.smithi118.stdout:2885 trunc from 0x51d6f01 to 0x9cec5bd 2024-09-13T07:02:36.662 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.662 INFO:teuthology.orchestra.run.smithi118.stdout:2888 read 0x1d65656 thru 0x1d73d17 (0xe6c2 bytes) 2024-09-13T07:02:36.663 INFO:teuthology.orchestra.run.smithi118.stdout:2889 punch from 0x490803d to 0x4916569, (0xe52c bytes) 2024-09-13T07:02:36.663 INFO:teuthology.orchestra.run.smithi118.stdout:2890 read 0x96f1cb thru 0x971315 (0x214b bytes) 2024-09-13T07:02:36.679 INFO:teuthology.orchestra.run.smithi118.stdout:2891 write 0xc123f51 thru 0xc130aec (0xcb9c bytes) 2024-09-13T07:02:36.686 INFO:teuthology.orchestra.run.smithi118.stdout:2892 read 0x624a2ba thru 0x62581fa (0xdf41 bytes) 2024-09-13T07:02:36.686 INFO:teuthology.orchestra.run.smithi118.stdout:2894 read 0x5ed553a thru 0x5ee3257 (0xdd1e bytes) 2024-09-13T07:02:36.687 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:36.690 INFO:teuthology.orchestra.run.smithi118.stdout:2897 trunc from 0xc130aed to 0xe14eafc 2024-09-13T07:02:36.700 INFO:teuthology.orchestra.run.smithi118.stdout:2899 punch from 0x7b91f to 0x8b126, (0xf807 bytes) 2024-09-13T07:02:36.704 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.64397013, 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:02:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:02:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:02:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:02:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:02:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:02:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:02:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:02:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:02:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:02:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:02:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:02:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:02:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:02:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:02:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:02:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93974 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2260992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616547+0000", 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086344155000001, 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041570436000000002, 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93974 2024-09-13T07:02:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616547+0000", 2024-09-13T07:02:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616547+0000", 2024-09-13T07:02:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616546+0000", 2024-09-13T07:02:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616549+0000", 2024-09-13T07:02:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2610000000000002e-06 2024-09-13T07:02:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616558+0000", 2024-09-13T07:02:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.67e-06 2024-09-13T07:02:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616565+0000", 2024-09-13T07:02:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4930000000000003e-06 2024-09-13T07:02:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.657979+0000", 2024-09-13T07:02:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041414765999999999 2024-09-13T07:02:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658048+0000", 2024-09-13T07:02:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9003000000000003e-05 2024-09-13T07:02:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658117+0000", 2024-09-13T07:02:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8752999999999997e-05 2024-09-13T07:02:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93975 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2375680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616571+0000", 2024-09-13T07:02:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086319733, 2024-09-13T07:02:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041711005000000002, 2024-09-13T07:02:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93975 2024-09-13T07:02:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616571+0000", 2024-09-13T07:02:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616571+0000", 2024-09-13T07:02:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616571+0000", 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616575+0000", 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0069999999999999e-06 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616583+0000", 2024-09-13T07:02:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1960000000000008e-06 2024-09-13T07:02:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616590+0000", 2024-09-13T07:02:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8419999999999999e-06 2024-09-13T07:02:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658142+0000", 2024-09-13T07:02:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041552736999999999 2024-09-13T07:02:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658215+0000", 2024-09-13T07:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2279000000000004e-05 2024-09-13T07:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658282+0000", 2024-09-13T07:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7471999999999994e-05 2024-09-13T07:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93976 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616595+0000", 2024-09-13T07:02:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086295572999999, 2024-09-13T07:02:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041851514999999999, 2024-09-13T07:02:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93976 2024-09-13T07:02:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616595+0000", 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616595+0000", 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616595+0000", 2024-09-13T07:02:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616599+0000", 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9079999999999997e-06 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616607+0000", 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0579999999999998e-06 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616613+0000", 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7449999999999998e-06 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658307+0000", 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041693725000000001 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658379+0000", 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2268000000000007e-05 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658447+0000", 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7327000000000001e-05 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93977 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616621+0000", 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086270026999999, 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041990496000000002, 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93977 2024-09-13T07:02:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616621+0000", 2024-09-13T07:02:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616621+0000", 2024-09-13T07:02:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616620+0000", 2024-09-13T07:02:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616624+0000", 2024-09-13T07:02:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7620000000000001e-06 2024-09-13T07:02:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616632+0000", 2024-09-13T07:02:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2910000000000007e-06 2024-09-13T07:02:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616640+0000", 2024-09-13T07:02:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.469e-06 2024-09-13T07:02:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658472+0000", 2024-09-13T07:02:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041832326000000003 2024-09-13T07:02:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658544+0000", 2024-09-13T07:02:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1957000000000002e-05 2024-09-13T07:02:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658611+0000", 2024-09-13T07:02:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7341999999999993e-05 2024-09-13T07:02:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93978 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2670592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616646+0000", 2024-09-13T07:02:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.08624526, 2024-09-13T07:02:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042128074000000001, 2024-09-13T07:02:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93978 2024-09-13T07:02:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616646+0000", 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616646+0000", 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616645+0000", 2024-09-13T07:02:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616657+0000", 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1776999999999999e-05 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616666+0000", 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.363e-06 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616674+0000", 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8839999999999994e-06 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658636+0000", 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041962157 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658706+0000", 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0302000000000006e-05 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658774+0000", 2024-09-13T07:02:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7191000000000001e-05 2024-09-13T07:02:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93979 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086210817, 2024-09-13T07:02:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042253801000000001, 2024-09-13T07:02:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93979 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616683+0000", 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2820000000000001e-06 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616691+0000", 2024-09-13T07:02:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0779999999999996e-06 2024-09-13T07:02:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616698+0000", 2024-09-13T07:02:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:02:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658799+0000", 2024-09-13T07:02:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042100763999999999 2024-09-13T07:02:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658869+0000", 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0065000000000001e-05 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658934+0000", 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5071000000000001e-05 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:02:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086186251999999, 2024-09-13T07:02:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:02:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:02:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:02:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:02:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:02:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:02:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086163285, 2024-09-13T07:02:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:02:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:02:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:02:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:02:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:02:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:02:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:02:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:02:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:02:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:02:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:02:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086146106999999, 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:02:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:02:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:02:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:02:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086132242, 2024-09-13T07:02:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:02:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:02:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086119096999999, 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:02:36.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:02:36.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:02:36.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.764 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.764 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.764 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:36.764 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086105890000001, 2024-09-13T07:02:36.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:02:36.764 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.764 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.764 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.764 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:02:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:02:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:02:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:02:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:02:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:02:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086091823, 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:02:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086071809, 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:02:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:02:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086052059, 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:02:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:02:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086027252999999, 2024-09-13T07:02:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:02:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:02:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:02:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:02:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:02:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:02:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.086002988000001, 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:02:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.085961255000001, 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:02:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:02:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:02:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:02:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:02:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:02:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:02:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:02:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:02:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:02:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:02:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:02:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.085924966, 2024-09-13T07:02:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:02:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:02:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:02:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:02:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:02:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:02:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:02:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:02:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:02:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:02:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:02:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:02:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:36.801 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:36.802 INFO:teuthology.orchestra.run.smithi118.stdout:2902 trunc from 0xe14eafc to 0x44f0878 2024-09-13T07:02:36.803 INFO:teuthology.orchestra.run.smithi118.stdout:2903 writesame 0x91e1438 thru 0x91ed1ed data_size 0x7(0xbdb6 bytes) 2024-09-13T07:02:36.803 INFO:teuthology.orchestra.run.smithi118.stdout:2904 write 0x87e5a50 thru 0x87e7882 (0x1e33 bytes) 2024-09-13T07:02:36.803 INFO:teuthology.orchestra.run.smithi118.stdout:2905 trunc from 0x91ed1ee to 0xdf5847d 2024-09-13T07:02:36.803 INFO:teuthology.orchestra.run.smithi118.stdout:2906 read 0x85492f2 thru 0x854b7ff (0x250e bytes) 2024-09-13T07:02:36.803 INFO:teuthology.orchestra.run.smithi118.stdout:2908 write 0x55fd662 thru 0x5601c3b (0x45da bytes) 2024-09-13T07:02:36.803 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:36.804 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:36.798+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:36.828 INFO:teuthology.orchestra.run.smithi118.stdout:2909 trunc from 0xdf5847d to 0x124cb8d 2024-09-13T07:02:36.854 INFO:teuthology.orchestra.run.smithi118.stdout:2910 read 0x27f4e6 thru 0x281482 (0x1f9d bytes) 2024-09-13T07:02:36.856 INFO:teuthology.orchestra.run.smithi118.stdout:2911 trunc from 0x124cb8d to 0x18fca62 2024-09-13T07:02:36.903 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:36.900+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:36.906 INFO:teuthology.orchestra.run.smithi118.stdout:2912 write 0xe51453c thru 0xe520810 (0xc2d5 bytes) 2024-09-13T07:02:36.913 INFO:teuthology.orchestra.run.smithi118.stdout:2914 read 0xb937fb1 thru 0xb93a37e (0x23ce bytes) 2024-09-13T07:02:36.913 INFO:teuthology.orchestra.run.smithi118.stdout:2915 punch from 0x7868929 to 0x786947f, (0xb56 bytes) 2024-09-13T07:02:36.913 INFO:teuthology.orchestra.run.smithi118.stdout:2916 writesame 0x201c12f thru 0x201d4cd data_size 0x1(0x139f bytes) 2024-09-13T07:02:36.919 INFO:teuthology.orchestra.run.smithi118.stdout:2917 write 0x8a9e7e6 thru 0x8aa5d44 (0x755f bytes) 2024-09-13T07:02:36.926 INFO:teuthology.orchestra.run.smithi118.stdout:2918 writesame 0x72ba0fd thru 0x72c6829 data_size 0x1(0xc72d bytes) 2024-09-13T07:02:36.960 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:36.960 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:36.960 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:36.960 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:36.965 INFO:teuthology.orchestra.run.smithi118.stdout:2920 write 0x5f4ee69 thru 0x5f584c2 (0x965a bytes) 2024-09-13T07:02:36.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:36.973 INFO:teuthology.orchestra.run.smithi118.stdout:2921 write 0xd0ba06f thru 0xd0bdebc (0x3e4e bytes) 2024-09-13T07:02:36.993 INFO:teuthology.orchestra.run.smithi118.stdout:2922 trunc from 0xe520811 to 0x9921491 2024-09-13T07:02:37.004 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:37.002+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:37.016 INFO:teuthology.orchestra.run.smithi118.stdout:2923 writesame 0x39fe5fb thru 0x3a039d3 data_size 0x1(0x53d9 bytes) 2024-09-13T07:02:37.044 INFO:teuthology.orchestra.run.smithi118.stdout:2924 trunc from 0x9921491 to 0xdff73ca 2024-09-13T07:02:37.056 INFO:teuthology.orchestra.run.smithi118.stdout:2926 write 0xe56981 thru 0xe5bbf6 (0x5276 bytes) 2024-09-13T07:02:37.060 INFO:teuthology.orchestra.run.smithi118.stdout:2927 writesame 0xd070b53 thru 0xd07910b data_size 0x1(0x85b9 bytes) 2024-09-13T07:02:37.079 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:37.079 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:37.080 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:37.080 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:37.080 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:37.090 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:37.102 INFO:teuthology.orchestra.run.smithi118.stdout:2928 trunc from 0xdff73ca to 0x789c0fc 2024-09-13T07:02:37.105 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:37.104+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:37.127 INFO:teuthology.orchestra.run.smithi118.stdout:2929 writesame 0x5022670 thru 0x5030369 data_size 0x5(0xdcfa bytes) 2024-09-13T07:02:37.145 INFO:teuthology.orchestra.run.smithi118.stdout:2931 read 0x6c68430 thru 0x6c76322 (0xdef3 bytes) 2024-09-13T07:02:37.145 INFO:teuthology.orchestra.run.smithi118.stdout:2932 write 0x589ff29 thru 0x58a4979 (0x4a51 bytes) 2024-09-13T07:02:37.148 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:02:37.175 INFO:teuthology.orchestra.run.smithi118.stdout:2933 write 0xe90ffa5 thru 0xe91f1e0 (0xf23c bytes) 2024-09-13T07:02:37.182 INFO:teuthology.orchestra.run.smithi118.stdout:2935 write 0x95c651f thru 0x95c75c8 (0x10aa bytes) 2024-09-13T07:02:37.185 INFO:teuthology.orchestra.run.smithi118.stdout:2936 write 0x85a141f thru 0x85aa6ad (0x928f bytes) 2024-09-13T07:02:37.190 INFO:teuthology.orchestra.run.smithi118.stdout:2939 write 0x8d2a619 thru 0x8d2cbe3 (0x25cb bytes) 2024-09-13T07:02:37.205 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:37.202+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:37.215 INFO:teuthology.orchestra.run.smithi118.stdout:2941 trunc from 0xe91f1e1 to 0x277ba13 2024-09-13T07:02:37.242 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:37.242 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:37.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:37.242 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:37.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.242 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:02:37.242 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:37.242 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.186640256999993, 2024-09-13T07:02:37.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:02:37.242 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:02:37.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 39.028759022999999, 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:37.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:37.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:02:37.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:37.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:02:37.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:02:37.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:02:37.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:02:37.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:02:37.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:02:37.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:37.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:02:37.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:02:37.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:37.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:02:37.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 39.002989311999997, 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.253 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.254 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.254 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:02:37.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.254 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:02:37.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:02:37.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:37.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.981838146000001, 2024-09-13T07:02:37.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:02:37.260 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.260 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:37.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.260 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:02:37.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.260 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:37.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:02:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:02:37.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:37.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.864793382999999, 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:02:37.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:02:37.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:37.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.270 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94595 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2318336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.660977+0000", 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.579836083, 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044248540000000003, 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94595 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660977+0000", 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660977+0000", 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660976+0000", 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660979+0000", 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5229999999999998e-06 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660985+0000", 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0039999999999996e-06 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660991+0000", 2024-09-13T07:02:37.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8850000000000004e-06 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705116+0000", 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044125315999999998 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705183+0000", 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6476000000000005e-05 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705225+0000", 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2651000000000002e-05 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94596 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.579816903999999, 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044330268999999999, 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94596 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660995+0000", 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660999+0000", 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2770000000000001e-06 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661007+0000", 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0029999999999994e-06 2024-09-13T07:02:37.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661012+0000", 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1279999999999999e-06 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705253+0000", 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044241603999999997 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705293+0000", 2024-09-13T07:02:37.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9684999999999997e-05 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705326+0000", 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2932000000000003e-05 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.579796138000001, 2024-09-13T07:02:37.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:02:37.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:02:37.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.284 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94598 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:37.284 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:02:37.284 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.579625417000001, 2024-09-13T07:02:37.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044365860999999999, 2024-09-13T07:02:37.284 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.284 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:37.284 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.284 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.284 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.284 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94598 2024-09-13T07:02:37.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.284 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661034+0000", 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9998465 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661192+0000", 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015813000000000001 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661199+0000", 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5689999999999999e-06 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:02:37.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705452+0000", 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044246273000000003 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705501+0000", 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8748000000000002e-05 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705553+0000", 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2241999999999999e-05 2024-09-13T07:02:37.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107060 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1003520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3281031280000004, 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044341754999999997, 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.288 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107060 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912709+0000", 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912712+0000", 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6000000000000001e-06 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912717+0000", 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3689999999999998e-06 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912722+0000", 2024-09-13T07:02:37.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7219999999999999e-06 2024-09-13T07:02:37.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956876+0000", 2024-09-13T07:02:37.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044154469000000002 2024-09-13T07:02:37.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956957+0000", 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1231999999999997e-05 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957051+0000", 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3858999999999997e-05 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.292 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:37.293 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:02:37.293 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.328087, 2024-09-13T07:02:37.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:02:37.293 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.293 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:37.293 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.293 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.293 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.293 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:02:37.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.293 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:02:37.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:02:37.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:37.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:02:37.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:02:37.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:02:37.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.328068966, 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:02:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:02:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:02:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:02:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:37.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.328052886, 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:02:37.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:02:37.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:02:37.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:02:37.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:02:37.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:02:37.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:02:37.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:02:37.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3280372099999997, 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.306 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.309 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3280142880000003, 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.310 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:02:37.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:02:37.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:02:37.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:02:37.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:02:37.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.313 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.314 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:37.314 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:02:37.314 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3279981909999998, 2024-09-13T07:02:37.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:02:37.314 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.314 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:37.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.314 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:02:37.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.315 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:02:37.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:02:37.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:02:37.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:02:37.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:02:37.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:02:37.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:02:37.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:02:37.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:02:37.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:02:37.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:02:37.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107067 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3274381740000001, 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04441055, 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:37.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.319 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107067 2024-09-13T07:02:37.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:02:37.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912832+0000", 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9994574 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913379+0000", 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00054719999999999997 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913386+0000", 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4549999999999998e-06 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913393+0000", 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6259999999999997e-06 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957776+0000", 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044383143999999999 2024-09-13T07:02:37.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957785+0000", 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9220000000000003e-06 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107068 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3274137430000001, 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044410502999999997, 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.322 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107068 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913401+0000", 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913407+0000", 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7520000000000002e-06 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913412+0000", 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8550000000000001e-06 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957802+0000", 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044389606999999998 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957809+0000", 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9160000000000001e-06 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3273965790000002, 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.326 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:02:37.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3273538030000003, 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:37.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:02:37.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:02:37.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:37.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:02:37.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:02:37.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:37.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:02:37.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:02:37.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:37.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:37.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:37.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:02:37.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:02:37.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.334 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:37.334 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:37.334 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:37.336 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:37.337 INFO:teuthology.orchestra.run.smithi118.stdout:2943 read 0x11a18a thru 0x124c18 (0xaa8f bytes) 2024-09-13T07:02:37.337 INFO:teuthology.orchestra.run.smithi118.stdout:2944 write 0xe3f9292 thru 0xe3fecef (0x5a5e bytes) 2024-09-13T07:02:37.337 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:37.337 INFO:teuthology.orchestra.run.smithi118.stdout:2947 write 0x22a2194 thru 0x22a3dd1 (0x1c3e bytes) 2024-09-13T07:02:37.337 INFO:teuthology.orchestra.run.smithi118.stdout:2949 punch from 0xd7c87a8 to 0xd7ceabc, (0x6314 bytes) 2024-09-13T07:02:37.337 INFO:teuthology.orchestra.run.smithi118.stdout:2950 read 0x9bbfd4a thru 0x9bc6871 (0x6b28 bytes) 2024-09-13T07:02:37.337 INFO:teuthology.orchestra.run.smithi118.stdout:2951 writesame 0x1e13de9 thru 0x1e15625 data_size 0x11(0x183d bytes) 2024-09-13T07:02:37.337 INFO:teuthology.orchestra.run.smithi118.stdout:2952 write 0xd0c893d thru 0xd0cdcd3 (0x5397 bytes) 2024-09-13T07:02:37.337 INFO:teuthology.orchestra.run.smithi118.stdout:2953 punch from 0x97699e6 to 0x977731c, (0xd936 bytes) 2024-09-13T07:02:37.338 INFO:teuthology.orchestra.run.smithi118.stdout:2955 write 0xa7f5ab3 thru 0xa7fd6c1 (0x7c0f bytes) 2024-09-13T07:02:37.338 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:37.338 INFO:teuthology.orchestra.run.smithi118.stdout:2956 trunc from 0xe3fecf0 to 0x5161b0e 2024-09-13T07:02:37.338 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:37.339 INFO:teuthology.orchestra.run.smithi118.stdout:2957 write 0x2c2cb17 thru 0x2c3605f (0x9549 bytes) 2024-09-13T07:02:37.339 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:37.332+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:37.349 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:37.354 INFO:teuthology.orchestra.run.smithi118.stdout:2959 write 0xa408b17 thru 0xa40eae1 (0x5fcb bytes) 2024-09-13T07:02:37.360 INFO:teuthology.orchestra.run.smithi118.stdout:2962 write 0x7ad956d thru 0x7adae06 (0x189a bytes) 2024-09-13T07:02:37.371 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:37.374 INFO:teuthology.orchestra.run.smithi118.stdout:2963 trunc from 0xa40eae2 to 0xdc2dc9e 2024-09-13T07:02:37.386 INFO:teuthology.orchestra.run.smithi118.stdout:2964 write 0x2bfca3e thru 0x2c039b0 (0x6f73 bytes) 2024-09-13T07:02:37.390 INFO:teuthology.orchestra.run.smithi118.stdout:2965 read 0xc6496e3 thru 0xc656acf (0xd3ed bytes) 2024-09-13T07:02:37.390 INFO:teuthology.orchestra.run.smithi118.stdout:2967 writesame 0x2a3f71a thru 0x2a45c2f data_size 0x39(0x6516 bytes) 2024-09-13T07:02:37.393 INFO:teuthology.orchestra.run.smithi118.stdout:2968 writesame 0xc3fd22b thru 0xc40469d data_size 0x13(0x7473 bytes) 2024-09-13T07:02:37.396 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:37.401 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:37.401 INFO:teuthology.orchestra.run.smithi118.stdout:2972 punch from 0x1b244d0 to 0x1b27dca, (0x38fa bytes) 2024-09-13T07:02:37.402 INFO:teuthology.orchestra.run.smithi118.stdout:2973 write 0x8ad3ddb thru 0x8adfa16 (0xbc3c bytes) 2024-09-13T07:02:37.416 INFO:teuthology.orchestra.run.smithi118.stdout:2976 trunc from 0xdc2dc9e to 0x81fe38d 2024-09-13T07:02:37.424 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:37.436 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:37.434+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:37.444 INFO:teuthology.orchestra.run.smithi118.stderr:set 2.5 pg_upmap mapping to [6,4] 2024-09-13T07:02:37.450 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:37.450 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:37.450 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:37.453 INFO:teuthology.orchestra.run.smithi118.stdout:2978 write 0x24fd789 thru 0x250d289 (0xfb01 bytes) 2024-09-13T07:02:37.456 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:37.456 INFO:teuthology.orchestra.run.smithi118.stdout:2982 read 0x7a3f389 thru 0x7a4e39c (0xf014 bytes) 2024-09-13T07:02:37.458 INFO:teuthology.orchestra.run.smithi118.stdout:2983 punch from 0x38b6db5 to 0x38c07f2, (0x9a3d bytes) 2024-09-13T07:02:37.458 INFO:teuthology.orchestra.run.smithi118.stdout:2984 writesame 0x1cab746 thru 0x1cb22ee data_size 0x1(0x6ba9 bytes) 2024-09-13T07:02:37.460 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:37.471 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:37.473 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:37.473 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:37.474 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:37.474 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:37.474 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:37.474 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:37.474 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:37.474 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:37.480 INFO:teuthology.orchestra.run.smithi118.stdout:2985 read 0x26113a7 thru 0x2614916 (0x3570 bytes) 2024-09-13T07:02:37.480 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:37.486 INFO:teuthology.orchestra.run.smithi118.stdout:2987 trunc from 0x81fe38d to 0xabbffd7 2024-09-13T07:02:37.496 INFO:teuthology.orchestra.run.smithi118.stdout:2988 read 0x347e154 thru 0x348a7aa (0xc657 bytes) 2024-09-13T07:02:37.496 INFO:teuthology.orchestra.run.smithi118.stdout:2989 write 0x7a6a4e0 thru 0x7a6e842 (0x4363 bytes) 2024-09-13T07:02:37.499 INFO:teuthology.orchestra.run.smithi118.stdout:2991 write 0x2f5e645 thru 0x2f68142 (0x9afe bytes) 2024-09-13T07:02:37.506 INFO:teuthology.orchestra.run.smithi118.stdout:2993 trunc from 0xabbffd7 to 0xcfe11ce 2024-09-13T07:02:37.518 INFO:teuthology.orchestra.run.smithi118.stdout:2994 write 0x5a63bbd thru 0x5a71b0b (0xdf4f bytes) 2024-09-13T07:02:37.526 INFO:teuthology.orchestra.run.smithi118.stdout:2996 write 0x20efd70 thru 0x20f4196 (0x4427 bytes) 2024-09-13T07:02:37.529 INFO:teuthology.orchestra.run.smithi118.stdout:2997 write 0x649ebe8 thru 0x64a5309 (0x6722 bytes) 2024-09-13T07:02:37.533 INFO:teuthology.orchestra.run.smithi118.stdout:2998 read 0x2c396b4 thru 0x2c46567 (0xceb4 bytes) 2024-09-13T07:02:37.537 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:37.534+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:37.553 INFO:teuthology.orchestra.run.smithi118.stdout:2999 trunc from 0xcfe11ce to 0x116ec02 2024-09-13T07:02:37.578 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:37.578 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:37.579 INFO:teuthology.orchestra.run.smithi118.stdout:3000 read 0x115339f thru 0x1158e93 (0x5af5 bytes) 2024-09-13T07:02:37.580 INFO:teuthology.orchestra.run.smithi118.stdout:3001 punch from 0x82b4ef to 0x8398a8, (0xe3b9 bytes) 2024-09-13T07:02:37.590 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:37.608 INFO:teuthology.orchestra.run.smithi118.stdout:3003 write 0xa38a1dd thru 0xa38c872 (0x2696 bytes) 2024-09-13T07:02:37.614 INFO:teuthology.orchestra.run.smithi118.stdout:3004 punch from 0x7db1b32 to 0x7db316a, (0x1638 bytes) 2024-09-13T07:02:37.614 INFO:teuthology.orchestra.run.smithi118.stdout:3006 punch from 0x44b1649 to 0x44be63d, (0xcff4 bytes) 2024-09-13T07:02:37.631 INFO:teuthology.orchestra.run.smithi118.stdout:3010 trunc from 0xa38c873 to 0x6d061e 2024-09-13T07:02:37.638 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:37.636+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:37.680 INFO:teuthology.orchestra.run.smithi118.stdout:3011 write 0x8a6e045 thru 0x8a742f8 (0x62b4 bytes) 2024-09-13T07:02:37.707 INFO:teuthology.orchestra.run.smithi118.stdout:3012 write 0xde0d5f1 thru 0xde1c535 (0xef45 bytes) 2024-09-13T07:02:37.714 INFO:teuthology.orchestra.run.smithi118.stdout:3013 write 0xbf97ab thru 0xc033aa (0x9c00 bytes) 2024-09-13T07:02:37.717 INFO:teuthology.orchestra.run.smithi118.stdout:3014 write 0xddb94ed thru 0xddc3ee4 (0xa9f8 bytes) 2024-09-13T07:02:37.719 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:37.719 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:37.724 INFO:teuthology.orchestra.run.smithi118.stdout:3015 write 0xcf9f463 thru 0xcfa733c (0x7eda bytes) 2024-09-13T07:02:37.727 INFO:teuthology.orchestra.run.smithi118.stdout:3016 write 0xc11bef8 thru 0xc123cad (0x7db6 bytes) 2024-09-13T07:02:37.730 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:37.735 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:37.735 INFO:teuthology.orchestra.run.smithi118.stdout:3018 writesame 0x2286938 thru 0x2289ad0 data_size 0x1(0x3199 bytes) 2024-09-13T07:02:37.747 INFO:teuthology.orchestra.run.smithi118.stdout:3019 read 0x1c843 thru 0x23175 (0x6933 bytes) 2024-09-13T07:02:37.747 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:37.744+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi118.stdout:3020 trunc from 0xde1c536 to 0x5d168da 2024-09-13T07:02:37.807 INFO:teuthology.orchestra.run.smithi118.stdout:3021 writesame 0xb0980c4 thru 0xb09957f data_size 0x2(0x14bc bytes) 2024-09-13T07:02:37.807 INFO:teuthology.orchestra.run.smithi118.stdout:3022 punch from 0x275011f to 0x275bc1b, (0xbafc bytes) 2024-09-13T07:02:37.807 INFO:teuthology.orchestra.run.smithi118.stdout:3023 write 0x1e8aae1 thru 0x1e8cc7b (0x219b bytes) 2024-09-13T07:02:37.810 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:37.810 INFO:teuthology.orchestra.run.smithi118.stdout:3027 read 0x1687df9 thru 0x1694c0e (0xce16 bytes) 2024-09-13T07:02:37.811 INFO:teuthology.orchestra.run.smithi118.stdout:3030 writesame 0x5547a80 thru 0x554f965 data_size 0x2(0x7ee6 bytes) 2024-09-13T07:02:37.829 INFO:teuthology.orchestra.run.smithi118.stdout:3032 read 0x544f7b6 thru 0x54525fe (0x2e49 bytes) 2024-09-13T07:02:37.829 INFO:teuthology.orchestra.run.smithi118.stdout:3033 write 0xad49832 thru 0xad4c104 (0x28d3 bytes) 2024-09-13T07:02:37.835 INFO:teuthology.orchestra.run.smithi118.stdout:3035 write 0x5be364f thru 0x5bf0980 (0xd332 bytes) 2024-09-13T07:02:37.837 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:37.837 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:37.840 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:37.836+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:37.840 INFO:teuthology.orchestra.run.smithi118.stdout:3038 write 0x95e909d thru 0x95ec4e7 (0x344b bytes) 2024-09-13T07:02:37.846 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:37.847 INFO:teuthology.orchestra.run.smithi118.stdout:3039 read 0x46cbc94 thru 0x46d44a7 (0x8814 bytes) 2024-09-13T07:02:37.847 INFO:teuthology.orchestra.run.smithi118.stdout:3040 read 0x76db664 thru 0x76e480c (0x91a9 bytes) 2024-09-13T07:02:37.848 INFO:teuthology.orchestra.run.smithi118.stdout:3041 write 0x23a855a thru 0x23b0926 (0x83cd bytes) 2024-09-13T07:02:37.865 INFO:teuthology.orchestra.run.smithi118.stdout:3043 write 0xca63886 thru 0xca64f79 (0x16f4 bytes) 2024-09-13T07:02:37.872 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:37.872 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:37.872 INFO:teuthology.orchestra.run.smithi118.stdout:3047 write 0x3b54a15 thru 0x3b5d653 (0x8c3f bytes) 2024-09-13T07:02:37.879 INFO:teuthology.orchestra.run.smithi118.stdout:3048 punch from 0xa0c8b9b to 0xa0cf559, (0x69be bytes) 2024-09-13T07:02:37.879 INFO:teuthology.orchestra.run.smithi118.stdout:3049 write 0x24c71dc thru 0x24d526d (0xe092 bytes) 2024-09-13T07:02:37.882 INFO:teuthology.orchestra.run.smithi118.stdout:3052 read 0x70a8c1d thru 0x70b4395 (0xb779 bytes) 2024-09-13T07:02:37.882 INFO:teuthology.orchestra.run.smithi118.stdout:3053 write 0x360f0b thru 0x36f879 (0xe96f bytes) 2024-09-13T07:02:37.886 INFO:teuthology.orchestra.run.smithi118.stdout:3054 write 0xb8acf1e thru 0xb8af363 (0x2446 bytes) 2024-09-13T07:02:37.901 INFO:teuthology.orchestra.run.smithi118.stdout:3055 write 0xcd5b5d8 thru 0xcd5feba (0x48e3 bytes) 2024-09-13T07:02:37.923 INFO:teuthology.orchestra.run.smithi118.stdout:3057 trunc from 0xcd5febb to 0x2bdb6b6 2024-09-13T07:02:37.940 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:37.938+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:37.956 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:37.956 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:37.968 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:37.970 INFO:teuthology.orchestra.run.smithi118.stdout:3058 write 0x9435be1 thru 0x943db73 (0x7f93 bytes) 2024-09-13T07:02:37.977 INFO:teuthology.orchestra.run.smithi118.stdout:3060 read 0x7ce5e92 thru 0x7ce90a1 (0x3210 bytes) 2024-09-13T07:02:37.980 INFO:teuthology.orchestra.run.smithi118.stdout:3061 trunc from 0x943db74 to 0xb4bc6b3 2024-09-13T07:02:37.991 INFO:teuthology.orchestra.run.smithi118.stdout:3062 read 0x3e575ac thru 0x3e64859 (0xd2ae bytes) 2024-09-13T07:02:38.008 INFO:teuthology.orchestra.run.smithi118.stdout:3063 write 0xd7fba97 thru 0xd807cf9 (0xc263 bytes) 2024-09-13T07:02:38.016 INFO:teuthology.orchestra.run.smithi118.stdout:3064 read 0x24e886d thru 0x24edc50 (0x53e4 bytes) 2024-09-13T07:02:38.017 INFO:teuthology.orchestra.run.smithi118.stdout:3065 write 0xab251bb thru 0xab2c3c0 (0x7206 bytes) 2024-09-13T07:02:38.024 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:38.024 INFO:teuthology.orchestra.run.smithi118.stdout:3068 read 0x781b30f thru 0x782b0e8 (0xfdda bytes) 2024-09-13T07:02:38.034 INFO:teuthology.orchestra.run.smithi118.stdout:3069 trunc from 0xd807cfa to 0x8a8c0d9 2024-09-13T07:02:38.041 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:38.040+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:38.058 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:38.058 INFO:teuthology.orchestra.run.smithi118.stdout:3071 writesame 0x16b05d2 thru 0x16bee48 data_size 0x53(0xe877 bytes) 2024-09-13T07:02:38.061 INFO:teuthology.orchestra.run.smithi118.stdout:3073 read 0xf2daa5 thru 0xf38a68 (0xafc4 bytes) 2024-09-13T07:02:38.062 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:38.062 INFO:teuthology.orchestra.run.smithi118.stdout:3075 write 0x33603d9 thru 0x336804f (0x7c77 bytes) 2024-09-13T07:02:38.065 INFO:teuthology.orchestra.run.smithi118.stdout:3076 write 0x35dd333 thru 0x35e797c (0xa64a bytes) 2024-09-13T07:02:38.068 INFO:teuthology.orchestra.run.smithi118.stdout:3079 read 0x2fc89d1 thru 0x2fd0178 (0x77a8 bytes) 2024-09-13T07:02:38.069 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:38.069 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:38.069 INFO:teuthology.orchestra.run.smithi118.stdout:3083 read 0x662057 thru 0x66e7ce (0xc778 bytes) 2024-09-13T07:02:38.073 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:38.073 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:38.084 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:38.089 INFO:teuthology.orchestra.run.smithi118.stdout:3084 writesame 0xabe9684 thru 0xabeedb4 data_size 0xd(0x5731 bytes) 2024-09-13T07:02:38.100 INFO:teuthology.orchestra.run.smithi118.stdout:3085 trunc from 0xabeedb5 to 0x54380d7 2024-09-13T07:02:38.138 INFO:teuthology.orchestra.run.smithi118.stdout:3086 trunc from 0x54380d7 to 0xec94cf6 2024-09-13T07:02:38.142 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:38.138+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:38.150 INFO:teuthology.orchestra.run.smithi118.stdout:3087 read 0x5037925 thru 0x503b4bb (0x3b97 bytes) 2024-09-13T07:02:38.150 INFO:teuthology.orchestra.run.smithi118.stdout:3088 write 0xe029654 thru 0xe037249 (0xdbf6 bytes) 2024-09-13T07:02:38.157 INFO:teuthology.orchestra.run.smithi118.stdout:3089 writesame 0x2689f71 thru 0x268b0b9 data_size 0x19(0x1149 bytes) 2024-09-13T07:02:38.160 INFO:teuthology.orchestra.run.smithi118.stdout:3090 read 0xc19ba16 thru 0xc19bf28 (0x513 bytes) 2024-09-13T07:02:38.161 INFO:teuthology.orchestra.run.smithi118.stdout:3092 write 0xa04121a thru 0xa04f4f8 (0xe2df bytes) 2024-09-13T07:02:38.167 INFO:teuthology.orchestra.run.smithi118.stdout:3093 write 0x41dbd42 thru 0x41dd430 (0x16ef bytes) 2024-09-13T07:02:38.184 INFO:teuthology.orchestra.run.smithi118.stdout:3095 trunc from 0xec94cf6 to 0x94bc553 2024-09-13T07:02:38.197 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:38.198 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:38.208 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:38.209 INFO:teuthology.orchestra.run.smithi118.stdout:3096 write 0x44c2dfd thru 0x44cd240 (0xa444 bytes) 2024-09-13T07:02:38.224 INFO:teuthology.orchestra.run.smithi118.stdout:3097 trunc from 0x94bc553 to 0xd029da1 2024-09-13T07:02:38.243 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:38.242+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:38.254 INFO:teuthology.orchestra.run.smithi118.stdout:3098 trunc from 0xd029da1 to 0x2d68a62 2024-09-13T07:02:38.293 INFO:teuthology.orchestra.run.smithi118.stdout:3099 write 0x31d8680 thru 0x31dca27 (0x43a8 bytes) 2024-09-13T07:02:38.300 INFO:teuthology.orchestra.run.smithi118.stdout:3100 trunc from 0x31dca28 to 0x1126fee 2024-09-13T07:02:38.325 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:38.325 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:38.333 INFO:teuthology.orchestra.run.smithi118.stdout:3101 writesame 0x40eb4a1 thru 0x40f6922 data_size 0x5(0xb482 bytes) 2024-09-13T07:02:38.335 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:38.347 INFO:teuthology.orchestra.run.smithi118.stdout:3102 write 0x4c658e9 thru 0x4c718fc (0xc014 bytes) 2024-09-13T07:02:38.348 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:38.344+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:38.354 INFO:teuthology.orchestra.run.smithi118.stdout:3104 read 0x1b02d63 thru 0x1b0487a (0x1b18 bytes) 2024-09-13T07:02:38.355 INFO:teuthology.orchestra.run.smithi118.stdout:3105 punch from 0x3a4efd to 0x3a83d6, (0x34d9 bytes) 2024-09-13T07:02:38.355 INFO:teuthology.orchestra.run.smithi118.stdout:3107 punch from 0x22a7a0a to 0x22b2279, (0xa86f bytes) 2024-09-13T07:02:38.355 INFO:teuthology.orchestra.run.smithi118.stdout:3108 read 0x1128f8b thru 0x112bc6a (0x2ce0 bytes) 2024-09-13T07:02:38.377 INFO:teuthology.orchestra.run.smithi118.stdout:3110 write 0x981b74e thru 0x982524d (0x9b00 bytes) 2024-09-13T07:02:38.384 INFO:teuthology.orchestra.run.smithi118.stdout:3112 write 0x370ec21 thru 0x371e015 (0xf3f5 bytes) 2024-09-13T07:02:38.387 INFO:teuthology.orchestra.run.smithi118.stdout:3113 read 0x10e07e1 thru 0x10e75de (0x6dfe bytes) 2024-09-13T07:02:38.388 INFO:teuthology.orchestra.run.smithi118.stdout:3114 read 0x29c9ed3 thru 0x29d9598 (0xf6c6 bytes) 2024-09-13T07:02:38.389 INFO:teuthology.orchestra.run.smithi118.stdout:3115 punch from 0x829ef8f to 0x82a9932, (0xa9a3 bytes) 2024-09-13T07:02:38.389 INFO:teuthology.orchestra.run.smithi118.stdout:3116 read 0x917ddc1 thru 0x9184eba (0x70fa bytes) 2024-09-13T07:02:38.390 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:38.390 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:38.391 INFO:teuthology.orchestra.run.smithi118.stdout:3120 read 0x9520363 thru 0x9523cae (0x394c bytes) 2024-09-13T07:02:38.391 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:38.391 INFO:teuthology.orchestra.run.smithi118.stdout:3126 read 0x8188f43 thru 0x818b98b (0x2a49 bytes) 2024-09-13T07:02:38.391 INFO:teuthology.orchestra.run.smithi118.stdout:3127 read 0x1c203db thru 0x1c2d3ed (0xd013 bytes) 2024-09-13T07:02:38.391 INFO:teuthology.orchestra.run.smithi118.stdout:3128 read 0x5b4d2d7 thru 0x5b51288 (0x3fb2 bytes) 2024-09-13T07:02:38.391 INFO:teuthology.orchestra.run.smithi118.stdout:3129 read 0x4d49e2f thru 0x4d59809 (0xf9db bytes) 2024-09-13T07:02:38.405 INFO:teuthology.orchestra.run.smithi118.stdout:3130 writesame 0xa1ed11a thru 0xa1f4b50 data_size 0x1(0x7a37 bytes) 2024-09-13T07:02:38.420 INFO:teuthology.orchestra.run.smithi118.stdout:3131 write 0xb76de8e thru 0xb773de9 (0x5f5c bytes) 2024-09-13T07:02:38.428 INFO:teuthology.orchestra.run.smithi118.stdout:3132 write 0x177183 thru 0x1870c2 (0xff40 bytes) 2024-09-13T07:02:38.433 INFO:teuthology.orchestra.run.smithi118.stdout:3133 write 0x8ea6883 thru 0x8eade77 (0x75f5 bytes) 2024-09-13T07:02:38.440 INFO:teuthology.orchestra.run.smithi118.stdout:3135 read 0x263876f thru 0x2648674 (0xff06 bytes) 2024-09-13T07:02:38.441 INFO:teuthology.orchestra.run.smithi118.stdout:3137 read 0x4316f8c thru 0x4322de2 (0xbe57 bytes) 2024-09-13T07:02:38.441 INFO:teuthology.orchestra.run.smithi118.stdout:3139 punch from 0x66186f7 to 0x661ff93, (0x789c bytes) 2024-09-13T07:02:38.441 INFO:teuthology.orchestra.run.smithi118.stdout:3142 read 0x58dcbd5 thru 0x58dea3b (0x1e67 bytes) 2024-09-13T07:02:38.441 INFO:teuthology.orchestra.run.smithi118.stdout:3143 write 0x47ce908 thru 0x47d2472 (0x3b6b bytes) 2024-09-13T07:02:38.445 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:38.444+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:38.446 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:02:38.449 INFO:teuthology.orchestra.run.smithi118.stdout:3144 write 0xa44eb96 thru 0xa4518d9 (0x2d44 bytes) 2024-09-13T07:02:38.457 INFO:teuthology.orchestra.run.smithi118.stdout:3146 write 0x3e91526 thru 0x3e9e3ef (0xceca bytes) 2024-09-13T07:02:38.462 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:38.462 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:38.466 INFO:teuthology.orchestra.run.smithi118.stdout:3148 trunc from 0xb773dea to 0x8c9d444 2024-09-13T07:02:38.474 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:38.486 INFO:teuthology.orchestra.run.smithi118.stdout:3149 write 0x89cf710 thru 0x89db564 (0xbe55 bytes) 2024-09-13T07:02:38.492 INFO:teuthology.orchestra.run.smithi118.stdout:3150 read 0xea5aa0 thru 0xeaa2b2 (0x4813 bytes) 2024-09-13T07:02:38.493 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:38.509 INFO:teuthology.orchestra.run.smithi118.stdout:3154 writesame 0xa964aca thru 0xa97194f data_size 0x55(0xce86 bytes) 2024-09-13T07:02:38.509 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:38.509 INFO:teuthology.orchestra.run.smithi118.stdout:3157 writesame 0x2b1e896 thru 0x2b266d1 data_size 0x6(0x7e3c bytes) 2024-09-13T07:02:38.516 INFO:teuthology.orchestra.run.smithi118.stdout:3158 write 0x1688850 thru 0x1692759 (0x9f0a bytes) 2024-09-13T07:02:38.519 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:38.519 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:38.519 INFO:teuthology.orchestra.run.smithi118.stdout:3161 read 0x709c0b1 thru 0x70a56c9 (0x9619 bytes) 2024-09-13T07:02:38.539 INFO:teuthology.orchestra.run.smithi118.stdout:3162 writesame 0xedffd93 thru 0xee06e4d data_size 0x1(0x70bb bytes) 2024-09-13T07:02:38.539 INFO:teuthology.orchestra.run.smithi118.stdout:3163 writesame 0xe6da876 thru 0xe6dfa06 data_size 0x13(0x5191 bytes) 2024-09-13T07:02:38.545 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:38.542+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:38.567 INFO:teuthology.orchestra.run.smithi118.stdout:3164 trunc from 0xee06e4e to 0x1f2f414 2024-09-13T07:02:38.595 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:38.596 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:38.608 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:38.611 INFO:teuthology.orchestra.run.smithi118.stdout:3165 writesame 0x65b7e83 thru 0x65c5774 data_size 0x7(0xd8f2 bytes) 2024-09-13T07:02:38.611 INFO:teuthology.orchestra.run.smithi118.stdout:3166 writesame 0x2d49ef1 thru 0x2d4a7f1 data_size 0x1(0x901 bytes) 2024-09-13T07:02:38.625 INFO:teuthology.orchestra.run.smithi118.stdout:3168 trunc from 0x65c5775 to 0x1f4f768 2024-09-13T07:02:38.646 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:38.644+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:38.651 INFO:teuthology.orchestra.run.smithi118.stdout:3169 write 0x392f7f9 thru 0x3938849 (0x9051 bytes) 2024-09-13T07:02:38.657 INFO:teuthology.orchestra.run.smithi118.stdout:3170 clone 31 order 25 su 524288 sc 7 2024-09-13T07:02:38.713 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:38.713 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:38.722 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:38.765 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:38.764+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:38.848 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:38.844+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:38.873 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:38.874 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:38.883 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:38.950 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:38.946+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:39.003 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:39.004 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:39.014 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:39.050 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:39.046+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:39.123 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:39.123 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:39.133 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:39.175 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:39.174+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:39.252 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:39.250+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:39.285 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:39.286 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:39.295 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:39.352 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:39.348+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:39.401 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:39.402 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:39.411 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:39.453 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:39.450+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:39.462 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:02:39.493 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:02:39.524 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:39.525 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:39.536 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:39.579 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:39.578+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:39.581 INFO:teuthology.orchestra.run.smithi118.stdout:leaving image image_client.0-clone30 intact 2024-09-13T07:02:39.655 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:39.654+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:39.694 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:39.695 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:39.695 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:39.705 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:39.705 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-13T07:02:39.714 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:39.732 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:39.750 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:39.756 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:39.754+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:39.771 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:39.794 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:39.809 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:39.809 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:39.817 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:39.825 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:39.843 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:39.845 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:39.845 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:39.845 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:39.845 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:39.845 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:39.845 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:39.845 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:39.845 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:39.856 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:39.854+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:39.924 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:39.924 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:39.924 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:39.924 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:39.924 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:39.933 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:39.957 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:39.956+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:40.057 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:40.054+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:40.086 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:40.086 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:40.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.025902314999996, 2024-09-13T07:02:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:02:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:02:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:02:40.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:02:40.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:02:40.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:02:40.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:02:40.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:02:40.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:02:40.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:40.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:02:40.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:02:40.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:40.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:02:40.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:02:40.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.092 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93974 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2260992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616547+0000", 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.468276339999999, 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041570436000000002, 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93974 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.093 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616547+0000", 2024-09-13T07:02:40.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616547+0000", 2024-09-13T07:02:40.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616546+0000", 2024-09-13T07:02:40.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616549+0000", 2024-09-13T07:02:40.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2610000000000002e-06 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616558+0000", 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.67e-06 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616565+0000", 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4930000000000003e-06 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.657979+0000", 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041414765999999999 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658048+0000", 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9003000000000003e-05 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658117+0000", 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8752999999999997e-05 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93975 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2375680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616571+0000", 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.468251918, 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041711005000000002, 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93975 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.099 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616571+0000", 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616571+0000", 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616571+0000", 2024-09-13T07:02:40.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616575+0000", 2024-09-13T07:02:40.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0069999999999999e-06 2024-09-13T07:02:40.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616583+0000", 2024-09-13T07:02:40.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1960000000000008e-06 2024-09-13T07:02:40.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616590+0000", 2024-09-13T07:02:40.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8419999999999999e-06 2024-09-13T07:02:40.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658142+0000", 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041552736999999999 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658215+0000", 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2279000000000004e-05 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658282+0000", 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7471999999999994e-05 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.103 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93976 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616595+0000", 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.468227757999999, 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041851514999999999, 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93976 2024-09-13T07:02:40.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.105 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616595+0000", 2024-09-13T07:02:40.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616595+0000", 2024-09-13T07:02:40.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616595+0000", 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616599+0000", 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9079999999999997e-06 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616607+0000", 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0579999999999998e-06 2024-09-13T07:02:40.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616613+0000", 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7449999999999998e-06 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658307+0000", 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041693725000000001 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658379+0000", 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2268000000000007e-05 2024-09-13T07:02:40.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658447+0000", 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7327000000000001e-05 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93977 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616621+0000", 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.468202212, 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041990496000000002, 2024-09-13T07:02:40.108 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.109 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.109 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93977 2024-09-13T07:02:40.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.109 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616621+0000", 2024-09-13T07:02:40.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616621+0000", 2024-09-13T07:02:40.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616620+0000", 2024-09-13T07:02:40.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616624+0000", 2024-09-13T07:02:40.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7620000000000001e-06 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616632+0000", 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2910000000000007e-06 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616640+0000", 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.469e-06 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658472+0000", 2024-09-13T07:02:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041832326000000003 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658544+0000", 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1957000000000002e-05 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658611+0000", 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7341999999999993e-05 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.113 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93978 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2670592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.113 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616646+0000", 2024-09-13T07:02:40.113 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.468177445, 2024-09-13T07:02:40.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042128074000000001, 2024-09-13T07:02:40.113 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.113 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.113 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93978 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616646+0000", 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616646+0000", 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616645+0000", 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616657+0000", 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1776999999999999e-05 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616666+0000", 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.363e-06 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616674+0000", 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8839999999999994e-06 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658636+0000", 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041962157 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658706+0000", 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0302000000000006e-05 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658774+0000", 2024-09-13T07:02:40.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7191000000000001e-05 2024-09-13T07:02:40.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.117 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.117 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93979 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.117 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:40.117 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.468143002, 2024-09-13T07:02:40.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042253801000000001, 2024-09-13T07:02:40.117 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93979 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:40.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:40.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616683+0000", 2024-09-13T07:02:40.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2820000000000001e-06 2024-09-13T07:02:40.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616691+0000", 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0779999999999996e-06 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616698+0000", 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658799+0000", 2024-09-13T07:02:40.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042100763999999999 2024-09-13T07:02:40.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658869+0000", 2024-09-13T07:02:40.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0065000000000001e-05 2024-09-13T07:02:40.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658934+0000", 2024-09-13T07:02:40.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5071000000000001e-05 2024-09-13T07:02:40.121 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.121 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.121 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.468118436999999, 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:02:40.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:02:40.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:02:40.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:02:40.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:02:40.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:02:40.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:02:40.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:02:40.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:02:40.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.46809547, 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.126 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:02:40.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:02:40.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:02:40.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:02:40.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:02:40.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:02:40.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:02:40.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:02:40.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.130 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.468078292, 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.131 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:02:40.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:02:40.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:02:40.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.468064427, 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:02:40.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:02:40.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:02:40.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.468051281999999, 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:02:40.139 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.468038075000001, 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.147 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.468024008, 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:02:40.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:40.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:02:40.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:02:40.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.152 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.152 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.152 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:40.152 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.468003994, 2024-09-13T07:02:40.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:02:40.152 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.152 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.153 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:02:40.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.153 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:40.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.156 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:02:40.157 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.467984244, 2024-09-13T07:02:40.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:02:40.157 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.157 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.157 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:02:40.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.157 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:02:40.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:02:40.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:02:40.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:02:40.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:02:40.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:02:40.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:02:40.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:02:40.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:02:40.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.467959437999999, 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:02:40.161 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:02:40.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:02:40.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:02:40.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:02:40.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:02:40.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:02:40.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.467935173000001, 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.170 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.170 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:02:40.170 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.467893439999999, 2024-09-13T07:02:40.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:02:40.170 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.170 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.170 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:02:40.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:02:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:02:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:02:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:02:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:02:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.467857151, 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.174 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:02:40.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:02:40.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:02:40.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:02:40.178 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.178 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.178 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.178 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.178 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.179 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:40.180 INFO:teuthology.orchestra.run.smithi118.stdout:3171 writesame 0xe7f9f6e thru 0xe8050f4 data_size 0x1(0xb187 bytes) 2024-09-13T07:02:40.180 INFO:teuthology.orchestra.run.smithi118.stdout:3174 read 0x71eaf85 thru 0x71f1489 (0x6505 bytes) 2024-09-13T07:02:40.180 INFO:teuthology.orchestra.run.smithi118.stdout:3176 trunc from 0xe8050f5 to 0xa4263f1 2024-09-13T07:02:40.180 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:40.176+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:40.181 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:40.189 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:40.197 INFO:teuthology.orchestra.run.smithi118.stdout:3179 trunc from 0xa4263f1 to 0x6c41b2e 2024-09-13T07:02:40.233 INFO:teuthology.orchestra.run.smithi118.stdout:3182 write 0x9613409 thru 0x961c176 (0x8d6e bytes) 2024-09-13T07:02:40.236 INFO:teuthology.orchestra.run.smithi118.stdout:3183 write 0x4094c18 thru 0x40a2a41 (0xde2a bytes) 2024-09-13T07:02:40.240 INFO:teuthology.orchestra.run.smithi118.stdout:3184 writesame 0x6d08277 thru 0x6d1682a data_size 0x3d(0xe5b4 bytes) 2024-09-13T07:02:40.256 INFO:teuthology.orchestra.run.smithi118.stdout:3185 trunc from 0x961c177 to 0xe51327c 2024-09-13T07:02:40.280 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:40.278+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:40.285 INFO:teuthology.orchestra.run.smithi118.stdout:3187 trunc from 0xe51327c to 0x6e00777 2024-09-13T07:02:40.286 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:123073 3.11 3:88d958e7:::rbd_data.119487223aab.0000000000000002:head [stat out=16b,truncate 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T07:02:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:40.287154+0000", 2024-09-13T07:02:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00062453599999999997, 2024-09-13T07:02:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00065172299999999999, 2024-09-13T07:02:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123073 2024-09-13T07:02:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:40.287154+0000", 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:40.287154+0000", 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:40.287151+0000", 2024-09-13T07:02:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:40.287167+0000", 2024-09-13T07:02:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5631e-05 2024-09-13T07:02:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:40.287209+0000", 2024-09-13T07:02:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2874999999999999e-05 2024-09-13T07:02:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:40.287231+0000", 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1611e-05 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:40.287365+0000", 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013367000000000001 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:40.287629+0000", 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000264505 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-13T07:02:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-13T07:02:40.291 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:40.298 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:40.301 INFO:teuthology.orchestra.run.smithi118.stdout:3188 writesame 0x37534a2 thru 0x375b051 data_size 0x2(0x7bb0 bytes) 2024-09-13T07:02:40.338 INFO:teuthology.orchestra.run.smithi118.stdout:3189 write 0xd26411a thru 0xd2642e0 (0x1c7 bytes) 2024-09-13T07:02:40.341 INFO:teuthology.orchestra.run.smithi118.stdout:3190 write 0x3fcecd7 thru 0x3fd9a79 (0xada3 bytes) 2024-09-13T07:02:40.344 INFO:teuthology.orchestra.run.smithi118.stdout:3191 write 0x56ef6af thru 0x56feeb0 (0xf802 bytes) 2024-09-13T07:02:40.347 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:40.347 INFO:teuthology.orchestra.run.smithi118.stdout:3196 write 0x93331bc thru 0x93365eb (0x3430 bytes) 2024-09-13T07:02:40.350 INFO:teuthology.orchestra.run.smithi118.stdout:3197 write 0xa456146 thru 0xa457459 (0x1314 bytes) 2024-09-13T07:02:40.352 INFO:teuthology.orchestra.run.smithi118.stdout:3199 read 0x2dea6ab thru 0x2def381 (0x4cd7 bytes) 2024-09-13T07:02:40.353 INFO:teuthology.orchestra.run.smithi118.stdout:3200 write 0xb8cf20d thru 0xb8d8ee3 (0x9cd7 bytes) 2024-09-13T07:02:40.356 INFO:teuthology.orchestra.run.smithi118.stdout:3202 read 0xa626144 thru 0xa62e255 (0x8112 bytes) 2024-09-13T07:02:40.357 INFO:teuthology.orchestra.run.smithi118.stdout:3203 read 0x5611374 thru 0x5615061 (0x3cee bytes) 2024-09-13T07:02:40.380 INFO:teuthology.orchestra.run.smithi118.stdout:3204 trunc from 0xd2642e1 to 0x11d74bd 2024-09-13T07:02:40.381 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:40.380+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:40.403 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:40.404 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:40.413 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:40.432 INFO:teuthology.orchestra.run.smithi118.stdout:3205 write 0xe92c1b6 thru 0xe92eecd (0x2d18 bytes) 2024-09-13T07:02:40.438 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:40.438 INFO:teuthology.orchestra.run.smithi118.stdout:3208 read 0xb0d1685 thru 0xb0d948b (0x7e07 bytes) 2024-09-13T07:02:40.439 INFO:teuthology.orchestra.run.smithi118.stdout:3210 writesame 0xd5e5278 thru 0xd5e7ebb data_size 0x2(0x2c44 bytes) 2024-09-13T07:02:40.445 INFO:teuthology.orchestra.run.smithi118.stdout:3212 read 0xac7a636 thru 0xac7acea (0x6b5 bytes) 2024-09-13T07:02:40.446 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:40.446 INFO:teuthology.orchestra.run.smithi118.stdout:3215 write 0xe4e7042 thru 0xe4f3688 (0xc647 bytes) 2024-09-13T07:02:40.452 INFO:teuthology.orchestra.run.smithi118.stdout:3217 read 0x11159d3 thru 0x111c39f (0x69cd bytes) 2024-09-13T07:02:40.452 INFO:teuthology.orchestra.run.smithi118.stdout:3218 read 0x15cf1b thru 0x169983 (0xca69 bytes) 2024-09-13T07:02:40.453 INFO:teuthology.orchestra.run.smithi118.stdout:3219 read 0x46dc02f thru 0x46e2c4b (0x6c1d bytes) 2024-09-13T07:02:40.454 INFO:teuthology.orchestra.run.smithi118.stdout:3221 writesame 0x5cd5f66 thru 0x5cdb5c7 data_size 0x1(0x5662 bytes) 2024-09-13T07:02:40.482 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:40.480+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:40.493 INFO:teuthology.orchestra.run.smithi118.stdout:3226 trunc from 0xe92eece to 0x4e0f285 2024-09-13T07:02:40.516 INFO:teuthology.orchestra.run.smithi118.stdout:3227 read 0x4011f2a thru 0x401eddb (0xceb2 bytes) 2024-09-13T07:02:40.517 INFO:teuthology.orchestra.run.smithi118.stdout:3228 writesame 0x3246c7d thru 0x32530f9 data_size 0x9(0xc47d bytes) 2024-09-13T07:02:40.523 INFO:teuthology.orchestra.run.smithi118.stdout:3229 punch from 0x49be4e8 to 0x49cb780, (0xd298 bytes) 2024-09-13T07:02:40.527 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:02:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.471895907000004, 2024-09-13T07:02:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:02:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:02:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:02:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:02:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:02:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:02:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:02:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:02:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:02:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:02:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:02:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.314014673000003, 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:02:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:02:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:02:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:02:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:02:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:02:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:02:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:02:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.288244962, 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:02:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:02:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:02:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:02:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:02:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:02:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:02:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:02:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:02:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:02:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.267093795999997, 2024-09-13T07:02:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:02:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:02:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:02:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:02:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:02:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:02:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.150049033000002, 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:02:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:02:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:02:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:02:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:02:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:02:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:02:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:02:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:02:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:02:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:02:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:02:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:02:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:02:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:02:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94595 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2318336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.660977+0000", 2024-09-13T07:02:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.865091733, 2024-09-13T07:02:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044248540000000003, 2024-09-13T07:02:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94595 2024-09-13T07:02:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660977+0000", 2024-09-13T07:02:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660977+0000", 2024-09-13T07:02:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660976+0000", 2024-09-13T07:02:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660979+0000", 2024-09-13T07:02:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5229999999999998e-06 2024-09-13T07:02:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660985+0000", 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0039999999999996e-06 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660991+0000", 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8850000000000004e-06 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705116+0000", 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044125315999999998 2024-09-13T07:02:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705183+0000", 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6476000000000005e-05 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705225+0000", 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2651000000000002e-05 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94596 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:02:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.865072553999999, 2024-09-13T07:02:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044330268999999999, 2024-09-13T07:02:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94596 2024-09-13T07:02:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:02:40.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:02:40.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660995+0000", 2024-09-13T07:02:40.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660999+0000", 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2770000000000001e-06 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661007+0000", 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0029999999999994e-06 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661012+0000", 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1279999999999999e-06 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705253+0000", 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044241603999999997 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705293+0000", 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9684999999999997e-05 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705326+0000", 2024-09-13T07:02:40.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2932000000000003e-05 2024-09-13T07:02:40.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.565 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.565 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.565 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:02:40.565 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.865051788000001, 2024-09-13T07:02:40.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:02:40.565 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.565 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.566 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:02:40.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.566 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:02:40.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94598 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:40.570 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:02:40.570 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.864881067000001, 2024-09-13T07:02:40.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044365860999999999, 2024-09-13T07:02:40.570 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.570 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.570 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94598 2024-09-13T07:02:40.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.570 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661034+0000", 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9998465 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661192+0000", 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015813000000000001 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661199+0000", 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5689999999999999e-06 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:02:40.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705452+0000", 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044246273000000003 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705501+0000", 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8748000000000002e-05 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705553+0000", 2024-09-13T07:02:40.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2241999999999999e-05 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107060 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1003520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6133587780000003, 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044341754999999997, 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107060 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912709+0000", 2024-09-13T07:02:40.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912712+0000", 2024-09-13T07:02:40.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6000000000000001e-06 2024-09-13T07:02:40.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912717+0000", 2024-09-13T07:02:40.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3689999999999998e-06 2024-09-13T07:02:40.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912722+0000", 2024-09-13T07:02:40.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7219999999999999e-06 2024-09-13T07:02:40.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956876+0000", 2024-09-13T07:02:40.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044154469000000002 2024-09-13T07:02:40.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956957+0000", 2024-09-13T07:02:40.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1231999999999997e-05 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957051+0000", 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3858999999999997e-05 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6133426499999999, 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:02:40.578 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.579 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.579 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:02:40.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.579 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:02:40.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:02:40.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:02:40.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:02:40.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:02:40.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:02:40.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6133246159999999, 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:02:40.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:02:40.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:02:40.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:02:40.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:02:40.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:02:40.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:02:40.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:02:40.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:02:40.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.587 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.587 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:40.587 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:02:40.587 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6133085359999999, 2024-09-13T07:02:40.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:02:40.587 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.588 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:40.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.588 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:02:40.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.588 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:02:40.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:02:40.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:02:40.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:02:40.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.592 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:40.592 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:02:40.592 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6132928599999996, 2024-09-13T07:02:40.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:02:40.592 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.592 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:40.592 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.592 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.592 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.592 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:02:40.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.595 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6132699380000002, 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:02:40.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:02:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:02:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:02:40.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:02:40.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:02:40.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:02:40.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:02:40.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:02:40.600 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.600 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.600 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.600 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:40.600 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:02:40.600 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6132538410000006, 2024-09-13T07:02:40.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:02:40.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:02:40.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:02:40.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107067 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:02:40.604 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6126938240000008, 2024-09-13T07:02:40.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04441055, 2024-09-13T07:02:40.605 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.605 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:40.605 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.605 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.605 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.605 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107067 2024-09-13T07:02:40.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.605 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:02:40.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912832+0000", 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9994574 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913379+0000", 2024-09-13T07:02:40.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00054719999999999997 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913386+0000", 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4549999999999998e-06 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913393+0000", 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6259999999999997e-06 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957776+0000", 2024-09-13T07:02:40.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044383143999999999 2024-09-13T07:02:40.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957785+0000", 2024-09-13T07:02:40.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9220000000000003e-06 2024-09-13T07:02:40.608 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.608 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.608 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.608 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107068 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:40.608 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6126693930000009, 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044410502999999997, 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107068 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:02:40.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913401+0000", 2024-09-13T07:02:40.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:02:40.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913407+0000", 2024-09-13T07:02:40.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7520000000000002e-06 2024-09-13T07:02:40.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913412+0000", 2024-09-13T07:02:40.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8550000000000001e-06 2024-09-13T07:02:40.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957802+0000", 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044389606999999998 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957809+0000", 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9160000000000001e-06 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.612 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:40.613 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:02:40.613 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.612652229, 2024-09-13T07:02:40.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:02:40.613 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.613 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:40.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.613 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:02:40.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.613 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:02:40.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:02:40.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:02:40.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:02:40.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:02:40.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:02:40.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:02:40.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:02:40.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:02:40.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:02:40.616 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.616 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.616 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6126094529999992, 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.617 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:02:40.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:40.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.620 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:40.621 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:40.621 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:40.623 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:40.623 INFO:teuthology.orchestra.run.smithi118.stdout:3230 write 0x896877a thru 0x8975c9b (0xd522 bytes) 2024-09-13T07:02:40.624 INFO:teuthology.orchestra.run.smithi118.stdout:3231 write 0xccbbf1a thru 0xccc5f64 (0xa04b bytes) 2024-09-13T07:02:40.624 INFO:teuthology.orchestra.run.smithi118.stdout:3232 write 0xba4f2a4 thru 0xba5c566 (0xd2c3 bytes) 2024-09-13T07:02:40.624 INFO:teuthology.orchestra.run.smithi118.stdout:3233 writesame 0x6ac3d8f thru 0x6acba6b data_size 0x5(0x7cdd bytes) 2024-09-13T07:02:40.624 INFO:teuthology.orchestra.run.smithi118.stdout:3234 read 0x9c64a6d thru 0x9c690c7 (0x465b bytes) 2024-09-13T07:02:40.624 INFO:teuthology.orchestra.run.smithi118.stdout:3235 write 0xb9ed00c thru 0xb9eeb1a (0x1b0f bytes) 2024-09-13T07:02:40.624 INFO:teuthology.orchestra.run.smithi118.stdout:3236 write 0xe885397 thru 0xe8886e6 (0x3350 bytes) 2024-09-13T07:02:40.624 INFO:teuthology.orchestra.run.smithi118.stdout:3237 write 0x6d67144 thru 0x6d71b02 (0xa9bf bytes) 2024-09-13T07:02:40.624 INFO:teuthology.orchestra.run.smithi118.stdout:3238 write 0x1ef260d thru 0x1ef853e (0x5f32 bytes) 2024-09-13T07:02:40.624 INFO:teuthology.orchestra.run.smithi118.stdout:3239 write 0xb1743c7 thru 0xb176154 (0x1d8e bytes) 2024-09-13T07:02:40.624 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:40.620+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:40.625 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:40.625 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:40.626 INFO:teuthology.orchestra.run.smithi118.stdout:3242 punch from 0x938a907 to 0x938d845, (0x2f3e bytes) 2024-09-13T07:02:40.626 INFO:teuthology.orchestra.run.smithi118.stdout:3243 punch from 0x7c15c6c to 0x7c25358, (0xf6ec bytes) 2024-09-13T07:02:40.626 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:40.626 INFO:teuthology.orchestra.run.smithi118.stdout:3245 flatten 2024-09-13T07:02:40.638 INFO:teuthology.orchestra.run.smithi118.stdout:3248 read 0xd6e8cd1 thru 0xd6ee5d9 (0x5909 bytes) 2024-09-13T07:02:40.640 INFO:teuthology.orchestra.run.smithi118.stdout:3249 write 0x49b3479 thru 0x49c2f82 (0xfb0a bytes) 2024-09-13T07:02:40.643 INFO:teuthology.orchestra.run.smithi118.stdout:3250 write 0xb1cf897 thru 0xb1d90f1 (0x985b bytes) 2024-09-13T07:02:40.646 INFO:teuthology.orchestra.run.smithi118.stdout:3251 write 0x3d22966 thru 0x3d2ea63 (0xc0fe bytes) 2024-09-13T07:02:40.677 INFO:teuthology.orchestra.run.smithi118.stdout:3253 trunc from 0xe8886e7 to 0x3300c69 2024-09-13T07:02:40.704 INFO:teuthology.orchestra.run.smithi118.stdout:3254 read 0x2dfbb88 thru 0x2dfc611 (0xa8a bytes) 2024-09-13T07:02:40.705 INFO:teuthology.orchestra.run.smithi118.stdout:3255 clone 32 order 20 su 4096 sc 8 2024-09-13T07:02:40.723 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:40.720+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:40.751 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:40.752 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:40.752 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:40.752 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:40.761 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:40.825 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:40.820+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:40.916 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:40.916 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:40.916 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:40.916 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:40.916 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:40.926 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:40.926 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:40.922+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:41.028 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:41.028 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:41.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:41.028 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:41.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.028 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:41.028 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:41.028 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 75.523371972000007, 2024-09-13T07:02:41.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:02:41.028 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:02:41.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:02:41.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:02:41.034 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.034 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.034 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.034 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:41.034 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:41.034 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 75.498544261999996, 2024-09-13T07:02:41.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:02:41.034 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.034 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:41.034 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.035 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.035 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.035 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:02:41.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.035 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:41.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:41.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:02:41.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:41.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.036 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.036 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:02:41.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:02:41.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:02:41.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:02:41.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:02:41.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.041 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:41.041 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:41.041 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.959872438000005, 2024-09-13T07:02:41.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:02:41.041 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.041 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:41.041 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.041 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.041 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.041 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:02:41.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:41.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:02:41.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:02:41.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:02:41.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:02:41.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:02:41.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:02:41.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:02:41.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:02:41.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:02:41.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:02:41.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:41.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:02:41.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:02:41.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:41.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:02:41.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.930247245000004, 2024-09-13T07:02:41.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:02:41.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:02:41.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:02:41.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.397533613999997, 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.053 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:02:41.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:02:41.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:41.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.058 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.059 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:41.059 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:41.059 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.397451157999996, 2024-09-13T07:02:41.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:02:41.059 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.059 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:41.059 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.059 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.059 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.059 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:02:41.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.060 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:41.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:41.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:02:41.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:41.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:02:41.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:02:41.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:02:41.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:41.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:02:41.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:02:41.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:41.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:02:41.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:02:41.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:41.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:02:41.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:02:41.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:02:41.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:02:41.065 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.065 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.065 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.065 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:41.065 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:41.065 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.397419348, 2024-09-13T07:02:41.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:02:41.065 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:02:41.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:02:41.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:02:41.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:41.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:02:41.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:02:41.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:41.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:02:41.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:02:41.071 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.071 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.071 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.071 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:41.071 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:41.071 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.397364190000005, 2024-09-13T07:02:41.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:02:41.071 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.071 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:41.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:02:41.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:41.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:02:41.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:02:41.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:02:41.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:02:41.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:02:41.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:41.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:02:41.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:02:41.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:41.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:02:41.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:02:41.077 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.077 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.077 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.077 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:41.077 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:41.077 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.397318910999999, 2024-09-13T07:02:41.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:02:41.077 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.077 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:41.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.082 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.397280377000001, 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.083 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:02:41.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:41.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:02:41.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:02:41.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:02:41.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:02:41.088 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.088 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.088 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.088 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.397218757999994, 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:02:41.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:02:41.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.382996074999994, 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:02:41.094 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:02:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:02:41.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.371117554999998, 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.100 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:02:41.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.370560494000003, 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:02:41.104 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:41.108 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.370538339000007, 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:41.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:02:41.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:02:41.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:02:41.111 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:41.024+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:41.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:02:41.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:02:41.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:02:41.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:02:41.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.370518622000006, 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.114 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:02:41.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:02:41.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.370501292, 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:02:41.118 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:02:41.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.122 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.370205630000001, 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.126 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.809002810999999, 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:02:41.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.132 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.776923908000001, 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:41.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:02:41.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:02:41.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:02:41.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:02:41.136 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.136 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.136 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.136 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.136 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.136 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:41.137 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:41.138 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:41.134+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:41.239 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:41.236+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:41.269 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:41.269 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:41.279 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:41.340 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:41.338+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:41.387 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:41.388 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:41.388 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:41.388 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:41.388 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:41.398 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:41.440 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:41.436+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:41.478 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:02:41.508 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:41.508 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:41.508 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:41.508 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:41.508 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 58.015528512000003, 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.509 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.510 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.511 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:02:41.512 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.513 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.514 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.514 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:41.514 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.529044947999999, 2024-09-13T07:02:41.514 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:02:41.514 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.514 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:41.514 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.514 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.514 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.514 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:02:41.514 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.514 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.514 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.516 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.517 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.528975019000001, 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.518 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:02:41.520 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:02:41.521 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:02:41.522 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.522 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.522 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.522 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.522 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.522 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.522 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:41.522 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.528959147000002, 2024-09-13T07:02:41.522 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:02:41.522 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.522 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:41.522 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.522 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:02:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:41.523 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.524 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.524 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.524 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.524 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:02:41.524 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.524 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.524 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.524 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.524 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:02:41.524 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:02:41.524 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.524 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.524 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.525 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:02:41.525 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:02:41.525 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.525 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.525 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.525 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:02:41.525 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:02:41.525 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.525 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.525 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.525 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:02:41.525 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:02:41.525 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.526 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.526 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.526 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:02:41.526 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:02:41.526 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.526 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.526 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.526 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:02:41.526 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:02:41.526 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.526 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.526 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.526 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.527 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.527 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.527 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:41.527 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.528888690000002, 2024-09-13T07:02:41.527 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:02:41.527 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.527 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:41.527 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.527 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.527 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.527 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:02:41.527 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.527 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.528 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.528 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.528 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:41.528 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.528 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.528 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.528 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.528 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:41.528 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.528 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.528 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.528 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.528 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.529 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.530 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525536652, 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.531 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:02:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:02:41.534 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.534 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.534 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.534 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:02:41.534 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:02:41.534 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.534 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.534 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.534 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:02:41.534 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:02:41.534 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.535 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.535 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:02:41.535 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:02:41.535 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.535 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.535 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.535 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.535 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:41.535 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525520350999997, 2024-09-13T07:02:41.535 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.537 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:02:41.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.539 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.539 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:02:41.539 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:02:41.539 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.539 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.539 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.539 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.539 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:41.539 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525500565999998, 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:02:41.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:41.543 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525484292999998, 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:02:41.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.546 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:41.547 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525466467999998, 2024-09-13T07:02:41.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:02:41.548 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.548 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.548 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.548 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.548 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.548 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:02:41.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.548 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:41.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.551 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:41.552 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525450374999998, 2024-09-13T07:02:41.552 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:02:41.552 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.552 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.552 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.552 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.552 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.552 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:02:41.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.552 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.552 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.552 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:41.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.553 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:41.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.553 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:02:41.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.555 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.555 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:02:41.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:02:41.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.555 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.555 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:02:41.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:02:41.555 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.555 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.555 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.556 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.556 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:41.556 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525429189, 2024-09-13T07:02:41.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:02:41.556 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.556 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.556 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.556 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.556 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.556 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:02:41.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.556 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.557 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:41.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.557 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:41.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.557 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:02:41.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:02:41.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:02:41.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:02:41.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:02:41.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.559 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.560 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.560 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.560 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:41.560 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525413155999999, 2024-09-13T07:02:41.560 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:02:41.560 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.560 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.560 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.560 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.560 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.560 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.563 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.564 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.564 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.564 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:41.564 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525363917, 2024-09-13T07:02:41.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:02:41.564 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.564 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.565 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.565 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.565 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.565 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:02:41.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.565 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:41.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:02:41.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:02:41.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:02:41.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:02:41.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:02:41.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525345856999998, 2024-09-13T07:02:41.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:02:41.569 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.569 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.569 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.569 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.569 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.569 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:02:41.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.569 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:41.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.572 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525329341999999, 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:41.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525310976999997, 2024-09-13T07:02:41.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:02:41.577 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.577 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.577 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.577 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.577 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.577 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:02:41.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.577 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:41.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:41.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:41.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:02:41.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:02:41.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:02:41.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:02:41.580 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.580 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.580 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.580 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525294879, 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:02:41.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:02:41.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:02:41.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:02:41.584 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.584 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.584 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.585 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.585 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:41.585 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525278944, 2024-09-13T07:02:41.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:02:41.585 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.585 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.585 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.585 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.585 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.585 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:02:41.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.585 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:02:41.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:02:41.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:02:41.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:02:41.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:02:41.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:02:41.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:02:41.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:02:41.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:02:41.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:02:41.588 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.588 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 43.525262447000003, 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:02:41.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.590 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:41.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:41.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:41.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:41.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:41.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:41.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:41.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:02:41.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:02:41.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:41.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:02:41.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:02:41.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:41.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:41.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:41.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:02:41.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:02:41.593 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.593 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.593 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.593 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:41.593 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:41.593 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:41.594 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:41.595 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:41.590+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:41.682 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-13T07:02:41.695 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:41.694+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:41.754 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:41.754 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:41.764 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:41.796 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:41.794+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:41.799 INFO:teuthology.orchestra.run.smithi118.stdout:3256 trunc from 0x3300c69 to 0x1578878 2024-09-13T07:02:41.859 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:02:41.881 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:41.881 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:41.881 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:41.895 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:41.927 INFO:teuthology.orchestra.run.smithi118.stdout:3257 read 0x88087 thru 0x8bbc7 (0x3b41 bytes) 2024-09-13T07:02:41.927 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:41.924+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:41.952 INFO:teuthology.orchestra.run.smithi118.stdout:3260 write 0x8a22ced thru 0x8a24ee2 (0x21f6 bytes) 2024-09-13T07:02:41.963 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:41.964 INFO:teuthology.orchestra.run.smithi118.stdout:3263 punch from 0x3067212 to 0x306e8d2, (0x76c0 bytes) 2024-09-13T07:02:41.964 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:41.964 INFO:teuthology.orchestra.run.smithi118.stdout:3267 writesame 0x4f6620f thru 0x4f6efab data_size 0x1(0x8d9d bytes) 2024-09-13T07:02:41.986 INFO:teuthology.orchestra.run.smithi118.stdout:3268 read 0x5acaf00 thru 0x5acb2da (0x3db bytes) 2024-09-13T07:02:41.998 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:41.994+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:41.999 INFO:teuthology.orchestra.run.smithi118.stdout:3269 write 0xad34c85 thru 0xad4260e (0xd98a bytes) 2024-09-13T07:02:42.007 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:42.030 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:42.031 INFO:teuthology.orchestra.run.smithi118.stdout:3270 trunc from 0xad4260f to 0xf05010 2024-09-13T07:02:42.048 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:42.049 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:42.049 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:42.062 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:42.067 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:42.113 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:42.112+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:42.115 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:42.163 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:42.178 INFO:teuthology.orchestra.run.smithi118.stdout:3271 punch from 0x1bb2c5 to 0x1c5331, (0xa06c bytes) 2024-09-13T07:02:42.190 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:42.192 INFO:teuthology.orchestra.run.smithi118.stdout:3272 writesame 0x348e3fa thru 0x348e948 data_size 0x3(0x54f bytes) 2024-09-13T07:02:42.192 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:42.192 INFO:teuthology.orchestra.run.smithi118.stdout:3275 read 0x59affd thru 0x5a4b7f (0x9b83 bytes) 2024-09-13T07:02:42.194 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:42.194 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:42.194 INFO:teuthology.orchestra.run.smithi118.stdout:3276 read 0x9bd8dc thru 0x9c8b45 (0xb26a bytes) 2024-09-13T07:02:42.196 INFO:teuthology.orchestra.run.smithi118.stdout:3277 punch from 0x317a5ff to 0x31879a9, (0xd3aa bytes) 2024-09-13T07:02:42.200 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:42.198+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:42.205 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:42.209 INFO:teuthology.orchestra.run.smithi118.stdout:3278 write 0x50060aa thru 0x5014069 (0xdfc0 bytes) 2024-09-13T07:02:42.212 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:42.215 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:42.215 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:42.215 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:42.215 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:42.215 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:42.215 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:42.216 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:42.216 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:42.232 INFO:teuthology.orchestra.run.smithi118.stdout:3280 trunc from 0x501406a to 0x1287aed 2024-09-13T07:02:42.250 INFO:teuthology.orchestra.run.smithi118.stdout:3281 trunc from 0x1287aed to 0x8d0349 2024-09-13T07:02:42.266 INFO:teuthology.orchestra.run.smithi118.stdout:3282 read 0x32f347 thru 0x33ebb7 (0xf871 bytes) 2024-09-13T07:02:42.294 INFO:teuthology.orchestra.run.smithi118.stdout:3285 write 0x6a5db5f thru 0x6a66c45 (0x90e7 bytes) 2024-09-13T07:02:42.302 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:42.298+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:42.310 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:42.310 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:42.319 INFO:teuthology.orchestra.run.smithi118.stdout:3286 trunc from 0x6a66c46 to 0xb161eb7 2024-09-13T07:02:42.322 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:42.330 INFO:teuthology.orchestra.run.smithi118.stdout:3287 writesame 0x1d528f5 thru 0x1d5ea46 data_size 0x31(0xc152 bytes) 2024-09-13T07:02:42.354 INFO:teuthology.orchestra.run.smithi118.stdout:3288 trunc from 0xb161eb7 to 0xbbe70d1 2024-09-13T07:02:42.363 INFO:teuthology.orchestra.run.smithi118.stdout:3289 punch from 0x959cf3 to 0x966a09, (0xcd16 bytes) 2024-09-13T07:02:42.384 INFO:teuthology.orchestra.run.smithi118.stdout:3290 trunc from 0xbbe70d1 to 0x122766e 2024-09-13T07:02:42.402 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:42.398+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:42.408 INFO:teuthology.orchestra.run.smithi118.stdout:3293 read 0xeede71 thru 0xef835e (0xa4ee bytes) 2024-09-13T07:02:42.409 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:42.409 INFO:teuthology.orchestra.run.smithi118.stdout:3295 read 0xc074d thru 0xd01ab (0xfa5f bytes) 2024-09-13T07:02:42.414 INFO:teuthology.orchestra.run.smithi118.stdout:3296 read 0xf6f72b thru 0xf7c8c5 (0xd19b bytes) 2024-09-13T07:02:42.415 INFO:teuthology.orchestra.run.smithi118.stdout:3298 writesame 0x10faba7 thru 0x1102c67 data_size 0x1(0x80c1 bytes) 2024-09-13T07:02:42.434 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:42.434 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:42.445 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:42.451 INFO:teuthology.orchestra.run.smithi118.stdout:3299 write 0x2db874f thru 0x2dc0ff0 (0x88a2 bytes) 2024-09-13T07:02:42.458 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7] out_osds: [2, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:02:42.458 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:02:42.458 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-13T07:02:42.458 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-13T07:02:42.473 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:02:42.474 INFO:teuthology.orchestra.run.smithi118.stdout:3301 trunc from 0x2dc0ff1 to 0x205399c 2024-09-13T07:02:42.489 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:42.489 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:42.489 INFO:teuthology.orchestra.run.smithi118.stdout:3304 read 0x1544909 thru 0x1546108 (0x1800 bytes) 2024-09-13T07:02:42.502 INFO:teuthology.orchestra.run.smithi118.stdout:3305 write 0x2c565eb thru 0x2c5fe1e (0x9834 bytes) 2024-09-13T07:02:42.503 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:42.502+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:42.527 INFO:teuthology.orchestra.run.smithi118.stdout:3307 write 0x2e47a5e thru 0x2e52385 (0xa928 bytes) 2024-09-13T07:02:42.532 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:42.532 INFO:teuthology.orchestra.run.smithi118.stdout:3312 read 0x2c9f885 thru 0x2ca6864 (0x6fe0 bytes) 2024-09-13T07:02:42.533 INFO:teuthology.orchestra.run.smithi118.stdout:3313 punch from 0x7d04b0 to 0x7dc0dd, (0xbc2d bytes) 2024-09-13T07:02:42.552 INFO:teuthology.orchestra.run.smithi118.stdout:3314 trunc from 0x2e52386 to 0xcbfee6d 2024-09-13T07:02:42.562 INFO:teuthology.orchestra.run.smithi118.stdout:3315 read 0x964710a thru 0x964cacd (0x59c4 bytes) 2024-09-13T07:02:42.563 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:42.564 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:42.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:42.577 INFO:teuthology.orchestra.run.smithi118.stdout:3316 trunc from 0xcbfee6d to 0x2e50e58 2024-09-13T07:02:42.603 INFO:teuthology.orchestra.run.smithi118.stdout:3317 trunc from 0x2e50e58 to 0x9d4ef1e 2024-09-13T07:02:42.603 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:42.600+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:42.621 INFO:teuthology.orchestra.run.smithi118.stdout:3318 trunc from 0x9d4ef1e to 0xeb683a0 2024-09-13T07:02:42.631 INFO:teuthology.orchestra.run.smithi118.stdout:3319 punch from 0x7922dab to 0x792fcbb, (0xcf10 bytes) 2024-09-13T07:02:42.631 INFO:teuthology.orchestra.run.smithi118.stdout:3320 read 0xe6a1ddb thru 0xe6ad187 (0xb3ad bytes) 2024-09-13T07:02:42.631 INFO:teuthology.orchestra.run.smithi118.stdout:3321 read 0xda9ed68 thru 0xdaa2dc2 (0x405b bytes) 2024-09-13T07:02:42.631 INFO:teuthology.orchestra.run.smithi118.stdout:3322 write 0x1611544 thru 0x161c1b0 (0xac6d bytes) 2024-09-13T07:02:42.637 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:42.638 INFO:teuthology.orchestra.run.smithi118.stdout:3324 read 0x6c31986 thru 0x6c40148 (0xe7c3 bytes) 2024-09-13T07:02:42.638 INFO:teuthology.orchestra.run.smithi118.stdout:3325 read 0x50c8b7e thru 0x50cb096 (0x2519 bytes) 2024-09-13T07:02:42.638 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:42.638 INFO:teuthology.orchestra.run.smithi118.stdout:3329 read 0x4a90c40 thru 0x4a94f2b (0x42ec bytes) 2024-09-13T07:02:42.638 INFO:teuthology.orchestra.run.smithi118.stdout:3330 writesame 0x9719b6 thru 0x974b65 data_size 0x28(0x31b0 bytes) 2024-09-13T07:02:42.640 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:42.644 INFO:teuthology.orchestra.run.smithi118.stdout:3332 trunc from 0xeb683a0 to 0xd29484a 2024-09-13T07:02:42.653 INFO:teuthology.orchestra.run.smithi118.stdout:3334 write 0x87a5a2f thru 0x87ace25 (0x73f7 bytes) 2024-09-13T07:02:42.683 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:42.683 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:42.686 INFO:teuthology.orchestra.run.smithi118.stdout:3335 trunc from 0xd29484a to 0x3c0c5c4 2024-09-13T07:02:42.694 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:42.703 INFO:teuthology.orchestra.run.smithi118.stdout:3338 read 0x18014c8 thru 0x1804cfa (0x3833 bytes) 2024-09-13T07:02:42.704 INFO:teuthology.orchestra.run.smithi118.stdout:3339 read 0x3bdd845 thru 0x3bed673 (0xfe2f bytes) 2024-09-13T07:02:42.704 INFO:teuthology.orchestra.run.smithi118.stdout:3340 write 0x2a653f5 thru 0x2a6e72c (0x9338 bytes) 2024-09-13T07:02:42.737 INFO:teuthology.orchestra.run.smithi118.stdout:3342 write 0xed4f189 thru 0xed54ac0 (0x5938 bytes) 2024-09-13T07:02:42.743 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:42.742+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:42.751 INFO:teuthology.orchestra.run.smithi118.stdout:3344 writesame 0xb497b3a thru 0xb49b251 data_size 0x52(0x3718 bytes) 2024-09-13T07:02:42.760 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:42.760 INFO:teuthology.orchestra.run.smithi118.stdout:3346 writesame 0x627508b thru 0x627974c data_size 0x3(0x46c2 bytes) 2024-09-13T07:02:42.775 INFO:teuthology.orchestra.run.smithi118.stdout:3347 read 0xe82ca39 thru 0xe82eae6 (0x20ae bytes) 2024-09-13T07:02:42.775 INFO:teuthology.orchestra.run.smithi118.stdout:3349 read 0x22fdc78 thru 0x2305ed0 (0x8259 bytes) 2024-09-13T07:02:42.776 INFO:teuthology.orchestra.run.smithi118.stdout:3352 read 0x4f0eaf thru 0x4f4d81 (0x3ed3 bytes) 2024-09-13T07:02:42.777 INFO:teuthology.orchestra.run.smithi118.stdout:3353 write 0x5ce3bf4 thru 0x5cf2881 (0xec8e bytes) 2024-09-13T07:02:42.780 INFO:tasks.ceph.mon.a.smithi118.stderr:2024-09-13T07:02:42.776+0000 7fd1ec319700 -1 mon.a@0(leader).osd e97 definitely_dead 0 2024-09-13T07:02:42.792 INFO:teuthology.orchestra.run.smithi118.stdout:3354 write 0x9b912d5 thru 0x9b91744 (0x470 bytes) 2024-09-13T07:02:42.795 INFO:teuthology.orchestra.run.smithi118.stdout:3355 write 0x73fbdb4 thru 0x73fd5f5 (0x1842 bytes) 2024-09-13T07:02:42.804 INFO:teuthology.orchestra.run.smithi118.stdout:3357 punch from 0x3164f48 to 0x31697ba, (0x4872 bytes) 2024-09-13T07:02:42.804 INFO:teuthology.orchestra.run.smithi118.stdout:3358 writesame 0xc134b17 thru 0xc139c56 data_size 0x50(0x5140 bytes) 2024-09-13T07:02:42.805 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:42.802+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:42.812 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:42.813 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:42.816 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:42.816 INFO:teuthology.orchestra.run.smithi118.stdout:3361 writesame 0x270f26f thru 0x27187a7 data_size 0x1(0x9539 bytes) 2024-09-13T07:02:42.824 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:42.842 INFO:teuthology.orchestra.run.smithi118.stdout:3362 trunc from 0xed54ac1 to 0xcb498c8 2024-09-13T07:02:42.860 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:42.860 INFO:teuthology.orchestra.run.smithi118.stdout:3366 write 0x2c53d72 thru 0x2c580af (0x433e bytes) 2024-09-13T07:02:42.865 INFO:teuthology.orchestra.run.smithi118.stdout:3367 writesame 0x79db982 thru 0x79e8bc9 data_size 0xc(0xd248 bytes) 2024-09-13T07:02:42.887 INFO:teuthology.orchestra.run.smithi118.stdout:3368 read 0x59f6031 thru 0x5a00c34 (0xac04 bytes) 2024-09-13T07:02:42.887 INFO:teuthology.orchestra.run.smithi118.stdout:3369 write 0xad25928 thru 0xad3568e (0xfd67 bytes) 2024-09-13T07:02:42.905 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:42.904+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:42.907 INFO:teuthology.orchestra.run.smithi118.stdout:3371 read 0x3b5b889 thru 0x3b60806 (0x4f7e bytes) 2024-09-13T07:02:42.907 INFO:teuthology.orchestra.run.smithi118.stdout:3372 write 0xbe24c25 thru 0xbe32df9 (0xe1d5 bytes) 2024-09-13T07:02:42.922 INFO:teuthology.orchestra.run.smithi118.stdout:3373 read 0x297fb86 thru 0x29831bc (0x3637 bytes) 2024-09-13T07:02:42.923 INFO:teuthology.orchestra.run.smithi118.stdout:3374 read 0x1725a35 thru 0x172c0f5 (0x66c1 bytes) 2024-09-13T07:02:42.924 INFO:teuthology.orchestra.run.smithi118.stdout:3375 read 0x35bf118 thru 0x35ca0b3 (0xaf9c bytes) 2024-09-13T07:02:42.924 INFO:teuthology.orchestra.run.smithi118.stdout:3377 read 0x23bcee9 thru 0x23ca0ac (0xd1c4 bytes) 2024-09-13T07:02:42.938 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:42.938 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:42.941 INFO:teuthology.orchestra.run.smithi118.stdout:3378 trunc from 0xcb498c8 to 0x599062f 2024-09-13T07:02:42.951 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:42.974 INFO:teuthology.orchestra.run.smithi118.stdout:3380 read 0x4a56001 thru 0x4a62353 (0xc353 bytes) 2024-09-13T07:02:42.998 INFO:teuthology.orchestra.run.smithi118.stdout:3381 write 0xb0ab317 thru 0xb0b2c62 (0x794c bytes) 2024-09-13T07:02:43.006 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:43.004+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:43.014 INFO:teuthology.orchestra.run.smithi118.stdout:3382 write 0x1b062a5 thru 0x1b0f60f (0x936b bytes) 2024-09-13T07:02:43.043 INFO:teuthology.orchestra.run.smithi118.stdout:3384 writesame 0xbdcd2ab thru 0xbdd7e04 data_size 0x9(0xab5a bytes) 2024-09-13T07:02:43.043 INFO:teuthology.orchestra.run.smithi118.stdout:3385 write 0x74f33f5 thru 0x74ff09e (0xbcaa bytes) 2024-09-13T07:02:43.057 INFO:teuthology.orchestra.run.smithi118.stdout:3386 punch from 0x7ed1f76 to 0x7ed6d5c, (0x4de6 bytes) 2024-09-13T07:02:43.057 INFO:teuthology.orchestra.run.smithi118.stdout:3387 read 0x9f2e9bb thru 0x9f39706 (0xad4c bytes) 2024-09-13T07:02:43.066 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:43.066 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:43.073 INFO:teuthology.orchestra.run.smithi118.stdout:3389 trunc from 0xbdd7e05 to 0x5781605 2024-09-13T07:02:43.076 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:43.099 INFO:teuthology.orchestra.run.smithi118.stdout:3390 punch from 0x366173f to 0x366256c, (0xe2d bytes) 2024-09-13T07:02:43.107 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:43.104+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:43.109 INFO:teuthology.orchestra.run.smithi118.stdout:3391 writesame 0x5ab061a thru 0x5abf1f0 data_size 0xa1(0xebd7 bytes) 2024-09-13T07:02:43.127 INFO:teuthology.orchestra.run.smithi118.stdout:3394 trunc from 0x5abf1f1 to 0xd9ba841 2024-09-13T07:02:43.138 INFO:teuthology.orchestra.run.smithi118.stdout:3395 read 0xa7f59fe thru 0xa7fb42f (0x5a32 bytes) 2024-09-13T07:02:43.139 INFO:teuthology.orchestra.run.smithi118.stdout:3396 read 0x7197d30 thru 0x71a048d (0x875e bytes) 2024-09-13T07:02:43.139 INFO:teuthology.orchestra.run.smithi118.stdout:3397 writesame 0xd008543 thru 0xd009653 data_size 0x1(0x1111 bytes) 2024-09-13T07:02:43.149 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:43.149 INFO:teuthology.orchestra.run.smithi118.stdout:3399 write 0x64bb433 thru 0x64c34b4 (0x8082 bytes) 2024-09-13T07:02:43.167 INFO:teuthology.orchestra.run.smithi118.stdout:3400 read 0x4f836fe thru 0x4f87d09 (0x460c bytes) 2024-09-13T07:02:43.181 INFO:teuthology.orchestra.run.smithi118.stdout:3401 trunc from 0xd9ba841 to 0x76d8963 2024-09-13T07:02:43.182 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:43.182 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:43.193 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:43.208 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:43.204+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:43.230 INFO:teuthology.orchestra.run.smithi118.stdout:3403 write 0xe3b42de thru 0xe3bfa90 (0xb7b3 bytes) 2024-09-13T07:02:43.251 INFO:teuthology.orchestra.run.smithi118.stdout:3404 read 0x1070495 thru 0x107f0c0 (0xec2c bytes) 2024-09-13T07:02:43.266 INFO:teuthology.orchestra.run.smithi118.stdout:3406 trunc from 0xe3bfa91 to 0x8359378 2024-09-13T07:02:43.287 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:43.287 INFO:teuthology.orchestra.run.smithi118.stdout:3409 writesame 0x6616d05 thru 0x6624756 data_size 0xcf(0xda52 bytes) 2024-09-13T07:02:43.291 INFO:teuthology.orchestra.run.smithi118.stdout:3410 write 0xf3c5ee thru 0xf44e83 (0x8896 bytes) 2024-09-13T07:02:43.296 INFO:teuthology.orchestra.run.smithi118.stdout:3411 write 0x416d0b6 thru 0x416f8c4 (0x280f bytes) 2024-09-13T07:02:43.302 INFO:teuthology.orchestra.run.smithi118.stdout:3412 read 0x4d9d46c thru 0x4da3d52 (0x68e7 bytes) 2024-09-13T07:02:43.303 INFO:teuthology.orchestra.run.smithi118.stdout:3414 punch from 0x2fa202 to 0x303f89, (0x9d87 bytes) 2024-09-13T07:02:43.303 INFO:teuthology.orchestra.run.smithi118.stdout:3415 write 0x7fddd6e thru 0x7fdde2d (0xc0 bytes) 2024-09-13T07:02:43.309 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:43.308+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:43.312 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:43.312 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:43.317 INFO:teuthology.orchestra.run.smithi118.stdout:3416 write 0x8d46cf2 thru 0x8d5274a (0xba59 bytes) 2024-09-13T07:02:43.324 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:43.336 INFO:teuthology.orchestra.run.smithi118.stdout:3417 punch from 0x167a01f to 0x1684ab1, (0xaa92 bytes) 2024-09-13T07:02:43.337 INFO:teuthology.orchestra.run.smithi118.stdout:3419 punch from 0x1eaf068 to 0x1eb03cc, (0x1364 bytes) 2024-09-13T07:02:43.337 INFO:teuthology.orchestra.run.smithi118.stdout:3420 read 0x493c3c3 thru 0x4940266 (0x3ea4 bytes) 2024-09-13T07:02:43.351 INFO:teuthology.orchestra.run.smithi118.stdout:3424 trunc from 0x8d5274b to 0x2a2d56d 2024-09-13T07:02:43.380 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:43.381 INFO:teuthology.orchestra.run.smithi118.stdout:3428 write 0x24b87ba thru 0x24c2149 (0x9990 bytes) 2024-09-13T07:02:43.385 INFO:teuthology.orchestra.run.smithi118.stdout:3429 read 0x1f8b39c thru 0x1f8fd66 (0x49cb bytes) 2024-09-13T07:02:43.386 INFO:teuthology.orchestra.run.smithi118.stdout:3430 punch from 0x1334578 to 0x133d8d7, (0x935f bytes) 2024-09-13T07:02:43.405 INFO:teuthology.orchestra.run.smithi118.stdout:3431 write 0x5612dc2 thru 0x562103f (0xe27e bytes) 2024-09-13T07:02:43.410 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:43.406+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:43.420 INFO:teuthology.orchestra.run.smithi118.stdout:3432 write 0x49703c4 thru 0x497286e (0x24ab bytes) 2024-09-13T07:02:43.427 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:43.428 INFO:teuthology.orchestra.run.smithi118.stdout:3434 punch from 0x2f7b9e9 to 0x2f7baad, (0xc4 bytes) 2024-09-13T07:02:43.428 INFO:teuthology.orchestra.run.smithi118.stdout:3435 read 0x1a1426a thru 0x1a20238 (0xbfcf bytes) 2024-09-13T07:02:43.429 INFO:teuthology.orchestra.run.smithi118.stdout:3436 read 0x3e24ae4 thru 0x3e28b1d (0x403a bytes) 2024-09-13T07:02:43.434 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:43.435 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:43.438 INFO:teuthology.orchestra.run.smithi118.stdout:3437 trunc from 0x5621040 to 0x94e797a 2024-09-13T07:02:43.444 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:43.449 INFO:teuthology.orchestra.run.smithi118.stdout:3438 read 0x403be2d thru 0x404955b (0xd72f bytes) 2024-09-13T07:02:43.449 INFO:teuthology.orchestra.run.smithi118.stdout:3439 write 0x72b5947 thru 0x72bd549 (0x7c03 bytes) 2024-09-13T07:02:43.469 INFO:teuthology.orchestra.run.smithi118.stdout:3440 trunc from 0x94e797a to 0x7fb7179 2024-09-13T07:02:43.484 INFO:teuthology.orchestra.run.smithi118.stderr:marked in osd.2. 2024-09-13T07:02:43.498 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-13T07:02:43.541 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:43.540+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:43.582 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:43.582 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:43.591 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:43.611 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:43.610+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:43.695 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:43.695 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:43.705 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:43.712 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:43.708+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:43.812 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:43.813 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:43.813 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:43.812+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:43.822 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:43.914 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:43.912+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:43.926 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:43.926 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:43.937 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:44.015 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:44.014+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:44.054 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:44.054 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:44.064 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:44.115 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:44.114+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:44.170 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:44.170 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:44.180 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:44.216 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:44.214+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:44.235 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:02:44.284 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:44.284 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:44.295 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:44.337 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:44.336+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:44.375 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:44.394 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:44.412 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:44.418 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:44.414+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:44.432 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:44.441 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:44.441 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:44.441 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:44.441 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:44.451 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:44.455 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:44.485 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:02:44.489 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:44.519 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:44.516+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:44.524 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:44.551 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:44.553 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:44.553 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:44.553 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:44.553 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:44.553 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:44.553 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:44.553 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:44.553 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:44.566 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:44.566 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:44.566 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:44.566 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:44.566 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:44.577 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:44.620 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:44.618+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:44.721 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:44.718+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:44.728 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:44.728 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:44.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:44.729 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:44.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.729 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:02:44.729 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:44.729 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.234300486999999, 2024-09-13T07:02:44.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:02:44.729 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.729 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:44.729 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.729 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.729 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.730 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:02:44.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.730 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:44.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:44.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:02:44.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:44.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:02:44.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:02:44.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:02:44.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:02:44.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:02:44.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:02:44.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:02:44.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:02:44.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:02:44.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:02:44.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:02:44.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:02:44.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:02:44.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:44.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:02:44.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:02:44.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:02:44.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:02:44.735 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.735 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.735 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.736 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.736 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:44.736 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.747816923000002, 2024-09-13T07:02:44.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:02:44.736 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.737 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:44.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.737 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:02:44.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.737 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:44.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:44.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:44.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:02:44.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:02:44.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:02:44.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:02:44.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:02:44.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:02:44.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:02:44.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:02:44.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:44.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:44.743 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.747746994000003, 2024-09-13T07:02:44.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:02:44.744 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.744 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:44.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.744 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:02:44.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.744 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:44.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:44.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:02:44.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:02:44.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:02:44.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:02:44.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:02:44.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:02:44.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:02:44.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:02:44.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:02:44.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:44.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:02:44.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:02:44.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:02:44.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:02:44.749 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.749 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.749 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.750 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.750 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:44.750 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.747731121999998, 2024-09-13T07:02:44.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:02:44.750 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.750 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:44.750 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.750 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.751 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.751 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:02:44.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.751 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:44.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:44.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:02:44.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:02:44.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:02:44.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:02:44.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:02:44.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:02:44.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:02:44.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:02:44.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:02:44.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:44.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:02:44.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:02:44.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:02:44.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:02:44.756 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.756 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.756 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.756 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.756 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:44.756 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.747660664999998, 2024-09-13T07:02:44.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:02:44.756 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.756 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:44.757 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.757 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.757 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.757 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:02:44.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.757 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:44.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:44.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:44.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:02:44.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:02:44.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:02:44.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:02:44.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:02:44.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:02:44.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:02:44.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:02:44.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:44.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:02:44.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:02:44.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:02:44.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:02:44.762 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.763 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.763 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.763 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.763 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:44.764 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744308627000002, 2024-09-13T07:02:44.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:02:44.764 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.764 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.764 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.764 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.764 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.764 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:02:44.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.765 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:44.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:44.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:02:44.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:02:44.768 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.768 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.768 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.768 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.768 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:44.768 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744292326, 2024-09-13T07:02:44.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:02:44.768 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.768 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.769 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:02:44.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.769 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:44.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:44.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:02:44.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:02:44.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:02:44.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:02:44.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:02:44.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:02:44.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:02:44.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:02:44.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:02:44.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:02:44.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:02:44.772 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.772 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.772 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.773 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.773 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:44.773 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744272541000001, 2024-09-13T07:02:44.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:02:44.773 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.773 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.773 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:02:44.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:02:44.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:02:44.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:02:44.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:02:44.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:02:44.777 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.777 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.777 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.777 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.777 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:44.777 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744256268000001, 2024-09-13T07:02:44.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:02:44.777 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.778 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.778 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.778 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.778 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.778 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:02:44.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.778 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:44.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:44.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:02:44.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:02:44.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:02:44.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:02:44.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:02:44.782 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.782 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.782 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.782 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.782 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:44.782 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744238443, 2024-09-13T07:02:44.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:02:44.782 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.782 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:44.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:02:44.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:02:44.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:02:44.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:02:44.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:02:44.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:02:44.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:44.786 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744222350000001, 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:02:44.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:02:44.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:02:44.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:02:44.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:02:44.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:02:44.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:02:44.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:02:44.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:02:44.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:02:44.790 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.790 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.790 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.791 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.791 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:44.791 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744201164000003, 2024-09-13T07:02:44.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:02:44.791 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.791 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.791 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:02:44.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.792 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:44.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:44.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:02:44.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:02:44.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:02:44.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:02:44.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:02:44.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:02:44.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:02:44.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:02:44.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:02:44.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:02:44.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:02:44.795 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.795 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.795 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.795 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.795 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:44.795 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744185131000002, 2024-09-13T07:02:44.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:02:44.796 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.796 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.796 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:02:44.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.796 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:44.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:44.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:02:44.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:02:44.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:02:44.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:02:44.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:02:44.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:02:44.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:02:44.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:02:44.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:02:44.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:02:44.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:02:44.800 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.800 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.800 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.801 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.801 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:44.801 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744135892000003, 2024-09-13T07:02:44.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:02:44.801 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.801 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.801 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:02:44.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.801 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:44.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:44.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:02:44.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:02:44.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:02:44.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:02:44.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:02:44.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:02:44.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:02:44.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.807 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.807 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:44.807 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744117832000001, 2024-09-13T07:02:44.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:02:44.807 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.807 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.808 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.808 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:02:44.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.808 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:44.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:44.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:44.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:02:44.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:02:44.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:02:44.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:02:44.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:02:44.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:02:44.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:02:44.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:02:44.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:02:44.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:02:44.812 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.812 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.812 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744101317000002, 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:44.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:44.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:02:44.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:02:44.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744082951999999, 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:02:44.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:02:44.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:02:44.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:02:44.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:02:44.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:02:44.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:02:44.821 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.821 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.821 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.821 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.821 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:44.821 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744066854000003, 2024-09-13T07:02:44.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:02:44.821 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.821 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.822 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.822 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.822 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.822 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:02:44.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.822 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:44.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:44.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:44.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:02:44.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:02:44.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:02:44.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:02:44.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.825 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.825 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744050919000003, 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:44.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:02:44.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:02:44.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:02:44.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:02:44.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.829 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.829 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:44.829 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.744034421999999, 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:02:44.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:02:44.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:44.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:02:44.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:02:44.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:44.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:02:44.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:02:44.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:44.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:02:44.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:02:44.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:44.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:44.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:44.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:02:44.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:02:44.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.833 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:44.833 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:44.833 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:44.835 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:44.835 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:44.832+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:44.936 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:44.934+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:44.945 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:44.945 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:44.945 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:44.945 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:44.955 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:45.037 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:45.034+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:45.060 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:45.060 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:45.060 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:45.060 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:45.060 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:45.070 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:45.094 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:02:45.137 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:45.136+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:45.175 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:45.175 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:434 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'4, mlcod=26'2)", 2024-09-13T07:02:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.639295630000007, 2024-09-13T07:02:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001931945, 2024-09-13T07:02:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535697+0000", 2024-09-13T07:02:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535695+0000", 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535709+0000", 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3577999999999999e-05 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535732+0000", 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3085e-05 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535742+0000", 2024-09-13T07:02:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.048e-05 2024-09-13T07:02:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535840+0000", 2024-09-13T07:02:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7159999999999998e-05 2024-09-13T07:02:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.535871+0000", 2024-09-13T07:02:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1510999999999997e-05 2024-09-13T07:02:45.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537572+0000", 2024-09-13T07:02:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017008279999999999 2024-09-13T07:02:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.537629+0000", 2024-09-13T07:02:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6520000000000001e-05 2024-09-13T07:02:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:435 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'5, mlcod=26'2)", 2024-09-13T07:02:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.638163923999997, 2024-09-13T07:02:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017126769999999999, 2024-09-13T07:02:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536828+0000", 2024-09-13T07:02:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536835+0000", 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4939999999999997e-06 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536854+0000", 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8655999999999999e-05 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536862+0000", 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6570000000000003e-06 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536926+0000", 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4037999999999999e-05 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.536958+0000", 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1584000000000001e-05 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538493+0000", 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00153513 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.538541+0000", 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7883999999999998e-05 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:483 3.4 e26/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 26'24, mlcod=26'21)", 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.559685294999994, 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018371780000000001, 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.185 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615307+0000", 2024-09-13T07:02:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615306+0000", 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615322+0000", 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5519000000000002e-05 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615335+0000", 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3854e-05 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615345+0000", 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4930000000000004e-06 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615431+0000", 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6346000000000003e-05 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.615480+0000", 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8303999999999997e-05 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617102+0000", 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016225250000000001 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:24.617144+0000", 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1936e-05 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.189 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:518 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'38, mlcod=26'36)", 2024-09-13T07:02:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 79.850108556999999, 2024-09-13T07:02:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018201179999999999, 2024-09-13T07:02:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324884+0000", 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324883+0000", 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324891+0000", 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1499999999999999e-06 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324924+0000", 2024-09-13T07:02:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2601999999999997e-05 2024-09-13T07:02:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.324942+0000", 2024-09-13T07:02:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7796999999999999e-05 2024-09-13T07:02:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325060+0000", 2024-09-13T07:02:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011786900000000001 2024-09-13T07:02:45.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.325123+0000", 2024-09-13T07:02:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3474999999999998e-05 2024-09-13T07:02:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326620+0000", 2024-09-13T07:02:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014968830000000001 2024-09-13T07:02:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.326704+0000", 2024-09-13T07:02:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3716999999999998e-05 2024-09-13T07:02:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:597 3.4 e27/22 3:21c7fe03:::rbd_object_map.1194d22ad098:head v 27'39, mlcod=26'37)", 2024-09-13T07:02:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 79.733281228999999, 2024-09-13T07:02:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017959829999999999, 2024-09-13T07:02:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.194 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441711+0000", 2024-09-13T07:02:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441709+0000", 2024-09-13T07:02:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441728+0000", 2024-09-13T07:02:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9627999999999999e-05 2024-09-13T07:02:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441765+0000", 2024-09-13T07:02:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6559e-05 2024-09-13T07:02:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441783+0000", 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7691000000000001e-05 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441904+0000", 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000121184 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.441966+0000", 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2428999999999995e-05 2024-09-13T07:02:45.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443448+0000", 2024-09-13T07:02:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00148197 2024-09-13T07:02:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.443507+0000", 2024-09-13T07:02:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8699000000000002e-05 2024-09-13T07:02:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 79.118400934999997, 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:02:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:02:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:02:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:02:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:02:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:02:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:02:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:02:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:02:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:02:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:02:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:02:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:02:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:02:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:02:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1548 3.4 e27/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 27'52, mlcod=27'50)", 2024-09-13T07:02:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 79.032779399000006, 2024-09-13T07:02:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017825779999999999, 2024-09-13T07:02:45.203 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142213+0000", 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142211+0000", 2024-09-13T07:02:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142232+0000", 2024-09-13T07:02:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0888000000000001e-05 2024-09-13T07:02:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142268+0000", 2024-09-13T07:02:45.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6261999999999999e-05 2024-09-13T07:02:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142285+0000", 2024-09-13T07:02:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7640000000000001e-05 2024-09-13T07:02:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142381+0000", 2024-09-13T07:02:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5171000000000001e-05 2024-09-13T07:02:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142450+0000", 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9778999999999997e-05 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143953+0000", 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001503129 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143995+0000", 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1878000000000001e-05 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.207 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1550 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'14, mlcod=27'12)", 2024-09-13T07:02:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 79.032131688000007, 2024-09-13T07:02:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002750452, 2024-09-13T07:02:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142861+0000", 2024-09-13T07:02:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142859+0000", 2024-09-13T07:02:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:45.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142874+0000", 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5149e-05 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142911+0000", 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6439000000000001e-05 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.142926+0000", 2024-09-13T07:02:45.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5079000000000001e-05 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143012+0000", 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.6230000000000006e-05 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.143063+0000", 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1369000000000002e-05 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145539+0000", 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024753420000000002 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.145611+0000", 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2392999999999996e-05 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1668 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'17, mlcod=27'15)", 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:45.212 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 78.833716226000007, 2024-09-13T07:02:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033976620000000001, 2024-09-13T07:02:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:45.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341276+0000", 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341281+0000", 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8109999999999999e-06 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341298+0000", 2024-09-13T07:02:45.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6575e-05 2024-09-13T07:02:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341308+0000", 2024-09-13T07:02:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0369e-05 2024-09-13T07:02:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341434+0000", 2024-09-13T07:02:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125454 2024-09-13T07:02:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341453+0000", 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9159000000000001e-05 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344638+0000", 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031851750000000002 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.344674+0000", 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5410999999999997e-05 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.216 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1666 3.4 e28/22 3:21da476b:::rbd_data.11943a018f34.0000000000000002:head v 28'55, mlcod=27'53)", 2024-09-13T07:02:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 78.833662554, 2024-09-13T07:02:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001805809, 2024-09-13T07:02:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341330+0000", 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341329+0000", 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341349+0000", 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.999e-05 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341357+0000", 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0620000000000003e-06 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341361+0000", 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341710+0000", 2024-09-13T07:02:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00034873400000000001 2024-09-13T07:02:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.341736+0000", 2024-09-13T07:02:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5772e-05 2024-09-13T07:02:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343097+0000", 2024-09-13T07:02:45.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0013615319999999999 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.343135+0000", 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8111999999999997e-05 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1800 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'20, mlcod=28'18)", 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 78.631292087999995, 2024-09-13T07:02:45.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001874398, 2024-09-13T07:02:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543700+0000", 2024-09-13T07:02:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543699+0000", 2024-09-13T07:02:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543713+0000", 2024-09-13T07:02:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4786e-05 2024-09-13T07:02:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543737+0000", 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3626e-05 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543752+0000", 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.489e-05 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543832+0000", 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0396999999999996e-05 2024-09-13T07:02:45.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.543994+0000", 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016129100000000001 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545471+0000", 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0014771960000000001 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.545575+0000", 2024-09-13T07:02:45.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103732 2024-09-13T07:02:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1882 3.0 e28/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 28'23, mlcod=28'21)", 2024-09-13T07:02:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 78.535659107000001, 2024-09-13T07:02:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001767172, 2024-09-13T07:02:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639333+0000", 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639332+0000", 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:45.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639343+0000", 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1436e-05 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639361+0000", 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7915e-05 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639370+0000", 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3189999999999995e-06 2024-09-13T07:02:45.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639425+0000", 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4917999999999998e-05 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.639454+0000", 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9862999999999999e-05 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641059+0000", 2024-09-13T07:02:45.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016048950000000001 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641100+0000", 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0883999999999997e-05 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3069 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'33, mlcod=31'31)", 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.012519390999998, 2024-09-13T07:02:45.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018597010000000001, 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162473+0000", 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162471+0000", 2024-09-13T07:02:45.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:45.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162520+0000", 2024-09-13T07:02:45.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8466e-05 2024-09-13T07:02:45.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162546+0000", 2024-09-13T07:02:45.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6089999999999999e-05 2024-09-13T07:02:45.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162562+0000", 2024-09-13T07:02:45.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5962000000000001e-05 2024-09-13T07:02:45.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162646+0000", 2024-09-13T07:02:45.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3985000000000003e-05 2024-09-13T07:02:45.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.162691+0000", 2024-09-13T07:02:45.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5785999999999998e-05 2024-09-13T07:02:45.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164265+0000", 2024-09-13T07:02:45.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015740890000000001 2024-09-13T07:02:45.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.164333+0000", 2024-09-13T07:02:45.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7014999999999994e-05 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3121 3.0 e31/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 31'36, mlcod=31'34)", 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 75.898573003999999, 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019657780000000001, 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:45.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276419+0000", 2024-09-13T07:02:45.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276417+0000", 2024-09-13T07:02:45.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:45.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276497+0000", 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9729999999999997e-05 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276537+0000", 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0102999999999999e-05 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276553+0000", 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6518999999999999e-05 2024-09-13T07:02:45.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276662+0000", 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000109252 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.276729+0000", 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7056000000000003e-05 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278270+0000", 2024-09-13T07:02:45.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015408819999999999 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:29.278385+0000", 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000114613 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3390 3.0 e32/22 3:05ab4703:::rbd_data.11945d91e99c.0000000000000000:head v 32'45, mlcod=31'42)", 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.679484657000003, 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002246587, 2024-09-13T07:02:45.239 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495508+0000", 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495507+0000", 2024-09-13T07:02:45.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495530+0000", 2024-09-13T07:02:45.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2592999999999999e-05 2024-09-13T07:02:45.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495550+0000", 2024-09-13T07:02:45.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0152999999999998e-05 2024-09-13T07:02:45.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495562+0000", 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2336e-05 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495662+0000", 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100317 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.495717+0000", 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4304e-05 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497707+0000", 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019906849999999999 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.497754+0000", 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6715999999999998e-05 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3399 3.4 e32/22 3:22863a52:::rbd_data.11945d91e99c.0000000000000013:head v 32'65, mlcod=31'62)", 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:45.243 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.675547984999994, 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028830779999999999, 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499444+0000", 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499442+0000", 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499459+0000", 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7042e-05 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499493+0000", 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3788000000000002e-05 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499512+0000", 2024-09-13T07:02:45.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9055999999999998e-05 2024-09-13T07:02:45.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499608+0000", 2024-09-13T07:02:45.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6168000000000005e-05 2024-09-13T07:02:45.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499658+0000", 2024-09-13T07:02:45.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9469999999999999e-05 2024-09-13T07:02:45.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502277+0000", 2024-09-13T07:02:45.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0026188639999999998 2024-09-13T07:02:45.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502327+0000", 2024-09-13T07:02:45.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0578999999999998e-05 2024-09-13T07:02:45.247 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.247 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.247 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.248 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:3395 3.0 e32/22 3:05869443:::rbd_data.11945d91e99c.0000000000000006:head v 32'47, mlcod=31'43)", 2024-09-13T07:02:45.248 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:45.248 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.675449900000004, 2024-09-13T07:02:45.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027505870000000001, 2024-09-13T07:02:45.248 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.248 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.248 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:45.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:45.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499542+0000", 2024-09-13T07:02:45.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499560+0000", 2024-09-13T07:02:45.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8416000000000001e-05 2024-09-13T07:02:45.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499576+0000", 2024-09-13T07:02:45.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5602000000000001e-05 2024-09-13T07:02:45.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499587+0000", 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1112e-05 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499654+0000", 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7164999999999998e-05 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.499674+0000", 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9638000000000001e-05 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502210+0000", 2024-09-13T07:02:45.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025367100000000002 2024-09-13T07:02:45.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.502293+0000", 2024-09-13T07:02:45.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2607000000000004e-05 2024-09-13T07:02:45.252 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.252 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.252 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.253 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:4534 3.0 e32/22 3:00392e9e:::rbd_data.11944cdc0656.000000000000000c:head v 32'51, mlcod=32'49)", 2024-09-13T07:02:45.253 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:45.253 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.263748714000002, 2024-09-13T07:02:45.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017597330000000001, 2024-09-13T07:02:45.253 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.253 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.253 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:45.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911243+0000", 2024-09-13T07:02:45.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911242+0000", 2024-09-13T07:02:45.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:45.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911306+0000", 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4480999999999995e-05 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911338+0000", 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1733000000000003e-05 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911351+0000", 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3241000000000001e-05 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911455+0000", 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000103723 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.911495+0000", 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0683000000000003e-05 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.912933+0000", 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001437219 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.913003+0000", 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0629000000000004e-05 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11205 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'50, mlcod=38'47)", 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 66.550408228999999, 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0031049490000000001, 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624584+0000", 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.624583+0000", 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625339+0000", 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00075563200000000001 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625358+0000", 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9916999999999999e-05 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625366+0000", 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8050000000000007e-06 2024-09-13T07:02:45.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625415+0000", 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8231e-05 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625447+0000", 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2703999999999997e-05 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627652+0000", 2024-09-13T07:02:45.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022050659999999999 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.627689+0000", 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6612000000000001e-05 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:11206 3.1f e40/36 3:faa1d020:::rbd_data.1194a7cedc35.0000000000000004:head v 40'52, mlcod=38'47)", 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 66.549137563000002, 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022238470000000002, 2024-09-13T07:02:45.261 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625855+0000", 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625854+0000", 2024-09-13T07:02:45.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625863+0000", 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5380000000000008e-06 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625883+0000", 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9247000000000001e-05 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.625893+0000", 2024-09-13T07:02:45.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0176e-05 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626411+0000", 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00051847600000000003 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.626431+0000", 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0199000000000001e-05 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628028+0000", 2024-09-13T07:02:45.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0015968110000000001 2024-09-13T07:02:45.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:38.628078+0000", 2024-09-13T07:02:45.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0170000000000002e-05 2024-09-13T07:02:45.265 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.265 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.265 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.265 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.265 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.265 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:45.267 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:45.267 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:45.264+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:45.368 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:45.366+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:45.425 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:45.426 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:45.426 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:45.426 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:45.436 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:45.469 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:45.470+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:45.554 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:45.554 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:45.554 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:45.554 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:45.554 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:45.563 INFO:teuthology.orchestra.run.smithi118.stdout:3441 read 0x460204a thru 0x460377a (0x1731 bytes) 2024-09-13T07:02:45.563 INFO:teuthology.orchestra.run.smithi118.stdout:3442 read 0x146daf2 thru 0x146fd69 (0x2278 bytes) 2024-09-13T07:02:45.565 INFO:teuthology.orchestra.run.smithi118.stdout:3443 trunc from 0x7fb7179 to 0x781aa10 2024-09-13T07:02:45.566 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:45.570 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:45.570+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:45.577 INFO:teuthology.orchestra.run.smithi118.stdout:3444 writesame 0x4ec84be thru 0x4eca5f6 data_size 0x2d(0x2139 bytes) 2024-09-13T07:02:45.592 INFO:teuthology.orchestra.run.smithi118.stdout:3448 writesame 0x8ee237f thru 0x8eec3dc data_size 0xd(0xa05e bytes) 2024-09-13T07:02:45.592 INFO:teuthology.orchestra.run.smithi118.stdout:3449 punch from 0x46f79c2 to 0x46fbea1, (0x44df bytes) 2024-09-13T07:02:45.593 INFO:teuthology.orchestra.run.smithi118.stdout:3452 write 0x8aece6 thru 0x8af5da (0x8f5 bytes) 2024-09-13T07:02:45.595 INFO:teuthology.orchestra.run.smithi118.stdout:3453 write 0x7018021 thru 0x701dcb3 (0x5c93 bytes) 2024-09-13T07:02:45.599 INFO:teuthology.orchestra.run.smithi118.stdout:3455 writesame 0x8d47530 thru 0x8d4e510 data_size 0x1(0x6fe1 bytes) 2024-09-13T07:02:45.619 INFO:teuthology.orchestra.run.smithi118.stdout:3456 read 0x6147a4f thru 0x6157503 (0xfab5 bytes) 2024-09-13T07:02:45.619 INFO:teuthology.orchestra.run.smithi118.stdout:3457 write 0x2b1ee9b thru 0x2b292c4 (0xa42a bytes) 2024-09-13T07:02:45.630 INFO:teuthology.orchestra.run.smithi118.stdout:3458 trunc from 0x8eec3dd to 0x5b65080 2024-09-13T07:02:45.648 INFO:teuthology.orchestra.run.smithi118.stdout:3459 write 0x2a22293 thru 0x2a26e41 (0x4baf bytes) 2024-09-13T07:02:45.651 INFO:teuthology.orchestra.run.smithi118.stdout:3461 write 0x1833c0e thru 0x1842514 (0xe907 bytes) 2024-09-13T07:02:45.656 INFO:teuthology.orchestra.run.smithi118.stdout:3462 punch from 0x1aed0b6 to 0x1af485d, (0x77a7 bytes) 2024-09-13T07:02:45.671 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:45.670+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:45.672 INFO:teuthology.orchestra.run.smithi118.stdout:3463 write 0x9779233 thru 0x9787848 (0xe616 bytes) 2024-09-13T07:02:45.683 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:45.683 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:45.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:45.683 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:45.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.683 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.683 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:45.683 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.560647027000002, 2024-09-13T07:02:45.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:02:45.684 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.684 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.684 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.684 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.684 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.684 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:02:45.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.685 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:45.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:45.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:02:45.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:02:45.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:02:45.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:02:45.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:02:45.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:02:45.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:02:45.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:02:45.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:02:45.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:02:45.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:02:45.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:02:45.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:02:45.689 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.689 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.689 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.689 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.689 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:45.689 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.560626549999999, 2024-09-13T07:02:45.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:02:45.689 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.689 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.690 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.690 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.690 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.690 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:02:45.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.690 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:45.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:02:45.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:02:45.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:02:45.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:02:45.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:02:45.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:02:45.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:02:45.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:02:45.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:02:45.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:02:45.693 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.693 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.693 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.694 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87036 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 6963200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.694 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:45.694 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553942484, 2024-09-13T07:02:45.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042468831999999998, 2024-09-13T07:02:45.694 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.694 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.694 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.694 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.694 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.694 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87036 2024-09-13T07:02:45.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127964+0000", 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7799999999999999e-06 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127967+0000", 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9730000000000002e-06 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127971+0000", 2024-09-13T07:02:45.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4349999999999999e-06 2024-09-13T07:02:45.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170202+0000", 2024-09-13T07:02:45.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042230940000000002 2024-09-13T07:02:45.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170293+0000", 2024-09-13T07:02:45.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0858000000000003e-05 2024-09-13T07:02:45.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170431+0000", 2024-09-13T07:02:45.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013821699999999999 2024-09-13T07:02:45.699 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.699 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.699 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.699 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.700 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:45.700 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553929947, 2024-09-13T07:02:45.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:02:45.700 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.700 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.700 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.700 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:02:45.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.700 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:45.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:45.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:02:45.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:02:45.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:02:45.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:02:45.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:45.704 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553917878, 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:45.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:45.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:45.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:02:45.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:02:45.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:02:45.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:02:45.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:02:45.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:02:45.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:02:45.709 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.709 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.709 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.709 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.709 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:45.709 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553905435000001, 2024-09-13T07:02:45.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:02:45.710 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.710 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.710 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.710 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.710 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.710 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:02:45.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.711 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:45.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:45.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:45.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:45.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:02:45.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:02:45.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:02:45.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:02:45.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:02:45.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:02:45.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:02:45.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:02:45.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:02:45.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:02:45.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:02:45.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:02:45.718 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.718 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.718 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.719 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.719 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:45.719 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553887837000001, 2024-09-13T07:02:45.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:02:45.719 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.720 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.720 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.720 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.720 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.720 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:02:45.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.720 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:45.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:45.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:45.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:02:45.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:02:45.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:02:45.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:02:45.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:02:45.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:02:45.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:02:45.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:02:45.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:02:45.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:02:45.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:02:45.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:02:45.727 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.727 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.728 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.728 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.728 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:45.728 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553877134, 2024-09-13T07:02:45.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:02:45.728 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.729 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.729 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.729 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.729 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.729 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:02:45.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.730 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:45.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:45.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:45.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:02:45.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:02:45.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:02:45.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:02:45.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553833201, 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.734 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:45.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.738 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553813415, 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:45.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:02:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:02:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:02:45.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:02:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:02:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:02:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:02:45.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553797665000001, 2024-09-13T07:02:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:02:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:02:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.744 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:02:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:02:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:02:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:02:45.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:02:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:02:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:02:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:02:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:02:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:02:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:02:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:02:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.748 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553786413000001, 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:02:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:02:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:02:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:02:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:02:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:02:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553775814000002, 2024-09-13T07:02:45.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:02:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:02:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:02:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:02:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:02:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:02:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:02:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:02:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:02:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:02:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:02:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:02:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553766321000001, 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.758 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:02:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:02:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:02:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:02:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:02:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:02:45.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553756777, 2024-09-13T07:02:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:02:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:02:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.763 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:02:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:02:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553724173999999, 2024-09-13T07:02:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:02:45.767 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:02:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:02:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:02:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:02:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:02:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:02:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:02:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:02:45.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:02:45.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:02:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.553711151000002, 2024-09-13T07:02:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:02:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:02:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:45.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:02:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:02:45.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.550360166000001, 2024-09-13T07:02:45.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:02:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:02:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:45.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:02:45.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:02:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:02:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:02:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:02:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.781 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.550339889, 2024-09-13T07:02:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:02:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:02:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:02:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:02:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:02:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:02:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:02:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:02:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:02:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:02:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:45.786 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.550319687999998, 2024-09-13T07:02:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:02:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:02:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:45.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:02:45.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:02:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:02:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:45.791 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:45.791 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:45.792 INFO:teuthology.orchestra.run.smithi118.stdout:3464 punch from 0x1e62e69 to 0x1e68cc1, (0x5e58 bytes) 2024-09-13T07:02:45.793 INFO:teuthology.orchestra.run.smithi118.stdout:3465 read 0x6d1d8d9 thru 0x6d2b749 (0xde71 bytes) 2024-09-13T07:02:45.793 INFO:teuthology.orchestra.run.smithi118.stdout:3467 punch from 0x2d6fbc0 to 0x2d7a58d, (0xa9cd bytes) 2024-09-13T07:02:45.793 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:45.793 INFO:teuthology.orchestra.run.smithi118.stdout:3469 read 0xde0948 thru 0xde6019 (0x56d2 bytes) 2024-09-13T07:02:45.794 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:45.794 INFO:teuthology.orchestra.run.smithi118.stdout:3470 write 0xe1c064f thru 0xe1ce15e (0xdb10 bytes) 2024-09-13T07:02:45.794 INFO:teuthology.orchestra.run.smithi118.stdout:3471 read 0x24720f2 thru 0x2472e5a (0xd69 bytes) 2024-09-13T07:02:45.795 INFO:teuthology.orchestra.run.smithi118.stdout:3474 write 0xa2639da thru 0xa270feb (0xd612 bytes) 2024-09-13T07:02:45.795 INFO:teuthology.orchestra.run.smithi118.stdout:3475 read 0x1747113 thru 0x174d66e (0x655c bytes) 2024-09-13T07:02:45.795 INFO:teuthology.orchestra.run.smithi118.stdout:3476 trunc from 0xe1ce15f to 0x2b9f827 2024-09-13T07:02:45.795 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:45.788+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:45.819 INFO:teuthology.orchestra.run.smithi118.stdout:3479 write 0x6be26c0 thru 0x6beaf74 (0x88b5 bytes) 2024-09-13T07:02:45.841 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:45.841 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:45.842 INFO:teuthology.orchestra.run.smithi118.stdout:3483 punch from 0x66c65ae to 0x66d3c3f, (0xd691 bytes) 2024-09-13T07:02:45.842 INFO:teuthology.orchestra.run.smithi118.stdout:3484 write 0x152156d thru 0x1522eb7 (0x194b bytes) 2024-09-13T07:02:45.854 INFO:teuthology.orchestra.run.smithi118.stdout:3485 trunc from 0x6beaf75 to 0xbe3cbc3 2024-09-13T07:02:45.862 INFO:teuthology.orchestra.run.smithi118.stdout:3486 read 0x8857404 thru 0x8859e41 (0x2a3e bytes) 2024-09-13T07:02:45.862 INFO:teuthology.orchestra.run.smithi118.stdout:3487 write 0xaa966da thru 0xaa976c5 (0xfec bytes) 2024-09-13T07:02:45.869 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:45.869 INFO:teuthology.orchestra.run.smithi118.stdout:3490 clone 33 order 24 su 262144 sc 12 2024-09-13T07:02:45.894 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:45.892+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:45.905 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:45.905 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:45.914 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:45.995 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:45.992+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:46.016 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:46.017 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:46.017 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:46.017 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:46.017 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:46.026 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:46.096 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:46.094+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:46.137 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1524 3.14 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.091077283000004, 2024-09-13T07:02:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061190880000000003, 2024-09-13T07:02:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1524 2024-09-13T07:02:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044769+0000", 2024-09-13T07:02:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.044767+0000", 2024-09-13T07:02:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052670+0000", 2024-09-13T07:02:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079022450000000004 2024-09-13T07:02:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052682+0000", 2024-09-13T07:02:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2336e-05 2024-09-13T07:02:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052696+0000", 2024-09-13T07:02:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-13T07:02:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052755+0000", 2024-09-13T07:02:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9298e-05 2024-09-13T07:02:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.052947+0000", 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019153900000000001 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.053694+0000", 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074663099999999999 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.102148+0000", 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.048454329999999997 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105816+0000", 2024-09-13T07:02:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036675950000000001 2024-09-13T07:02:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105921+0000", 2024-09-13T07:02:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105874 2024-09-13T07:02:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105959+0000", 2024-09-13T07:02:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7991000000000003e-05 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:48369 3.f e58/22 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head v 58'78, mlcod=57'75)", 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.895379417999997, 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.057439072000000001, 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240466+0000", 2024-09-13T07:02:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.240464+0000", 2024-09-13T07:02:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245716+0000", 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005252329 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245745+0000", 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8844e-05 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245758+0000", 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2242000000000001e-05 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245849+0000", 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1187999999999995e-05 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.245898+0000", 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8881000000000002e-05 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297837+0000", 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051938947999999999 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297906+0000", 2024-09-13T07:02:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8955999999999995e-05 2024-09-13T07:02:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94699 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 1826816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674309+0000", 2024-09-13T07:02:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.461537060000001, 2024-09-13T07:02:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051428639999999998, 2024-09-13T07:02:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94699 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674309+0000", 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674309+0000", 2024-09-13T07:02:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674308+0000", 2024-09-13T07:02:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674311+0000", 2024-09-13T07:02:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4200000000000001e-06 2024-09-13T07:02:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674317+0000", 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9429999999999997e-06 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674321+0000", 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6460000000000001e-06 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725637+0000", 2024-09-13T07:02:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051315686999999999 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725687+0000", 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0096000000000003e-05 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725737+0000", 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0232999999999998e-05 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94700 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674326+0000", 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.46152013, 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051537695000000001, 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94700 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674326+0000", 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674326+0000", 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674325+0000", 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674328+0000", 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4399999999999999e-06 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674333+0000", 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4569999999999996e-06 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674338+0000", 2024-09-13T07:02:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6659999999999999e-06 2024-09-13T07:02:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725762+0000", 2024-09-13T07:02:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051423979000000002 2024-09-13T07:02:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725816+0000", 2024-09-13T07:02:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4509999999999998e-05 2024-09-13T07:02:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725863+0000", 2024-09-13T07:02:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7021999999999998e-05 2024-09-13T07:02:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94701 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2056192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674342+0000", 2024-09-13T07:02:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.4615039, 2024-09-13T07:02:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051643316000000002, 2024-09-13T07:02:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94701 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674342+0000", 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674342+0000", 2024-09-13T07:02:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674342+0000", 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674344+0000", 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674349+0000", 2024-09-13T07:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1880000000000001e-06 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674354+0000", 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3800000000000004e-06 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725889+0000", 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051535091999999998 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725937+0000", 2024-09-13T07:02:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8680000000000001e-05 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.725985+0000", 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7871000000000003e-05 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94702 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2121728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674358+0000", 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.461488012, 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051754434000000002, 2024-09-13T07:02:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94702 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674358+0000", 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674358+0000", 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674358+0000", 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674360+0000", 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4099999999999998e-06 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674365+0000", 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0860000000000001e-06 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674370+0000", 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7740000000000004e-06 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726009+0000", 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051639184999999997 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726061+0000", 2024-09-13T07:02:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1613000000000002e-05 2024-09-13T07:02:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726112+0000", 2024-09-13T07:02:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1684999999999997e-05 2024-09-13T07:02:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94703 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2236416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674374+0000", 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.461471938999999, 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051856764999999999, 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94703 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674374+0000", 2024-09-13T07:02:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674374+0000", 2024-09-13T07:02:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674373+0000", 2024-09-13T07:02:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674385+0000", 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1894e-05 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674391+0000", 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2280000000000003e-06 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674397+0000", 2024-09-13T07:02:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.164e-06 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726136+0000", 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051739072999999997 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726181+0000", 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4914999999999999e-05 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726231+0000", 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0120000000000001e-05 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94704 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2351104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674401+0000", 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.461444539999999, 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051952104999999998, 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94704 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674401+0000", 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674401+0000", 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674401+0000", 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674403+0000", 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.655e-06 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674439+0000", 2024-09-13T07:02:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5126999999999998e-05 2024-09-13T07:02:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674444+0000", 2024-09-13T07:02:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3970000000000002e-06 2024-09-13T07:02:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726254+0000", 2024-09-13T07:02:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051810387999999999 2024-09-13T07:02:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726305+0000", 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1057000000000003e-05 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726353+0000", 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8013999999999998e-05 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94705 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2465792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674452+0000", 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.461394209000002, 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.052023257000000003, 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94705 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674452+0000", 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674452+0000", 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674451+0000", 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674455+0000", 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.506e-06 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674461+0000", 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.455e-06 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674466+0000", 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7999999999999998e-06 2024-09-13T07:02:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726382+0000", 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051916074999999999 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726428+0000", 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5914000000000001e-05 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726475+0000", 2024-09-13T07:02:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6993999999999998e-05 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94706 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674678+0000", 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.461168060999999, 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051920368000000001, 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94706 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674678+0000", 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674678+0000", 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674470+0000", 2024-09-13T07:02:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9997926 2024-09-13T07:02:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674682+0000", 2024-09-13T07:02:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000211695 2024-09-13T07:02:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674781+0000", 2024-09-13T07:02:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9296000000000006e-05 2024-09-13T07:02:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674981+0000", 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019990000000000001 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726497+0000", 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051516144 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726549+0000", 2024-09-13T07:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1721000000000002e-05 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726598+0000", 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9039999999999998e-05 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94707 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2646016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.674991+0000", 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.46085506, 2024-09-13T07:02:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051730683, 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94707 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674991+0000", 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674991+0000", 2024-09-13T07:02:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674990+0000", 2024-09-13T07:02:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.674999+0000", 2024-09-13T07:02:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675007+0000", 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0870000000000007e-06 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675013+0000", 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726625+0000", 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051612492000000003 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726674+0000", 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.884e-05 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726721+0000", 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7348999999999997e-05 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94708 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2760704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675018+0000", 2024-09-13T07:02:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.460827945999998, 2024-09-13T07:02:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051826217000000001, 2024-09-13T07:02:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94708 2024-09-13T07:02:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675018+0000", 2024-09-13T07:02:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675018+0000", 2024-09-13T07:02:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675018+0000", 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675021+0000", 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1269999999999999e-06 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675026+0000", 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7610000000000004e-06 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675031+0000", 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6160000000000004e-06 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726747+0000", 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051716170999999998 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726796+0000", 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8718999999999999e-05 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726844+0000", 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8214000000000003e-05 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94709 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675035+0000", 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.460810408, 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051926446000000001, 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94709 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675035+0000", 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675035+0000", 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675035+0000", 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675038+0000", 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6910000000000002e-06 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675044+0000", 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3879999999999997e-06 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675049+0000", 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726868+0000", 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051818636000000001 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726917+0000", 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9586999999999998e-05 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726962+0000", 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4807e-05 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94710 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675054+0000", 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.460792023, 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.052041476000000003, 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94710 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675054+0000", 2024-09-13T07:02:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675054+0000", 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675053+0000", 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675056+0000", 2024-09-13T07:02:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8619999999999999e-06 2024-09-13T07:02:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675062+0000", 2024-09-13T07:02:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5709999999999997e-06 2024-09-13T07:02:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675370+0000", 2024-09-13T07:02:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030795900000000001 2024-09-13T07:02:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.726989+0000", 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051619842999999999 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727038+0000", 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8792000000000003e-05 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727095+0000", 2024-09-13T07:02:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7095000000000001e-05 2024-09-13T07:02:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94711 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3104768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675377+0000", 2024-09-13T07:02:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.460469247999999, 2024-09-13T07:02:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051839969, 2024-09-13T07:02:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94711 2024-09-13T07:02:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675377+0000", 2024-09-13T07:02:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675377+0000", 2024-09-13T07:02:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675376+0000", 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675380+0000", 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7749999999999999e-06 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675385+0000", 2024-09-13T07:02:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6559999999999997e-06 2024-09-13T07:02:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675388+0000", 2024-09-13T07:02:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.022e-06 2024-09-13T07:02:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727124+0000", 2024-09-13T07:02:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051735712000000003 2024-09-13T07:02:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727174+0000", 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9716999999999998e-05 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727217+0000", 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2710999999999998e-05 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94712 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3170304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.675391+0000", 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.460454895000002, 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051946194000000001, 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94712 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675391+0000", 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675391+0000", 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675391+0000", 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675392+0000", 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5340000000000001e-06 2024-09-13T07:02:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675395+0000", 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8140000000000001e-06 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.675398+0000", 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5809999999999999e-06 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727240+0000", 2024-09-13T07:02:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051842436999999998 2024-09-13T07:02:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727288+0000", 2024-09-13T07:02:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7901000000000001e-05 2024-09-13T07:02:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727337+0000", 2024-09-13T07:02:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9177e-05 2024-09-13T07:02:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94714 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.676292+0000", 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.459553709000001, 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051320047000000001, 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94714 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676292+0000", 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676292+0000", 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676291+0000", 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676295+0000", 2024-09-13T07:02:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8689999999999997e-06 2024-09-13T07:02:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676302+0000", 2024-09-13T07:02:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4230000000000002e-06 2024-09-13T07:02:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676307+0000", 2024-09-13T07:02:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9320000000000004e-06 2024-09-13T07:02:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727488+0000", 2024-09-13T07:02:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051180941000000001 2024-09-13T07:02:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727552+0000", 2024-09-13T07:02:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4376999999999997e-05 2024-09-13T07:02:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727612+0000", 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0198999999999997e-05 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94715 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.676311+0000", 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.459534651999999, 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051421327000000003, 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94715 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676311+0000", 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676311+0000", 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676311+0000", 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676320+0000", 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3989999999999993e-06 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676326+0000", 2024-09-13T07:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9939999999999997e-06 2024-09-13T07:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676331+0000", 2024-09-13T07:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.028e-06 2024-09-13T07:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727641+0000", 2024-09-13T07:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051309886999999998 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727688+0000", 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6755999999999998e-05 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727733+0000", 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4668e-05 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94716 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3629056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.676336+0000", 2024-09-13T07:02:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.459510153, 2024-09-13T07:02:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051505305000000001, 2024-09-13T07:02:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94716 2024-09-13T07:02:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676336+0000", 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676336+0000", 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676335+0000", 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676338+0000", 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3329999999999999e-06 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676343+0000", 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3569999999999997e-06 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676347+0000", 2024-09-13T07:02:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2769999999999999e-06 2024-09-13T07:02:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727752+0000", 2024-09-13T07:02:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051404367999999999 2024-09-13T07:02:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727796+0000", 2024-09-13T07:02:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4057999999999997e-05 2024-09-13T07:02:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727841+0000", 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5256000000000002e-05 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94717 3.1a 3:5d8d4a04:::rbd_data.1194a0700cb4.0000000000000003:1c [read 3694592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.676351+0000", 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.459494450000001, 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051655690999999997, 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94717 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676351+0000", 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676351+0000", 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676351+0000", 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676358+0000", 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6549999999999998e-06 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676363+0000", 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-13T07:02:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.676368+0000", 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727869+0000", 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051501208 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.727943+0000", 2024-09-13T07:02:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4120000000000002e-05 2024-09-13T07:02:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.728007+0000", 2024-09-13T07:02:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4096000000000004e-05 2024-09-13T07:02:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.227 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:46.229 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:46.229 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:46.226+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:46.329 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:46.326+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:46.385 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:46.386 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:46.396 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:46.430 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:46.430+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:46.505 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:46.506 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:46.506 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:46.506 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:46.506 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:46.516 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:46.531 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:46.528+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:46.564 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:02:46.592 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone32 from 0xbe3cbc3 (overlap 0x8d0349) to 0x0 2024-09-13T07:02:46.633 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:46.630+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:46.699 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:46.699 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:46.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:46.699 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:46.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.699 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1531 3.c 3:34898caa:::rbd_data.11943a018f34.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:46.699 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:46.699 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.648497012000007, 2024-09-13T07:02:46.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063168486999999995, 2024-09-13T07:02:46.699 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.699 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:46.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.700 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1531 2024-09-13T07:02:46.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.700 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:46.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049512+0000", 2024-09-13T07:02:46.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.049511+0000", 2024-09-13T07:02:46.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:46.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069234+0000", 2024-09-13T07:02:46.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019723920999999998 2024-09-13T07:02:46.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069269+0000", 2024-09-13T07:02:46.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4446999999999997e-05 2024-09-13T07:02:46.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069286+0000", 2024-09-13T07:02:46.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7405999999999999e-05 2024-09-13T07:02:46.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069370+0000", 2024-09-13T07:02:46.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3839999999999997e-05 2024-09-13T07:02:46.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.069511+0000", 2024-09-13T07:02:46.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014134299999999999 2024-09-13T07:02:46.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:46.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.070463+0000", 2024-09-13T07:02:46.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095167899999999998 2024-09-13T07:02:46.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:46.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105076+0000", 2024-09-13T07:02:46.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034613139000000001 2024-09-13T07:02:46.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:46.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112562+0000", 2024-09-13T07:02:46.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074861429999999998 2024-09-13T07:02:46.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:46.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112631+0000", 2024-09-13T07:02:46.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.112680+0000", 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9607000000000001e-05 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:7628 3.1f 3:fdf2e50b:::rbd_data.1194c5936dae.000000000000001e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1032192~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.935570854999995, 2024-09-13T07:02:46.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.630407088, 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7628 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762438+0000", 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762435+0000", 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999967 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762543+0000", 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010850500000000001 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762597+0000", 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4262000000000003e-05 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762626+0000", 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9176e-05 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762761+0000", 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013405599999999999 2024-09-13T07:02:46.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:46.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:34.762776+0000", 2024-09-13T07:02:46.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5679999999999999e-05 2024-09-13T07:02:46.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388492+0000", 2024-09-13T07:02:46.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.62571606300000004 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for missing object", 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.388614+0000", 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012126 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390837+0000", 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002223806 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.390963+0000", 2024-09-13T07:02:46.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012550300000000001 2024-09-13T07:02:46.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-13T07:02:46.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391127+0000", 2024-09-13T07:02:46.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163995 2024-09-13T07:02:46.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:46.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.391989+0000", 2024-09-13T07:02:46.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086230499999999995 2024-09-13T07:02:46.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:46.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392662+0000", 2024-09-13T07:02:46.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067260799999999999 2024-09-13T07:02:46.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:46.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392728+0000", 2024-09-13T07:02:46.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6501000000000006e-05 2024-09-13T07:02:46.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:35.392845+0000", 2024-09-13T07:02:46.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116665 2024-09-13T07:02:46.714 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.714 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.714 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.715 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:02:46.715 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:46.715 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 57.579420546000001, 2024-09-13T07:02:46.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:02:46.715 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.715 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:46.715 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.715 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.716 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:02:46.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.716 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:46.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:46.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:02:46.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:02:46.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:02:46.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:02:46.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:02:46.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:02:46.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:02:46.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:02:46.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:46.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:02:46.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:02:46.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:46.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:02:46.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:02:46.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:46.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:02:46.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:02:46.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:46.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:02:46.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:02:46.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:02:46.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:02:46.723 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.723 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.723 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.723 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48376 3.c 3:350dc8a4:::rbd_data.1194fc792839.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:46.723 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:46.723 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.468799703000002, 2024-09-13T07:02:46.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.068448320000000007, 2024-09-13T07:02:46.723 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.723 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:46.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.724 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.724 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48376 2024-09-13T07:02:46.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.724 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:46.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229209+0000", 2024-09-13T07:02:46.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.229207+0000", 2024-09-13T07:02:46.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:46.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.241745+0000", 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012537726000000001 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244174+0000", 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024284760000000002 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244198+0000", 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4629000000000001e-05 2024-09-13T07:02:46.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244312+0000", 2024-09-13T07:02:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114282 2024-09-13T07:02:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.244679+0000", 2024-09-13T07:02:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000366399 2024-09-13T07:02:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:46.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.246046+0000", 2024-09-13T07:02:46.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013672479999999999 2024-09-13T07:02:46.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:46.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.266272+0000", 2024-09-13T07:02:46.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020226381000000002 2024-09-13T07:02:46.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:46.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297510+0000", 2024-09-13T07:02:46.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031237576999999999 2024-09-13T07:02:46.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:46.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297595+0000", 2024-09-13T07:02:46.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5413000000000004e-05 2024-09-13T07:02:46.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297657+0000", 2024-09-13T07:02:46.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1860999999999996e-05 2024-09-13T07:02:46.729 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.729 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.730 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.730 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129740 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 774144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.730 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.620910+0000", 2024-09-13T07:02:46.730 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.077098602000000002, 2024-09-13T07:02:46.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.062679381000000006, 2024-09-13T07:02:46.730 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.730 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.730 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129740 2024-09-13T07:02:46.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.731 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620910+0000", 2024-09-13T07:02:46.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620910+0000", 2024-09-13T07:02:46.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620910+0000", 2024-09-13T07:02:46.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620912+0000", 2024-09-13T07:02:46.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4040000000000002e-06 2024-09-13T07:02:46.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620917+0000", 2024-09-13T07:02:46.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2329999999999997e-06 2024-09-13T07:02:46.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620922+0000", 2024-09-13T07:02:46.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6269999999999999e-06 2024-09-13T07:02:46.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.683124+0000", 2024-09-13T07:02:46.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.062202398999999998 2024-09-13T07:02:46.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.683236+0000", 2024-09-13T07:02:46.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111742 2024-09-13T07:02:46.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.683589+0000", 2024-09-13T07:02:46.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000353278 2024-09-13T07:02:46.734 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.734 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.735 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129741 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 827392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.735 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.620931+0000", 2024-09-13T07:02:46.735 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.077077846000000005, 2024-09-13T07:02:46.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063270913999999998, 2024-09-13T07:02:46.735 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.735 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.735 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.735 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.735 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129741 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620931+0000", 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620931+0000", 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620931+0000", 2024-09-13T07:02:46.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620933+0000", 2024-09-13T07:02:46.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.272e-06 2024-09-13T07:02:46.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620938+0000", 2024-09-13T07:02:46.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2220000000000001e-06 2024-09-13T07:02:46.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620942+0000", 2024-09-13T07:02:46.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4039999999999997e-06 2024-09-13T07:02:46.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.683652+0000", 2024-09-13T07:02:46.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.062709615999999996 2024-09-13T07:02:46.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.683776+0000", 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123628 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.684202+0000", 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00042610599999999999 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.739 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129742 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 806912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.740 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.620946+0000", 2024-09-13T07:02:46.740 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.077062801, 2024-09-13T07:02:46.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063743142000000003, 2024-09-13T07:02:46.740 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.740 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.740 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129742 2024-09-13T07:02:46.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.740 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620946+0000", 2024-09-13T07:02:46.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620946+0000", 2024-09-13T07:02:46.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620946+0000", 2024-09-13T07:02:46.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620948+0000", 2024-09-13T07:02:46.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0770000000000002e-06 2024-09-13T07:02:46.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620952+0000", 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.792e-06 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620957+0000", 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.684227+0000", 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063270205999999996 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.684336+0000", 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108461 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.684689+0000", 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035333199999999998 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129743 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 860160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.620961+0000", 2024-09-13T07:02:46.744 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.077047710000000005, 2024-09-13T07:02:46.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.064414609999999997, 2024-09-13T07:02:46.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.745 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.745 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129743 2024-09-13T07:02:46.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.745 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620961+0000", 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620961+0000", 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620961+0000", 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620963+0000", 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6220000000000001e-06 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620969+0000", 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5110000000000003e-06 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620974+0000", 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8210000000000001e-06 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.684938+0000", 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063964097999999997 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.685052+0000", 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114385 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.685376+0000", 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032352500000000002 2024-09-13T07:02:46.748 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.749 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.749 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129744 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 839680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.749 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.620977+0000", 2024-09-13T07:02:46.749 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.077031464999999993, 2024-09-13T07:02:46.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065784564000000004, 2024-09-13T07:02:46.749 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.749 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.749 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.749 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.749 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129744 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620977+0000", 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620977+0000", 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620977+0000", 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620979+0000", 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3250000000000002e-06 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620984+0000", 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0590000000000002e-06 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620989+0000", 2024-09-13T07:02:46.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4309999999999996e-06 2024-09-13T07:02:46.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.686250+0000", 2024-09-13T07:02:46.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065261171000000007 2024-09-13T07:02:46.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.686395+0000", 2024-09-13T07:02:46.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144831 2024-09-13T07:02:46.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.686762+0000", 2024-09-13T07:02:46.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036704499999999998 2024-09-13T07:02:46.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.753 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129745 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.620992+0000", 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.077016294999999999, 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066302905999999995, 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129745 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620992+0000", 2024-09-13T07:02:46.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620992+0000", 2024-09-13T07:02:46.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620992+0000", 2024-09-13T07:02:46.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.620994+0000", 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3829999999999999e-06 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621000+0000", 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.101e-06 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621004+0000", 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4279999999999998e-06 2024-09-13T07:02:46.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.686795+0000", 2024-09-13T07:02:46.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065791266000000001 2024-09-13T07:02:46.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.686915+0000", 2024-09-13T07:02:46.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119397 2024-09-13T07:02:46.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.687295+0000", 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038068800000000001 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129746 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 872448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.621007+0000", 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.077001275999999994, 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066793206999999993, 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129746 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621007+0000", 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621007+0000", 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621007+0000", 2024-09-13T07:02:46.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621010+0000", 2024-09-13T07:02:46.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4099999999999998e-06 2024-09-13T07:02:46.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621015+0000", 2024-09-13T07:02:46.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0189999999999997e-06 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621019+0000", 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4469999999999999e-06 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.687330+0000", 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066310956000000004 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.687449+0000", 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118602 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.687801+0000", 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035208800000000002 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129747 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.621023+0000", 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.076986049000000001, 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067353474999999996, 2024-09-13T07:02:46.762 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129747 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621023+0000", 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621023+0000", 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621022+0000", 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621030+0000", 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8529999999999992e-06 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621037+0000", 2024-09-13T07:02:46.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3640000000000004e-06 2024-09-13T07:02:46.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621041+0000", 2024-09-13T07:02:46.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8289999999999998e-06 2024-09-13T07:02:46.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.687867+0000", 2024-09-13T07:02:46.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066825879000000005 2024-09-13T07:02:46.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.687996+0000", 2024-09-13T07:02:46.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012922399999999999 2024-09-13T07:02:46.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.688376+0000", 2024-09-13T07:02:46.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00037965200000000002 2024-09-13T07:02:46.766 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.766 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129748 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.621046+0000", 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.076963127000000006, 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.068182534000000003, 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129748 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.767 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621046+0000", 2024-09-13T07:02:46.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621046+0000", 2024-09-13T07:02:46.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621045+0000", 2024-09-13T07:02:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621048+0000", 2024-09-13T07:02:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9119999999999998e-06 2024-09-13T07:02:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621053+0000", 2024-09-13T07:02:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.254e-06 2024-09-13T07:02:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621058+0000", 2024-09-13T07:02:46.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3889999999999998e-06 2024-09-13T07:02:46.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.688710+0000", 2024-09-13T07:02:46.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067652706000000007 2024-09-13T07:02:46.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.688837+0000", 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012629899999999999 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.689228+0000", 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00039137200000000002 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129749 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.621061+0000", 2024-09-13T07:02:46.772 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.076947475000000001, 2024-09-13T07:02:46.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069757730000000004, 2024-09-13T07:02:46.772 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.772 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.772 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.772 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.772 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.772 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129749 2024-09-13T07:02:46.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.772 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621061+0000", 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621061+0000", 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621061+0000", 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621063+0000", 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3379999999999999e-06 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621069+0000", 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5180000000000004e-06 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621073+0000", 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5959999999999997e-06 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.690193+0000", 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069119555999999999 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.690360+0000", 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016662299999999999 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.690819+0000", 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000459424 2024-09-13T07:02:46.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.776 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.776 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.776 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129750 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.776 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.621077+0000", 2024-09-13T07:02:46.776 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.076931834000000004, 2024-09-13T07:02:46.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070265782999999998, 2024-09-13T07:02:46.776 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.776 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.777 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129750 2024-09-13T07:02:46.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.777 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621077+0000", 2024-09-13T07:02:46.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621077+0000", 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621077+0000", 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621079+0000", 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4119999999999999e-06 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621085+0000", 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6670000000000001e-06 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621089+0000", 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7639999999999996e-06 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.690865+0000", 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069775366000000005 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.691041+0000", 2024-09-13T07:02:46.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017626799999999999 2024-09-13T07:02:46.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.691343+0000", 2024-09-13T07:02:46.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030164800000000001 2024-09-13T07:02:46.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.780 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.780 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129751 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.621093+0000", 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.076915477999999995, 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070670691999999993, 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129751 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621093+0000", 2024-09-13T07:02:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621093+0000", 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621093+0000", 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621096+0000", 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6620000000000001e-06 2024-09-13T07:02:46.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621105+0000", 2024-09-13T07:02:46.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5580000000000006e-06 2024-09-13T07:02:46.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621110+0000", 2024-09-13T07:02:46.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1959999999999999e-06 2024-09-13T07:02:46.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.691369+0000", 2024-09-13T07:02:46.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070259099000000005 2024-09-13T07:02:46.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.691459+0000", 2024-09-13T07:02:46.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9769000000000001e-05 2024-09-13T07:02:46.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.691764+0000", 2024-09-13T07:02:46.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000304705 2024-09-13T07:02:46.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.785 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.785 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129752 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.785 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.621115+0000", 2024-09-13T07:02:46.785 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.076894029000000003, 2024-09-13T07:02:46.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071070864999999997, 2024-09-13T07:02:46.785 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.786 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.786 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129752 2024-09-13T07:02:46.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.786 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621115+0000", 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621115+0000", 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621114+0000", 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621117+0000", 2024-09-13T07:02:46.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6910000000000002e-06 2024-09-13T07:02:46.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621126+0000", 2024-09-13T07:02:46.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2350000000000005e-06 2024-09-13T07:02:46.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621131+0000", 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5889999999999996e-06 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.691794+0000", 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070663365000000006 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.691867+0000", 2024-09-13T07:02:46.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2570999999999993e-05 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.692186+0000", 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031880199999999997 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129753 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.621135+0000", 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.076873346999999995, 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071785386000000007, 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.790 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.791 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129753 2024-09-13T07:02:46.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.791 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621135+0000", 2024-09-13T07:02:46.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621135+0000", 2024-09-13T07:02:46.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621135+0000", 2024-09-13T07:02:46.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621138+0000", 2024-09-13T07:02:46.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.649e-06 2024-09-13T07:02:46.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621144+0000", 2024-09-13T07:02:46.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9229999999999999e-06 2024-09-13T07:02:46.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621148+0000", 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6079999999999998e-06 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.692452+0000", 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071304211000000006 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.692571+0000", 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118262 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.692921+0000", 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035014900000000001 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129754 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.621152+0000", 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.076857129999999996, 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072271715, 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.795 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.796 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129754 2024-09-13T07:02:46.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.796 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621152+0000", 2024-09-13T07:02:46.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621152+0000", 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621151+0000", 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621158+0000", 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6239999999999996e-06 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621163+0000", 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.485e-06 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621168+0000", 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6450000000000004e-06 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.692967+0000", 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071799425 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.693100+0000", 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013254299999999999 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.693423+0000", 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032332200000000002 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.800 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129755 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 1056768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:46.800 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.621172+0000", 2024-09-13T07:02:46.800 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.076836949000000002, 2024-09-13T07:02:46.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.073060280000000005, 2024-09-13T07:02:46.800 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:46.800 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:46.800 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:46.800 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:46.800 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:46.800 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129755 2024-09-13T07:02:46.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.801 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:46.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:46.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621172+0000", 2024-09-13T07:02:46.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:46.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621172+0000", 2024-09-13T07:02:46.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:46.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:46.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621171+0000", 2024-09-13T07:02:46.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:46.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:46.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621174+0000", 2024-09-13T07:02:46.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6520000000000002e-06 2024-09-13T07:02:46.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:46.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621179+0000", 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1059999999999999e-06 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.621184+0000", 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.899e-06 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.693835+0000", 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072650632000000007 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:46.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.693983+0000", 2024-09-13T07:02:46.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014792399999999999 2024-09-13T07:02:46.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:46.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:46.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:46.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.694232+0000", 2024-09-13T07:02:46.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024944000000000002 2024-09-13T07:02:46.804 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.804 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.804 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.804 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:46.804 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:46.805 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:46.806 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:46.806 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:46.806 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:46.806 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:46.807 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:46.804+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:46.807 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:46.907 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:46.904+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:46.921 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:46.922 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:46.931 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:47.008 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:47.008+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:47.033 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:47.034 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:47.043 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:47.106 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:02:47.109 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:47.106+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:47.151 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:47.151 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:47.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:47.151 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:47.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.152 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:47.152 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:47.152 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 81.090606257000005, 2024-09-13T07:02:47.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:02:47.152 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.152 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:47.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.152 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:02:47.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.153 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:47.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:47.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:02:47.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:02:47.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:02:47.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:02:47.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:02:47.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:02:47.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:02:47.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:02:47.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:47.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:02:47.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:02:47.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:47.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:02:47.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:02:47.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:47.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:02:47.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:02:47.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:47.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:02:47.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:02:47.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:02:47.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:02:47.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.160 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.161 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93979 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.161 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:47.161 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532846943999999, 2024-09-13T07:02:47.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042253801000000001, 2024-09-13T07:02:47.161 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.162 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.162 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93979 2024-09-13T07:02:47.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.163 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:47.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:47.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:47.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616683+0000", 2024-09-13T07:02:47.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2820000000000001e-06 2024-09-13T07:02:47.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616691+0000", 2024-09-13T07:02:47.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0779999999999996e-06 2024-09-13T07:02:47.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616698+0000", 2024-09-13T07:02:47.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:02:47.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658799+0000", 2024-09-13T07:02:47.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042100763999999999 2024-09-13T07:02:47.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658869+0000", 2024-09-13T07:02:47.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0065000000000001e-05 2024-09-13T07:02:47.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658934+0000", 2024-09-13T07:02:47.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5071000000000001e-05 2024-09-13T07:02:47.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.169 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.170 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.170 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.170 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:02:47.171 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532822378999999, 2024-09-13T07:02:47.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:02:47.171 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.171 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.171 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.171 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.172 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.172 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:02:47.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.172 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:02:47.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:02:47.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:02:47.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:02:47.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:02:47.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:02:47.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:02:47.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:02:47.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:02:47.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:02:47.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:02:47.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:02:47.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:02:47.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:02:47.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:02:47.178 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.178 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.178 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.179 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.179 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:02:47.179 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532799411999999, 2024-09-13T07:02:47.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:02:47.179 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.179 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.179 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:02:47.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.180 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:02:47.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:02:47.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:02:47.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:02:47.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:02:47.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:02:47.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:02:47.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:02:47.183 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532782233999999, 2024-09-13T07:02:47.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:02:47.184 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.184 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.184 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:02:47.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.184 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:02:47.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:02:47.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:02:47.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:02:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:02:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:02:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:02:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:02:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:02:47.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.188 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.188 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.188 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:02:47.188 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532768368999999, 2024-09-13T07:02:47.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:02:47.188 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.188 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.189 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:02:47.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.190 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:02:47.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:02:47.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:02:47.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:02:47.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:02:47.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:02:47.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:02:47.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:02:47.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:02:47.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:02:47.194 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.194 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.194 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.194 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.194 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:02:47.194 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532755223999999, 2024-09-13T07:02:47.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:02:47.194 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.194 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.194 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:02:47.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:02:47.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:02:47.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:02:47.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.198 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.199 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:47.199 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532742017, 2024-09-13T07:02:47.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:02:47.199 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.199 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.199 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:02:47.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.199 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:47.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:47.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:47.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:02:47.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:02:47.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:02:47.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:02:47.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:02:47.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:02:47.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:02:47.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:02:47.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:02:47.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:02:47.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:02:47.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:02:47.204 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.204 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.204 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.204 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.204 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:47.204 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532727950000002, 2024-09-13T07:02:47.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:02:47.205 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.205 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.205 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.205 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.205 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.205 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:02:47.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.206 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:47.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:47.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:47.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:02:47.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:02:47.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:02:47.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:02:47.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:02:47.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:02:47.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:02:47.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:02:47.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:02:47.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.212 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.212 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:47.212 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532707936000001, 2024-09-13T07:02:47.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:02:47.212 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.212 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.213 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.213 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.213 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.213 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:02:47.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.213 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:47.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:47.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:47.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:02:47.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:02:47.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:02:47.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:02:47.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532688186000001, 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.218 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.219 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:02:47.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.219 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:02:47.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:02:47.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.223 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.223 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:02:47.223 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532663379999999, 2024-09-13T07:02:47.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:02:47.223 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.223 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.223 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:02:47.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.223 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:02:47.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:02:47.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:02:47.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:02:47.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:02:47.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:02:47.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:02:47.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:02:47.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:02:47.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:02:47.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:02:47.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:02:47.228 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532639114999998, 2024-09-13T07:02:47.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:02:47.228 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.228 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.228 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.228 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.228 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.228 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:02:47.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.228 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:02:47.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:02:47.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:02:47.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:02:47.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532597381999999, 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:02:47.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:02:47.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:02:47.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:02:47.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:02:47.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.236 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.532561092999998, 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:02:47.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:02:47.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:02:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:02:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:02:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:02:47.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:02:47.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:02:47.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:02:47.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:02:47.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:02:47.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:02:47.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:02:47.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.0894577610000002, 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.242 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.243 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.243 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.243 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.243 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:02:47.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.243 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:02:47.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:02:47.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:02:47.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:02:47.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:02:47.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:02:47.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:02:47.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:02:47.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:02:47.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:02:47.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:02:47.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:02:47.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:02:47.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.0894408789999996, 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.247 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:02:47.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:02:47.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:02:47.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:02:47.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:02:47.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:02:47.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:02:47.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:02:47.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.251 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.252 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126633 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2408448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:02:47.252 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061112+0000", 2024-09-13T07:02:47.252 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.0884149140000003, 2024-09-13T07:02:47.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042234086999999997, 2024-09-13T07:02:47.252 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.252 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.252 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126633 2024-09-13T07:02:47.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.253 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061112+0000", 2024-09-13T07:02:47.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061112+0000", 2024-09-13T07:02:47.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061112+0000", 2024-09-13T07:02:47.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061115+0000", 2024-09-13T07:02:47.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6569999999999999e-06 2024-09-13T07:02:47.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061125+0000", 2024-09-13T07:02:47.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0224000000000001e-05 2024-09-13T07:02:47.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061131+0000", 2024-09-13T07:02:47.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7130000000000002e-06 2024-09-13T07:02:47.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103179+0000", 2024-09-13T07:02:47.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042047582 2024-09-13T07:02:47.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103252+0000", 2024-09-13T07:02:47.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2822000000000002e-05 2024-09-13T07:02:47.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103346+0000", 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4539999999999999e-05 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.0883903220000004, 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.257 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.258 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:02:47.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.258 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:02:47.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:02:47.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:02:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:02:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:02:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:02:47.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:02:47.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:02:47.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:02:47.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:02:47.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:02:47.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:02:47.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:02:47.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:02:47.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:02:47.261 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.261 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.261 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.0867157729999999, 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.262 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:47.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:47.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:02:47.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:47.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:02:47.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:47.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:47.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:02:47.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:47.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:47.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:02:47.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:02:47.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:47.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:02:47.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:02:47.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:47.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:02:47.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:02:47.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:47.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:02:47.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:02:47.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:47.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:02:47.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:02:47.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:47.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:47.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:47.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:02:47.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:02:47.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.268 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:47.269 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:47.269 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:47.270 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:47.271 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:47.266+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:47.370 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:47.368+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:47.430 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:47.430 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:47.430 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:47.430 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:47.440 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:47.471 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:47.470+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:47.512 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:02:47.537 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:47.562 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:47.571 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:47.571 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:47.572 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:47.568+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:47.583 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:47.586 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:47.604 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:47.606 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:47.607 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:47.607 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:47.607 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:47.607 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:47.607 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:47.607 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:47.607 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:47.672 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:47.670+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:47.680 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:47.680 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:47.688 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:47.774 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:47.770+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:47.804 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:47.805 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:47.814 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:47.875 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:47.874+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:47.921 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:47.921 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:47.930 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:47.975 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:47.974+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:48.030 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:48.031 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:48.040 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:48.076 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:48.074+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:48.142 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:48.142 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:48.152 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:48.177 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:48.174+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:48.257 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:48.257 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:48.267 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:48.278 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:48.276+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:48.374 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:48.374 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:48.379 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:48.376+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:48.384 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:48.480 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:48.478+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:48.499 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7, 2] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:02:48.499 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:02:48.499 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T07:02:48.499 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T07:02:48.543 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:48.543 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:48.557 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:48.580 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:48.578+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:48.628 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-13T07:02:48.682 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:48.678+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:48.710 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:48.710 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:48.720 INFO:teuthology.orchestra.run.smithi118.stdout:3491 read 0x29d1ab2 thru 0x29dea18 (0xcf67 bytes) 2024-09-13T07:02:48.720 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:48.723 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:48.723 INFO:teuthology.orchestra.run.smithi118.stdout:3493 punch from 0x1ba96b0 to 0x1bae782, (0x50d2 bytes) 2024-09-13T07:02:48.730 INFO:teuthology.orchestra.run.smithi118.stdout:3494 write 0x5182eb4 thru 0x518c10f (0x925c bytes) 2024-09-13T07:02:48.768 INFO:teuthology.orchestra.run.smithi118.stdout:3495 read 0x80ff4c thru 0x811275 (0x132a bytes) 2024-09-13T07:02:48.770 INFO:teuthology.orchestra.run.smithi118.stdout:3497 trunc from 0xbe3cbc3 to 0xcffef1b 2024-09-13T07:02:48.779 INFO:teuthology.orchestra.run.smithi118.stdout:3499 write 0xacde36c thru 0xace07c5 (0x245a bytes) 2024-09-13T07:02:48.782 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:48.780+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:48.831 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:02:48.831 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:48.831 INFO:teuthology.orchestra.run.smithi118.stdout: "epoch": 103, 2024-09-13T07:02:48.831 INFO:teuthology.orchestra.run.smithi118.stdout: "fsid": "25bb30d1-9541-4e40-8659-18d83bb1bd8a", 2024-09-13T07:02:48.831 INFO:teuthology.orchestra.run.smithi118.stdout: "created": "2024-09-13T07:00:49.323883+0000", 2024-09-13T07:02:48.831 INFO:teuthology.orchestra.run.smithi118.stdout: "modified": "2024-09-13T07:02:48.501058+0000", 2024-09-13T07:02:48.831 INFO:teuthology.orchestra.run.smithi118.stdout: "last_up_change": "2024-09-13T07:01:00.146832+0000", 2024-09-13T07:02:48.831 INFO:teuthology.orchestra.run.smithi118.stdout: "last_in_change": "2024-09-13T07:02:43.471763+0000", 2024-09-13T07:02:48.831 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T07:02:48.831 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_num": 5804032, 2024-09-13T07:02:48.832 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_set": [ 2024-09-13T07:02:48.832 INFO:teuthology.orchestra.run.smithi118.stdout: "nodeep-scrub", 2024-09-13T07:02:48.832 INFO:teuthology.orchestra.run.smithi118.stdout: "pglog_hardlimit", 2024-09-13T07:02:48.832 INFO:teuthology.orchestra.run.smithi118.stdout: "purged_snapdirs", 2024-09-13T07:02:48.832 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_deletes", 2024-09-13T07:02:48.832 INFO:teuthology.orchestra.run.smithi118.stdout: "sortbitwise" 2024-09-13T07:02:48.832 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:48.832 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_version": 4, 2024-09-13T07:02:48.832 INFO:teuthology.orchestra.run.smithi118.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T07:02:48.832 INFO:teuthology.orchestra.run.smithi118.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T07:02:48.832 INFO:teuthology.orchestra.run.smithi118.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T07:02:48.832 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_snapshot": "", 2024-09-13T07:02:48.832 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_max": 3, 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: "max_osd": 8, 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: "require_min_compat_client": "luminous", 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: "min_compat_client": "luminous", 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: "require_osd_release": "reef", 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: "allow_crimson": false, 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: "pools": [ 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 1, 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": ".mgr", 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:00.675365+0000", 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 1, 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool", 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:02:48.833 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:02:48.834 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:02:48.834 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:02:48.834 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:02:48.834 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:02:48.834 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:02:48.834 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:02:48.834 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:02:48.834 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:02:48.834 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 1, 2024-09-13T07:02:48.834 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 1, 2024-09-13T07:02:48.834 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 1, 2024-09-13T07:02:48.834 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 1, 2024-09-13T07:02:48.835 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 1, 2024-09-13T07:02:48.835 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:02:48.835 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:02:48.835 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:02:48.835 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:02:48.835 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:02:48.835 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:02:48.835 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:02:48.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.835 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "16", 2024-09-13T07:02:48.835 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:02:48.835 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:02:48.835 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:02:48.836 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:02:48.836 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:02:48.836 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 0, 2024-09-13T07:02:48.836 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 0, 2024-09-13T07:02:48.836 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:02:48.836 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:02:48.836 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:02:48.836 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:02:48.836 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:02:48.836 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:02:48.836 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:02:48.836 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:02:48.836 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:02:48.837 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:02:48.838 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:02:48.838 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:02:48.838 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:02:48.838 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:02:48.838 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:02:48.838 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:02:48.838 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:02:48.838 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:02:48.838 INFO:teuthology.orchestra.run.smithi118.stdout: "options": { 2024-09-13T07:02:48.838 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_max": 32, 2024-09-13T07:02:48.838 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_min": 1 2024-09-13T07:02:48.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.838 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": {} 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 8, 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 8, 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.25, 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 2, 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 2, 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 0.92424774169921875, 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 0.92000001668930054, 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 0.92000001668930054 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.840 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 2, 2024-09-13T07:02:48.840 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": "rbd", 2024-09-13T07:02:48.840 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:07.404263+0000", 2024-09-13T07:02:48.840 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 8193, 2024-09-13T07:02:48.840 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:02:48.840 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:02:48.840 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:02:48.840 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:02:48.840 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:02:48.840 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:02:48.840 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:02:48.840 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:02:48.840 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:02:48.841 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:02:48.841 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:02:48.841 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 8, 2024-09-13T07:02:48.841 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 8, 2024-09-13T07:02:48.841 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 8, 2024-09-13T07:02:48.841 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 8, 2024-09-13T07:02:48.841 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 8, 2024-09-13T07:02:48.841 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:02:48.841 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:02:48.841 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:02:48.841 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:02:48.841 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:02:48.841 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:02:48.842 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:02:48.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.842 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "20", 2024-09-13T07:02:48.842 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:02:48.842 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:02:48.842 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:02:48.842 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:02:48.842 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:02:48.842 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 3, 2024-09-13T07:02:48.842 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 20, 2024-09-13T07:02:48.842 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:02:48.842 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:02:48.842 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:02:48.843 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:02:48.843 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:02:48.843 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:02:48.843 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:02:48.843 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:02:48.843 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:02:48.843 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:02:48.843 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:02:48.843 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:02:48.843 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:02:48.843 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:02:48.844 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:02:48.844 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:02:48.844 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:02:48.844 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:02:48.844 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:02:48.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.844 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:02:48.844 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:02:48.844 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:02:48.844 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:02:48.844 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:02:48.844 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:02:48.844 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: "options": {}, 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd": {} 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 3, 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 3, 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.75, 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 2.25, 2024-09-13T07:02:48.845 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 2.25, 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 0.92424774169921875, 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 0.92000001668930054, 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 0.92000001668930054 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 3, 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": "pool_client.0", 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:17.321011+0000", 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 8193, 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:02:48.846 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 32, 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 32, 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 32, 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 32, 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 32, 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:02:48.847 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:02:48.848 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:02:48.848 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:02:48.848 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:02:48.848 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:02:48.848 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:02:48.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.848 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "101", 2024-09-13T07:02:48.848 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:02:48.848 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:02:48.848 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:02:48.848 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:02:48.848 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:02:48.849 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 36, 2024-09-13T07:02:48.849 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 101, 2024-09-13T07:02:48.849 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:02:48.849 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:02:48.849 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:02:48.849 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:02:48.849 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:02:48.849 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:02:48.849 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:02:48.849 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:02:48.849 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:02:48.849 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:02:48.849 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:02:48.850 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: "options": {}, 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd": {} 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 2.25, 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 2.25, 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.75, 2024-09-13T07:02:48.851 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 1.690000057220459, 2024-09-13T07:02:48.852 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 1.690000057220459, 2024-09-13T07:02:48.852 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 0.92424774169921875, 2024-09-13T07:02:48.852 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 0.92000001668930054, 2024-09-13T07:02:48.852 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 0.92000001668930054 2024-09-13T07:02:48.852 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.852 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.852 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:48.852 INFO:teuthology.orchestra.run.smithi118.stdout: "osds": [ 2024-09-13T07:02:48.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.852 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 0, 2024-09-13T07:02:48.852 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "21736eee-00cd-4eef-9725-eca7776957b4", 2024-09-13T07:02:48.852 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:48.852 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 0, 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0, 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 22, 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6817", 2024-09-13T07:02:48.853 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:48.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.854 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.854 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6819", 2024-09-13T07:02:48.854 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:48.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.854 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.854 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:48.854 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.855 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.855 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6821", 2024-09-13T07:02:48.855 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:48.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.855 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.855 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6823", 2024-09-13T07:02:48.855 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:48.855 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.855 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.855 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:48.855 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6829", 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6831", 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:48.856 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.857 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.857 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6825", 2024-09-13T07:02:48.857 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:48.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.857 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.857 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6827", 2024-09-13T07:02:48.857 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:02:48.857 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.857 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.857 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6819/3731386737", 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6823/3731386737", 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6831/3731386737", 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6827/3731386737", 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 1, 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "05818db2-841d-406d-be17-8fbdab4fff69", 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:48.858 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.1844329833984375, 2024-09-13T07:02:48.859 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:48.859 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:48.859 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:48.859 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:48.859 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 84, 2024-09-13T07:02:48.859 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:48.859 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:48.859 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:48.859 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.859 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.859 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6816", 2024-09-13T07:02:48.860 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:48.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.860 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.860 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6818", 2024-09-13T07:02:48.860 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:48.860 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.860 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.860 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:48.860 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.860 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.861 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6820", 2024-09-13T07:02:48.861 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:48.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.861 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.861 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6822", 2024-09-13T07:02:48.861 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:48.861 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.861 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.861 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:48.861 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.862 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.862 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6828", 2024-09-13T07:02:48.862 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:48.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.862 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.862 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6830", 2024-09-13T07:02:48.862 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:48.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.862 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.862 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:48.862 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6824", 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6826", 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6818/876310814", 2024-09-13T07:02:48.863 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6822/876310814", 2024-09-13T07:02:48.864 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6830/876310814", 2024-09-13T07:02:48.864 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6826/876310814", 2024-09-13T07:02:48.864 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:48.864 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:48.864 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:48.864 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.864 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 2, 2024-09-13T07:02:48.864 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "2387761c-42d2-481b-b5b5-b16d3a943b22", 2024-09-13T07:02:48.864 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:48.864 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:48.865 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:02:48.865 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:48.865 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:48.865 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:48.865 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:48.865 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 102, 2024-09-13T07:02:48.866 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:48.866 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:48.866 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:48.866 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.866 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.866 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6808", 2024-09-13T07:02:48.866 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:48.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.866 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.866 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6809", 2024-09-13T07:02:48.867 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:48.867 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.867 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.867 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:48.867 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.867 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.867 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6810", 2024-09-13T07:02:48.867 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:48.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.867 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.868 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:48.868 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:48.869 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6811", 2024-09-13T07:02:48.869 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:48.869 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.869 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.869 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:48.869 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.869 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.869 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6814", 2024-09-13T07:02:48.869 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:48.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6815", 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6812", 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:48.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.871 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.871 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6813", 2024-09-13T07:02:48.871 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:02:48.871 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.871 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.871 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6809/690188762", 2024-09-13T07:02:48.871 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6811/690188762", 2024-09-13T07:02:48.872 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6815/690188762", 2024-09-13T07:02:48.872 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6813/690188762", 2024-09-13T07:02:48.872 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:48.872 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:48.872 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:48.872 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.872 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 3, 2024-09-13T07:02:48.872 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "56500e90-f76b-47c8-a649-893400359c53", 2024-09-13T07:02:48.872 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:48.872 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:48.872 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.8596343994140625, 2024-09-13T07:02:48.873 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 0.39398193359375, 2024-09-13T07:02:48.873 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:48.873 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:48.873 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:48.873 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 100, 2024-09-13T07:02:48.873 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:48.873 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:48.873 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:48.873 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.874 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.874 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6800", 2024-09-13T07:02:48.874 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:48.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.874 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.874 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6801", 2024-09-13T07:02:48.874 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:48.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.874 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.874 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:48.874 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6802", 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6803", 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:48.875 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.876 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.876 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6806", 2024-09-13T07:02:48.876 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:48.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.876 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.877 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6807", 2024-09-13T07:02:48.877 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:48.877 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.877 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.878 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:48.878 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.878 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.878 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6804", 2024-09-13T07:02:48.878 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:48.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.879 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.879 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6805", 2024-09-13T07:02:48.879 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:02:48.879 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.880 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.880 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6801/2198175481", 2024-09-13T07:02:48.880 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6803/2198175481", 2024-09-13T07:02:48.880 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6807/2198175481", 2024-09-13T07:02:48.880 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6805/2198175481", 2024-09-13T07:02:48.880 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:48.881 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:48.881 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:48.881 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.881 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 4, 2024-09-13T07:02:48.881 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "f594a612-aa07-4565-a09e-08368cf397b6", 2024-09-13T07:02:48.882 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:48.882 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:48.882 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.9017486572265625, 2024-09-13T07:02:48.882 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:48.882 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:48.882 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:48.882 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:48.883 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 102, 2024-09-13T07:02:48.883 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:48.883 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:48.883 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:48.883 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.884 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.884 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6810", 2024-09-13T07:02:48.884 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:48.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.884 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.885 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6813", 2024-09-13T07:02:48.885 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:48.885 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.885 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.885 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:48.885 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.886 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.886 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6817", 2024-09-13T07:02:48.886 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:48.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.886 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.887 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6819", 2024-09-13T07:02:48.887 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:48.887 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.887 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.887 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:48.887 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.888 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.888 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6827", 2024-09-13T07:02:48.888 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:48.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.888 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.888 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6830", 2024-09-13T07:02:48.889 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:48.889 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.889 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.889 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:48.889 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.890 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.890 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6822", 2024-09-13T07:02:48.890 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:48.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.891 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.891 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6825", 2024-09-13T07:02:48.891 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:02:48.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.892 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.892 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6813/1835601651", 2024-09-13T07:02:48.892 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6819/1835601651", 2024-09-13T07:02:48.893 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6830/1835601651", 2024-09-13T07:02:48.893 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6825/1835601651", 2024-09-13T07:02:48.893 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:48.893 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:48.893 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:48.893 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.894 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 5, 2024-09-13T07:02:48.894 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "1013b81f-e6b4-430b-938d-88759d43b881", 2024-09-13T07:02:48.894 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:48.895 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:48.895 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.95001220703125, 2024-09-13T07:02:48.895 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:48.895 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:48.896 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:48.896 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:48.896 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 101, 2024-09-13T07:02:48.896 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:48.896 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:48.896 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:48.896 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.897 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.897 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6800", 2024-09-13T07:02:48.897 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:48.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.897 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.897 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6801", 2024-09-13T07:02:48.897 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:48.898 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.898 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.898 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:48.898 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.899 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.899 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6802", 2024-09-13T07:02:48.899 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:48.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.899 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.900 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6803", 2024-09-13T07:02:48.900 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:48.900 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.900 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.900 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:48.900 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.901 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.901 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6806", 2024-09-13T07:02:48.901 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:48.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.901 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.901 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6807", 2024-09-13T07:02:48.901 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:48.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.901 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6804", 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6805", 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6801/2853696331", 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6803/2853696331", 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6807/2853696331", 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6805/2853696331", 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 6, 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "367f5f5b-5919-4534-b88c-229f327e4950", 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:48.903 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:48.904 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:02:48.904 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:48.904 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:48.904 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:48.904 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:48.904 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 93, 2024-09-13T07:02:48.904 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:48.904 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:48.904 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:48.904 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.904 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.904 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6808", 2024-09-13T07:02:48.905 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:48.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.905 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.905 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6812", 2024-09-13T07:02:48.905 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:48.905 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.905 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.905 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:48.905 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.906 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.906 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6815", 2024-09-13T07:02:48.906 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:48.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.907 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.908 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6818", 2024-09-13T07:02:48.908 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:48.908 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.908 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.909 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:48.909 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.909 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.909 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6828", 2024-09-13T07:02:48.910 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:48.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.910 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.910 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6831", 2024-09-13T07:02:48.910 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:48.910 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.910 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.910 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:48.910 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.911 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.911 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6821", 2024-09-13T07:02:48.911 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:48.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.911 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.912 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6824", 2024-09-13T07:02:48.912 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:02:48.912 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.912 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.912 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6812/131309660", 2024-09-13T07:02:48.912 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6818/131309660", 2024-09-13T07:02:48.912 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6831/131309660", 2024-09-13T07:02:48.912 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6824/131309660", 2024-09-13T07:02:48.912 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:48.912 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:48.912 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:48.913 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.913 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 7, 2024-09-13T07:02:48.913 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "90d9e486-e1ae-4305-80a6-6f483e534643", 2024-09-13T07:02:48.913 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:02:48.913 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:02:48.913 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:02:48.913 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:02:48.913 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:02:48.913 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:02:48.913 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:02:48.914 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 98, 2024-09-13T07:02:48.914 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:02:48.914 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:02:48.914 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:02:48.914 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.914 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.914 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6809", 2024-09-13T07:02:48.914 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:48.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.914 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.914 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6811", 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6814", 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.915 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6816", 2024-09-13T07:02:48.916 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:48.916 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.916 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.916 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:02:48.916 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.916 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.916 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6826", 2024-09-13T07:02:48.916 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:48.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.916 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.917 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6829", 2024-09-13T07:02:48.917 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:48.917 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.917 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.917 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:02:48.917 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:02:48.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.917 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:02:48.917 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6820", 2024-09-13T07:02:48.917 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:48.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.918 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:02:48.918 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6823", 2024-09-13T07:02:48.918 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:02:48.918 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.918 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.918 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6811/652173699", 2024-09-13T07:02:48.918 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6816/652173699", 2024-09-13T07:02:48.918 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6829/652173699", 2024-09-13T07:02:48.918 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6823/652173699", 2024-09-13T07:02:48.918 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:02:48.918 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_xinfo": [ 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 0, 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 54166, 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.059233+0000", 2024-09-13T07:02:48.919 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:48.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.920 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 1, 2024-09-13T07:02:48.920 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:48.920 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:48.920 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:48.920 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:48.920 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:48.920 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.083130+0000", 2024-09-13T07:02:48.920 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:48.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.920 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 2, 2024-09-13T07:02:48.921 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:48.921 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:48.921 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:48.921 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:48.921 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:48.921 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.848896+0000", 2024-09-13T07:02:48.921 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:48.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.921 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 3, 2024-09-13T07:02:48.921 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:48.921 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:48.922 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:48.922 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:48.922 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:48.922 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.847456+0000", 2024-09-13T07:02:48.922 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:48.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.922 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 4, 2024-09-13T07:02:48.922 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:48.922 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:48.922 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:48.922 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:48.923 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:48.923 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.914135+0000", 2024-09-13T07:02:48.923 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:48.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.923 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 5, 2024-09-13T07:02:48.923 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:48.923 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:48.923 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:48.923 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:48.923 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:48.923 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.834284+0000", 2024-09-13T07:02:48.923 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:48.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.924 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 6, 2024-09-13T07:02:48.924 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:48.924 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:48.924 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:48.924 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:48.924 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:48.924 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.003715+0000", 2024-09-13T07:02:48.924 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:48.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.924 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 7, 2024-09-13T07:02:48.925 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:02:48.925 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:02:48.925 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:02:48.925 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:02:48.925 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:02:48.925 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.033514+0000", 2024-09-13T07:02:48.925 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:02:48.925 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.925 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:48.925 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap": [ 2024-09-13T07:02:48.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:48.925 INFO:teuthology.orchestra.run.smithi118.stdout: "pgid": "2.5", 2024-09-13T07:02:48.925 INFO:teuthology.orchestra.run.smithi118.stdout: "osds": [ 2024-09-13T07:02:48.926 INFO:teuthology.orchestra.run.smithi118.stdout: 6, 2024-09-13T07:02:48.926 INFO:teuthology.orchestra.run.smithi118.stdout: 4 2024-09-13T07:02:48.926 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:48.926 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.926 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:02:48.926 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap_items": [], 2024-09-13T07:02:48.926 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap_primaries": [], 2024-09-13T07:02:48.926 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_temp": [], 2024-09-13T07:02:48.926 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_temp": [], 2024-09-13T07:02:48.926 INFO:teuthology.orchestra.run.smithi118.stdout: "blocklist": {}, 2024-09-13T07:02:48.926 INFO:teuthology.orchestra.run.smithi118.stdout: "range_blocklist": {}, 2024-09-13T07:02:48.927 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profiles": { 2024-09-13T07:02:48.927 INFO:teuthology.orchestra.run.smithi118.stdout: "default": { 2024-09-13T07:02:48.927 INFO:teuthology.orchestra.run.smithi118.stdout: "crush-failure-domain": "osd", 2024-09-13T07:02:48.927 INFO:teuthology.orchestra.run.smithi118.stdout: "k": "2", 2024-09-13T07:02:48.927 INFO:teuthology.orchestra.run.smithi118.stdout: "m": "1", 2024-09-13T07:02:48.927 INFO:teuthology.orchestra.run.smithi118.stdout: "plugin": "jerasure", 2024-09-13T07:02:48.927 INFO:teuthology.orchestra.run.smithi118.stdout: "technique": "reed_sol_van" 2024-09-13T07:02:48.927 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:48.927 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps_queue": [], 2024-09-13T07:02:48.927 INFO:teuthology.orchestra.run.smithi118.stdout: "new_removed_snaps": [], 2024-09-13T07:02:48.928 INFO:teuthology.orchestra.run.smithi118.stdout: "new_purged_snaps": [], 2024-09-13T07:02:48.928 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_node_flags": {}, 2024-09-13T07:02:48.928 INFO:teuthology.orchestra.run.smithi118.stdout: "device_class_flags": {}, 2024-09-13T07:02:48.929 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode": { 2024-09-13T07:02:48.929 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode_enabled": false, 2024-09-13T07:02:48.929 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_bucket_count": 0, 2024-09-13T07:02:48.929 INFO:teuthology.orchestra.run.smithi118.stdout: "degraded_stretch_mode": 0, 2024-09-13T07:02:48.929 INFO:teuthology.orchestra.run.smithi118.stdout: "recovering_stretch_mode": 0, 2024-09-13T07:02:48.929 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode_bucket": 0 2024-09-13T07:02:48.929 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:48.929 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:48.930 INFO:teuthology.orchestra.run.smithi118.stdout:3500 trunc from 0xcffef1b to 0x4f31f0e 2024-09-13T07:02:48.931 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:48.932 INFO:tasks.thrashosds.thrasher:j is {'epoch': 103, 'fsid': '25bb30d1-9541-4e40-8659-18d83bb1bd8a', 'created': '2024-09-13T07:00:49.323883+0000', 'modified': '2024-09-13T07:02:48.501058+0000', 'last_up_change': '2024-09-13T07:01:00.146832+0000', 'last_in_change': '2024-09-13T07:02:43.471763+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T07:01:00.675365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9242477416992188, 'average_primary_affinity': 0.9200000166893005, 'average_primary_affinity_weighted': 0.9200000166893005}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T07:01:07.404263+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9242477416992188, 'average_primary_affinity': 0.9200000166893005, 'average_primary_affinity_weighted': 0.9200000166893005}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T07:01:17.321011+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 36, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 0.75, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9242477416992188, 'average_primary_affinity': 0.9200000166893005, 'average_primary_affinity_weighted': 0.9200000166893005}}], 'osds': [{'osd': 0, 'uuid': '21736eee-00cd-4eef-9725-eca7776957b4', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6817', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6819', 'nonce': 3731386737}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6821', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6823', 'nonce': 3731386737}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6829', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6831', 'nonce': 3731386737}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6825', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6827', 'nonce': 3731386737}]}, 'public_addr': '172.21.15.118:6819/3731386737', 'cluster_addr': '172.21.15.118:6823/3731386737', 'heartbeat_back_addr': '172.21.15.118:6831/3731386737', 'heartbeat_front_addr': '172.21.15.118:6827/3731386737', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '05818db2-841d-406d-be17-8fbdab4fff69', 'up': 1, 'in': 1, 'weight': 0.1844329833984375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6816', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6818', 'nonce': 876310814}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6820', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6822', 'nonce': 876310814}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6828', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6830', 'nonce': 876310814}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6824', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6826', 'nonce': 876310814}]}, 'public_addr': '172.21.15.118:6818/876310814', 'cluster_addr': '172.21.15.118:6822/876310814', 'heartbeat_back_addr': '172.21.15.118:6830/876310814', 'heartbeat_front_addr': '172.21.15.118:6826/876310814', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2387761c-42d2-481b-b5b5-b16d3a943b22', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 102, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6808', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6809', 'nonce': 690188762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6810', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6811', 'nonce': 690188762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6814', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6815', 'nonce': 690188762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6812', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6813', 'nonce': 690188762}]}, 'public_addr': '172.21.15.118:6809/690188762', 'cluster_addr': '172.21.15.118:6811/690188762', 'heartbeat_back_addr': '172.21.15.118:6815/690188762', 'heartbeat_front_addr': '172.21.15.118:6813/690188762', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '56500e90-f76b-47c8-a649-893400359c53', 'up': 1, 'in': 1, 'weight': 0.8596343994140625, 'primary_affinity': 0.39398193359375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6800', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6801', 'nonce': 2198175481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6802', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6803', 'nonce': 2198175481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6806', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6807', 'nonce': 2198175481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6804', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6805', 'nonce': 2198175481}]}, 'public_addr': '172.21.15.118:6801/2198175481', 'cluster_addr': '172.21.15.118:6803/2198175481', 'heartbeat_back_addr': '172.21.15.118:6807/2198175481', 'heartbeat_front_addr': '172.21.15.118:6805/2198175481', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f594a612-aa07-4565-a09e-08368cf397b6', 'up': 1, 'in': 1, 'weight': 0.9017486572265625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 102, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 1835601651}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6817', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 1835601651}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6827', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6830', 'nonce': 1835601651}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 1835601651}]}, 'public_addr': '172.21.15.164:6813/1835601651', 'cluster_addr': '172.21.15.164:6819/1835601651', 'heartbeat_back_addr': '172.21.15.164:6830/1835601651', 'heartbeat_front_addr': '172.21.15.164:6825/1835601651', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1013b81f-e6b4-430b-938d-88759d43b881', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 101, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 2853696331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 2853696331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 2853696331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 2853696331}]}, 'public_addr': '172.21.15.164:6801/2853696331', 'cluster_addr': '172.21.15.164:6803/2853696331', 'heartbeat_back_addr': '172.21.15.164:6807/2853696331', 'heartbeat_front_addr': '172.21.15.164:6805/2853696331', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '367f5f5b-5919-4534-b88c-229f327e4950', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6812', 'nonce': 131309660}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6815', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6818', 'nonce': 131309660}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 131309660}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6821', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6824', 'nonce': 131309660}]}, 'public_addr': '172.21.15.164:6812/131309660', 'cluster_addr': '172.21.15.164:6818/131309660', 'heartbeat_back_addr': '172.21.15.164:6831/131309660', 'heartbeat_front_addr': '172.21.15.164:6824/131309660', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '90d9e486-e1ae-4305-80a6-6f483e534643', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 98, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6809', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 652173699}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6816', 'nonce': 652173699}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 652173699}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 652173699}]}, 'public_addr': '172.21.15.164:6811/652173699', 'cluster_addr': '172.21.15.164:6816/652173699', 'heartbeat_back_addr': '172.21.15.164:6829/652173699', 'heartbeat_front_addr': '172.21.15.164:6823/652173699', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 54166, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.059233+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.083130+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.848896+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.847456+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.914135+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.834284+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.003715+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.033514+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.5', 'osds': [6, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T07:02:48.932 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:02:48.934 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:48.930+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:49.019 INFO:teuthology.orchestra.run.smithi118.stdout:3501 read 0x5bbd2e thru 0x5cb88c (0xfb5f bytes) 2024-09-13T07:02:49.020 INFO:teuthology.orchestra.run.smithi118.stdout:3502 clone 34 order 21 su 524288 sc 10 2024-09-13T07:02:49.035 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:49.032+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:49.105 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:49.105 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:49.115 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:49.137 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:49.134+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:49.209 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:49.209 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:49.219 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:49.237 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:49.236+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:49.298 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:02:49.300 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:02:49.312 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-13T07:02:48.625152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1334895172,"num_objects":1537,"num_object_clones":350,"num_object_copies":3074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":105,"num_objects_unfound":0,"num_objects_dirty":1537,"num_whiteouts":649,"num_read":162141,"num_read_kb":559462,"num_write":13040,"num_write_kb":307127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":739,"num_bytes_recovered":31484389,"num_keys_recovered":153,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10094,"ondisk_log_size":10094,"up":82,"acting":77,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":6,"kb":660602880,"kb_used":542084,"kb_used_data":279956,"kb_used_omap":10,"kb_used_meta":262005,"kb_avail":660060796,"statfs":{"total":676457349120,"available":675902255104,"internally_reserved":0,"allocated":286674944,"data_stored":319286107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11150,"internal_metadata":268293234},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-68985063,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":105,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":7338,"num_read_kb":22620,"num_write":1216,"num_write_kb":9440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":2594561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001029"},"pg_stats":[{"pgid":"3.1f","version":"100'180","reported_seq":5586,"reported_epoch":100,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-13T07:02:45.687610+0000","last_change":"2024-09-13T07:02:45.561681+0000","last_active":"2024-09-13T07:02:45.687610+0000","last_peered":"2024-09-13T07:02:45.687610+0000","last_clean":"2024-09-13T07:02:43.365558+0000","last_became_active":"2024-09-13T07:02:45.556657+0000","last_became_peered":"2024-09-13T07:02:45.556657+0000","last_unstale":"2024-09-13T07:02:45.687610+0000","last_undegraded":"2024-09-13T07:02:45.687610+0000","last_fullsized":"2024-09-13T07:02:45.091786+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:16:16.435135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013321120000000001,"stat_sum":{"num_bytes":55956462,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":23,"num_read":5204,"num_read_kb":21698,"num_write":262,"num_write_kb":6733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3428702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5],"avail_no_missing":["5","6"],"object_location_counts":[{"shards":"5","objects":35},{"shards":"5,6","objects":14}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"100'213","reported_seq":3049,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.600552+0000","last_change":"2024-09-13T07:02:26.373963+0000","last_active":"2024-09-13T07:02:45.600552+0000","last_peered":"2024-09-13T07:02:45.600552+0000","last_clean":"2024-09-13T07:02:45.600552+0000","last_became_active":"2024-09-13T07:02:26.373467+0000","last_became_peered":"2024-09-13T07:02:26.373467+0000","last_unstale":"2024-09-13T07:02:45.600552+0000","last_undegraded":"2024-09-13T07:02:45.600552+0000","last_fullsized":"2024-09-13T07:02:45.600552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:30.228938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001950586,"stat_sum":{"num_bytes":75433516,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":24,"num_read":2868,"num_read_kb":30468,"num_write":236,"num_write_kb":4453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"96'276","reported_seq":1704,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.088739+0000","last_change":"2024-09-13T07:01:21.293288+0000","last_active":"2024-09-13T07:02:44.088739+0000","last_peered":"2024-09-13T07:02:44.088739+0000","last_clean":"2024-09-13T07:02:44.088739+0000","last_became_active":"2024-09-13T07:01:19.276445+0000","last_became_peered":"2024-09-13T07:01:19.276445+0000","last_unstale":"2024-09-13T07:02:44.088739+0000","last_undegraded":"2024-09-13T07:02:44.088739+0000","last_fullsized":"2024-09-13T07:02:44.088739+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:16:37.731881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076373900000000004,"stat_sum":{"num_bytes":18737018,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":26,"num_read":2390,"num_read_kb":10875,"num_write":339,"num_write_kb":5199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"100'209","reported_seq":4397,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.582143+0000","last_change":"2024-09-13T07:01:21.292234+0000","last_active":"2024-09-13T07:02:45.582143+0000","last_peered":"2024-09-13T07:02:45.582143+0000","last_clean":"2024-09-13T07:02:45.582143+0000","last_became_active":"2024-09-13T07:01:19.276459+0000","last_became_peered":"2024-09-13T07:01:19.276459+0000","last_unstale":"2024-09-13T07:02:45.582143+0000","last_undegraded":"2024-09-13T07:02:45.582143+0000","last_fullsized":"2024-09-13T07:02:45.582143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:50:50.173262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050866599999999996,"stat_sum":{"num_bytes":34741838,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":22,"num_read":4834,"num_read_kb":13417,"num_write":255,"num_write_kb":5405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"100'180","reported_seq":3578,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.638512+0000","last_change":"2024-09-13T07:01:34.715025+0000","last_active":"2024-09-13T07:02:45.638512+0000","last_peered":"2024-09-13T07:02:45.638512+0000","last_clean":"2024-09-13T07:02:45.638512+0000","last_became_active":"2024-09-13T07:01:34.676996+0000","last_became_peered":"2024-09-13T07:01:34.676996+0000","last_unstale":"2024-09-13T07:02:45.638512+0000","last_undegraded":"2024-09-13T07:02:45.638512+0000","last_fullsized":"2024-09-13T07:02:45.638512+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:26:51.849086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083111000000000003,"stat_sum":{"num_bytes":59916002,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":20,"num_read":3397,"num_read_kb":24674,"num_write":240,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":67840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"100'450","reported_seq":3652,"reported_epoch":100,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-13T07:02:45.652785+0000","last_change":"2024-09-13T07:02:45.564061+0000","last_active":"2024-09-13T07:02:45.652785+0000","last_peered":"2024-09-13T07:02:45.652785+0000","last_clean":"2024-09-13T07:02:43.449892+0000","last_became_active":"2024-09-13T07:02:45.561736+0000","last_became_peered":"2024-09-13T07:02:45.561736+0000","last_unstale":"2024-09-13T07:02:45.652785+0000","last_undegraded":"2024-09-13T07:02:45.093481+0000","last_fullsized":"2024-09-13T07:02:45.090357+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:44:23.308305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001799911,"stat_sum":{"num_bytes":25316213,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":64,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":5773,"num_read_kb":11473,"num_write":525,"num_write_kb":7418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":104677,"num_keys_recovered":34,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4],"avail_no_missing":["4","7"],"object_location_counts":[{"shards":"4","objects":1},{"shards":"4,7","objects":36}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"97'241","reported_seq":5594,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.087296+0000","last_change":"2024-09-13T07:02:23.332539+0000","last_active":"2024-09-13T07:02:44.087296+0000","last_peered":"2024-09-13T07:02:44.087296+0000","last_clean":"2024-09-13T07:02:44.087296+0000","last_became_active":"2024-09-13T07:02:23.331645+0000","last_became_peered":"2024-09-13T07:02:23.331645+0000","last_unstale":"2024-09-13T07:02:44.087296+0000","last_undegraded":"2024-09-13T07:02:44.087296+0000","last_fullsized":"2024-09-13T07:02:44.087296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:26.278406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055727200000000004,"stat_sum":{"num_bytes":33539978,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":14,"num_read":5578,"num_read_kb":10006,"num_write":307,"num_write_kb":12314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":1288675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"100'282","reported_seq":5017,"reported_epoch":100,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-13T07:02:45.658509+0000","last_change":"2024-09-13T07:02:45.561441+0000","last_active":"2024-09-13T07:02:45.658509+0000","last_peered":"2024-09-13T07:02:45.658509+0000","last_clean":"2024-09-13T07:02:43.387792+0000","last_became_active":"2024-09-13T07:02:45.560707+0000","last_became_peered":"2024-09-13T07:02:45.560707+0000","last_unstale":"2024-09-13T07:02:45.658509+0000","last_undegraded":"2024-09-13T07:02:45.089764+0000","last_fullsized":"2024-09-13T07:02:45.088743+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:02:21.557578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082396300000000004,"stat_sum":{"num_bytes":37791840,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":89,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":27,"num_read":6175,"num_read_kb":15501,"num_write":343,"num_write_kb":8853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":4637802,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[3],"avail_no_missing":["3","7"],"object_location_counts":[{"shards":"3","objects":5},{"shards":"3,7","objects":51}],"blocked_by":[],"up_primary":2,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"100'419","reported_seq":9661,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.093648+0000","last_change":"2024-09-13T07:01:50.079211+0000","last_active":"2024-09-13T07:02:45.093648+0000","last_peered":"2024-09-13T07:02:45.093648+0000","last_clean":"2024-09-13T07:02:45.093648+0000","last_became_active":"2024-09-13T07:01:50.078679+0000","last_became_peered":"2024-09-13T07:01:50.078679+0000","last_unstale":"2024-09-13T07:02:45.093648+0000","last_undegraded":"2024-09-13T07:02:45.093648+0000","last_fullsized":"2024-09-13T07:02:45.093648+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:30:15.075968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001182647,"stat_sum":{"num_bytes":34591492,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":23,"num_read":10842,"num_read_kb":23370,"num_write":562,"num_write_kb":14629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2756020,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"100'635","reported_seq":10752,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.685527+0000","last_change":"2024-09-13T07:01:21.293012+0000","last_active":"2024-09-13T07:02:45.685527+0000","last_peered":"2024-09-13T07:02:45.685527+0000","last_clean":"2024-09-13T07:02:45.685527+0000","last_became_active":"2024-09-13T07:01:19.277380+0000","last_became_peered":"2024-09-13T07:01:19.277380+0000","last_unstale":"2024-09-13T07:02:45.685527+0000","last_undegraded":"2024-09-13T07:02:45.685527+0000","last_fullsized":"2024-09-13T07:02:45.685527+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:05:57.870409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010463009999999999,"stat_sum":{"num_bytes":34043120,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":12531,"num_read_kb":16911,"num_write":1045,"num_write_kb":4907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"100'184","reported_seq":819,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.637903+0000","last_change":"2024-09-13T07:01:21.314353+0000","last_active":"2024-09-13T07:02:45.637903+0000","last_peered":"2024-09-13T07:02:45.637903+0000","last_clean":"2024-09-13T07:02:45.637903+0000","last_became_active":"2024-09-13T07:01:19.280203+0000","last_became_peered":"2024-09-13T07:01:19.280203+0000","last_unstale":"2024-09-13T07:02:45.637903+0000","last_undegraded":"2024-09-13T07:02:45.637903+0000","last_fullsized":"2024-09-13T07:02:45.637903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:38.598517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024699230000000002,"stat_sum":{"num_bytes":18124233,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":1222,"num_read_kb":2785,"num_write":181,"num_write_kb":1680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"97'299","reported_seq":4021,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.094078+0000","last_change":"2024-09-13T07:02:09.224417+0000","last_active":"2024-09-13T07:02:45.094078+0000","last_peered":"2024-09-13T07:02:45.094078+0000","last_clean":"2024-09-13T07:02:45.094078+0000","last_became_active":"2024-09-13T07:02:09.223575+0000","last_became_peered":"2024-09-13T07:02:09.223575+0000","last_unstale":"2024-09-13T07:02:45.094078+0000","last_undegraded":"2024-09-13T07:02:45.094078+0000","last_fullsized":"2024-09-13T07:02:45.094078+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:57:31.053474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001937825,"stat_sum":{"num_bytes":44659456,"num_objects":60,"num_object_clones":13,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":29,"num_read":4190,"num_read_kb":16205,"num_write":347,"num_write_kb":12845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":4572463,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":95,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.094428+0000","last_change":"2024-09-13T07:01:11.221939+0000","last_active":"2024-09-13T07:02:44.094428+0000","last_peered":"2024-09-13T07:02:44.094428+0000","last_clean":"2024-09-13T07:02:44.094428+0000","last_became_active":"2024-09-13T07:01:09.212302+0000","last_became_peered":"2024-09-13T07:01:09.212302+0000","last_unstale":"2024-09-13T07:02:44.094428+0000","last_undegraded":"2024-09-13T07:02:44.094428+0000","last_fullsized":"2024-09-13T07:02:44.094428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:49.220240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"100'162","reported_seq":740,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.094385+0000","last_change":"2024-09-13T07:02:20.373122+0000","last_active":"2024-09-13T07:02:45.094385+0000","last_peered":"2024-09-13T07:02:45.094385+0000","last_clean":"2024-09-13T07:02:45.094385+0000","last_became_active":"2024-09-13T07:02:20.311230+0000","last_became_peered":"2024-09-13T07:02:20.311230+0000","last_unstale":"2024-09-13T07:02:45.094385+0000","last_undegraded":"2024-09-13T07:02:45.094385+0000","last_fullsized":"2024-09-13T07:02:45.094385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:07:45.257392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038364399999999999,"stat_sum":{"num_bytes":26603297,"num_objects":37,"num_object_clones":3,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":17,"num_read":482,"num_read_kb":7036,"num_write":142,"num_write_kb":3004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":268161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":32,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.094064+0000","last_change":"2024-09-13T07:02:20.313654+0000","last_active":"2024-09-13T07:02:44.094064+0000","last_peered":"2024-09-13T07:02:44.094064+0000","last_clean":"2024-09-13T07:02:44.094064+0000","last_became_active":"2024-09-13T07:02:20.313362+0000","last_became_peered":"2024-09-13T07:02:20.313362+0000","last_unstale":"2024-09-13T07:02:44.094064+0000","last_undegraded":"2024-09-13T07:02:44.094064+0000","last_fullsized":"2024-09-13T07:02:44.094064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:58.845083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"100'330","reported_seq":3005,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.671809+0000","last_change":"2024-09-13T07:02:26.368320+0000","last_active":"2024-09-13T07:02:45.671809+0000","last_peered":"2024-09-13T07:02:45.671809+0000","last_clean":"2024-09-13T07:02:45.671809+0000","last_became_active":"2024-09-13T07:02:26.367717+0000","last_became_peered":"2024-09-13T07:02:26.367717+0000","last_unstale":"2024-09-13T07:02:45.671809+0000","last_undegraded":"2024-09-13T07:02:45.671809+0000","last_fullsized":"2024-09-13T07:02:45.671809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:36:49.852981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016908909999999999,"stat_sum":{"num_bytes":68092888,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":24,"num_read":3593,"num_read_kb":23248,"num_write":388,"num_write_kb":5038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.093936+0000","last_change":"2024-09-13T07:02:38.457050+0000","last_active":"2024-09-13T07:02:44.093936+0000","last_peered":"2024-09-13T07:02:44.093936+0000","last_clean":"2024-09-13T07:02:44.093936+0000","last_became_active":"2024-09-13T07:02:38.456519+0000","last_became_peered":"2024-09-13T07:02:38.456519+0000","last_unstale":"2024-09-13T07:02:44.093936+0000","last_undegraded":"2024-09-13T07:02:44.093936+0000","last_fullsized":"2024-09-13T07:02:44.093936+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:37:49.633079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"90'205","reported_seq":738,"reported_epoch":100,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-13T07:02:45.557672+0000","last_change":"2024-09-13T07:02:45.558001+0000","last_active":"2024-09-13T07:02:45.557672+0000","last_peered":"2024-09-13T07:02:45.557672+0000","last_clean":"2024-09-13T07:02:42.466437+0000","last_became_active":"2024-09-13T07:02:45.557069+0000","last_became_peered":"2024-09-13T07:02:45.557069+0000","last_unstale":"2024-09-13T07:02:45.557672+0000","last_undegraded":"2024-09-13T07:02:45.093044+0000","last_fullsized":"2024-09-13T07:02:45.091612+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:04:06.996007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085579199999999997,"stat_sum":{"num_bytes":38500725,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":71,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":29,"num_read":599,"num_read_kb":3913,"num_write":253,"num_write_kb":4115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":156677,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[5],"avail_no_missing":["5","6"],"object_location_counts":[{"shards":"5,6","objects":51}],"blocked_by":[],"up_primary":2,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":85,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.092923+0000","last_change":"2024-09-13T07:02:44.092174+0000","last_active":"2024-09-13T07:02:44.092923+0000","last_peered":"2024-09-13T07:02:44.092923+0000","last_clean":"2024-09-13T07:02:44.092923+0000","last_became_active":"2024-09-13T07:02:44.091876+0000","last_became_peered":"2024-09-13T07:02:44.091876+0000","last_unstale":"2024-09-13T07:02:44.092923+0000","last_undegraded":"2024-09-13T07:02:44.092923+0000","last_fullsized":"2024-09-13T07:02:44.092923+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:40:01.958529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.5","version":"100'373","reported_seq":3019,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.640306+0000","last_change":"2024-09-13T07:01:21.293697+0000","last_active":"2024-09-13T07:02:45.640306+0000","last_peered":"2024-09-13T07:02:45.640306+0000","last_clean":"2024-09-13T07:02:45.640306+0000","last_became_active":"2024-09-13T07:01:19.279719+0000","last_became_peered":"2024-09-13T07:01:19.279719+0000","last_unstale":"2024-09-13T07:02:45.640306+0000","last_undegraded":"2024-09-13T07:02:45.640306+0000","last_fullsized":"2024-09-13T07:02:45.640306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:05:06.593984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085976499999999997,"stat_sum":{"num_bytes":42233778,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":23,"num_read":4247,"num_read_kb":15238,"num_write":469,"num_write_kb":3212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"97'272","reported_seq":5127,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.089257+0000","last_change":"2024-09-13T07:01:47.540796+0000","last_active":"2024-09-13T07:02:44.089257+0000","last_peered":"2024-09-13T07:02:44.089257+0000","last_clean":"2024-09-13T07:02:44.089257+0000","last_became_active":"2024-09-13T07:01:47.491983+0000","last_became_peered":"2024-09-13T07:01:47.491983+0000","last_unstale":"2024-09-13T07:02:44.089257+0000","last_undegraded":"2024-09-13T07:02:44.089257+0000","last_fullsized":"2024-09-13T07:02:44.089257+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:18.382486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017265959999999999,"stat_sum":{"num_bytes":25679662,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":5464,"num_read_kb":18945,"num_write":386,"num_write_kb":9416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":3083106,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":110,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.085672+0000","last_change":"2024-09-13T07:02:20.315421+0000","last_active":"2024-09-13T07:02:44.085672+0000","last_peered":"2024-09-13T07:02:44.085672+0000","last_clean":"2024-09-13T07:02:44.085672+0000","last_became_active":"2024-09-13T07:02:20.312417+0000","last_became_peered":"2024-09-13T07:02:20.312417+0000","last_unstale":"2024-09-13T07:02:44.085672+0000","last_undegraded":"2024-09-13T07:02:44.085672+0000","last_fullsized":"2024-09-13T07:02:44.085672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:48.435411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"100'323","reported_seq":12028,"reported_epoch":100,"state":"active+recovery_wait+remapped","last_fresh":"2024-09-13T07:02:45.582110+0000","last_change":"2024-09-13T07:02:45.564688+0000","last_active":"2024-09-13T07:02:45.582110+0000","last_peered":"2024-09-13T07:02:45.582110+0000","last_clean":"2024-09-13T07:02:43.385619+0000","last_became_active":"2024-09-13T07:02:45.562521+0000","last_became_peered":"2024-09-13T07:02:45.562521+0000","last_unstale":"2024-09-13T07:02:45.582110+0000","last_undegraded":"2024-09-13T07:02:45.582110+0000","last_fullsized":"2024-09-13T07:02:45.582110+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:24:05.282598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065804299999999997,"stat_sum":{"num_bytes":64604640,"num_objects":63,"num_object_clones":16,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":91,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":24,"num_read":11939,"num_read_kb":24314,"num_write":359,"num_write_kb":16620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3447375,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[{"shards":"5,6","objects":63}],"blocked_by":[],"up_primary":2,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.558839+0000","last_change":"2024-09-13T07:02:44.091196+0000","last_active":"2024-09-13T07:02:45.558839+0000","last_peered":"2024-09-13T07:02:45.558839+0000","last_clean":"2024-09-13T07:02:45.558839+0000","last_became_active":"2024-09-13T07:02:44.090715+0000","last_became_peered":"2024-09-13T07:02:44.090715+0000","last_unstale":"2024-09-13T07:02:45.558839+0000","last_undegraded":"2024-09-13T07:02:45.558839+0000","last_fullsized":"2024-09-13T07:02:45.558839+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:08.303433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"100'188","reported_seq":3897,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.616441+0000","last_change":"2024-09-13T07:02:16.282550+0000","last_active":"2024-09-13T07:02:45.616441+0000","last_peered":"2024-09-13T07:02:45.616441+0000","last_clean":"2024-09-13T07:02:45.616441+0000","last_became_active":"2024-09-13T07:02:16.282010+0000","last_became_peered":"2024-09-13T07:02:16.282010+0000","last_unstale":"2024-09-13T07:02:45.616441+0000","last_undegraded":"2024-09-13T07:02:45.616441+0000","last_fullsized":"2024-09-13T07:02:45.616441+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:07:01.767243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044811299999999998,"stat_sum":{"num_bytes":34925940,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":4162,"num_read_kb":21803,"num_write":201,"num_write_kb":2826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":240544,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":104,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.087172+0000","last_change":"2024-09-13T07:02:20.307961+0000","last_active":"2024-09-13T07:02:44.087172+0000","last_peered":"2024-09-13T07:02:44.087172+0000","last_clean":"2024-09-13T07:02:44.087172+0000","last_became_active":"2024-09-13T07:02:20.307607+0000","last_became_peered":"2024-09-13T07:02:20.307607+0000","last_unstale":"2024-09-13T07:02:44.087172+0000","last_undegraded":"2024-09-13T07:02:44.087172+0000","last_fullsized":"2024-09-13T07:02:44.087172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:05.765371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"97'280","reported_seq":2779,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.085416+0000","last_change":"2024-09-13T07:02:26.368592+0000","last_active":"2024-09-13T07:02:44.085416+0000","last_peered":"2024-09-13T07:02:44.085416+0000","last_clean":"2024-09-13T07:02:44.085416+0000","last_became_active":"2024-09-13T07:02:26.367948+0000","last_became_peered":"2024-09-13T07:02:26.367948+0000","last_unstale":"2024-09-13T07:02:44.085416+0000","last_undegraded":"2024-09-13T07:02:44.085416+0000","last_fullsized":"2024-09-13T07:02:44.085416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:37:10.553378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064294199999999997,"stat_sum":{"num_bytes":51716012,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":18,"num_read":4587,"num_read_kb":17097,"num_write":293,"num_write_kb":14284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":121,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.095700+0000","last_change":"2024-09-13T07:02:44.094792+0000","last_active":"2024-09-13T07:02:44.095700+0000","last_peered":"2024-09-13T07:02:44.095700+0000","last_clean":"2024-09-13T07:02:44.095700+0000","last_became_active":"2024-09-13T07:02:44.091356+0000","last_became_peered":"2024-09-13T07:02:44.091356+0000","last_unstale":"2024-09-13T07:02:44.095700+0000","last_undegraded":"2024-09-13T07:02:44.095700+0000","last_fullsized":"2024-09-13T07:02:44.095700+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:58:57.641823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":265,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.088524+0000","last_change":"2024-09-13T07:01:50.080564+0000","last_active":"2024-09-13T07:02:44.088524+0000","last_peered":"2024-09-13T07:02:44.088524+0000","last_clean":"2024-09-13T07:02:44.088524+0000","last_became_active":"2024-09-13T07:01:50.080072+0000","last_became_peered":"2024-09-13T07:01:50.080072+0000","last_unstale":"2024-09-13T07:02:44.088524+0000","last_undegraded":"2024-09-13T07:02:44.088524+0000","last_fullsized":"2024-09-13T07:02:44.088524+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:14:58.662626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"100'395","reported_seq":1916,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.623225+0000","last_change":"2024-09-13T07:02:23.331997+0000","last_active":"2024-09-13T07:02:45.623225+0000","last_peered":"2024-09-13T07:02:45.623225+0000","last_clean":"2024-09-13T07:02:45.623225+0000","last_became_active":"2024-09-13T07:02:23.331206+0000","last_became_peered":"2024-09-13T07:02:23.331206+0000","last_unstale":"2024-09-13T07:02:45.623225+0000","last_undegraded":"2024-09-13T07:02:45.623225+0000","last_fullsized":"2024-09-13T07:02:45.623225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:30:37.551347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001264996,"stat_sum":{"num_bytes":22270602,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":2024,"num_read_kb":11876,"num_write":553,"num_write_kb":3599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":861185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"97'210","reported_seq":7748,"reported_epoch":100,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-13T07:02:45.560858+0000","last_change":"2024-09-13T07:02:45.561287+0000","last_active":"2024-09-13T07:02:45.560858+0000","last_peered":"2024-09-13T07:02:45.560858+0000","last_clean":"2024-09-13T07:02:43.087370+0000","last_became_active":"2024-09-13T07:02:45.560175+0000","last_became_peered":"2024-09-13T07:02:45.560175+0000","last_unstale":"2024-09-13T07:02:45.560858+0000","last_undegraded":"2024-09-13T07:02:45.092775+0000","last_fullsized":"2024-09-13T07:02:45.090252+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:19:06.103516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022509729999999999,"stat_sum":{"num_bytes":30720023,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":21,"num_read":7939,"num_read_kb":20468,"num_write":270,"num_write_kb":16965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":26238,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4],"avail_no_missing":["4","7"],"object_location_counts":[{"shards":"4,7","objects":51}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"100'486","reported_seq":2683,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.611418+0000","last_change":"2024-09-13T07:01:22.025365+0000","last_active":"2024-09-13T07:02:45.611418+0000","last_peered":"2024-09-13T07:02:45.611418+0000","last_clean":"2024-09-13T07:02:45.611418+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:02:45.611418+0000","last_undegraded":"2024-09-13T07:02:45.611418+0000","last_fullsized":"2024-09-13T07:02:45.611418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:11.537644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064478199999999999,"stat_sum":{"num_bytes":42920730,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":23,"num_read":3402,"num_read_kb":21413,"num_write":683,"num_write_kb":17701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"97'264","reported_seq":4927,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.088861+0000","last_change":"2024-09-13T07:02:26.369574+0000","last_active":"2024-09-13T07:02:44.088861+0000","last_peered":"2024-09-13T07:02:44.088861+0000","last_clean":"2024-09-13T07:02:44.088861+0000","last_became_active":"2024-09-13T07:02:26.368387+0000","last_became_peered":"2024-09-13T07:02:26.368387+0000","last_unstale":"2024-09-13T07:02:44.088861+0000","last_undegraded":"2024-09-13T07:02:44.088861+0000","last_fullsized":"2024-09-13T07:02:44.088861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:23:15.792514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001533932,"stat_sum":{"num_bytes":45675461,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":6050,"num_read_kb":17349,"num_write":297,"num_write_kb":7392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1474279,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"100'394","reported_seq":11226,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.623425+0000","last_change":"2024-09-13T07:01:21.292633+0000","last_active":"2024-09-13T07:02:45.623425+0000","last_peered":"2024-09-13T07:02:45.623425+0000","last_clean":"2024-09-13T07:02:45.623425+0000","last_became_active":"2024-09-13T07:01:19.279813+0000","last_became_peered":"2024-09-13T07:01:19.279813+0000","last_unstale":"2024-09-13T07:02:45.623425+0000","last_undegraded":"2024-09-13T07:02:45.623425+0000","last_fullsized":"2024-09-13T07:02:45.623425+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:39.697691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072600099999999999,"stat_sum":{"num_bytes":70224611,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":15,"num_read":12982,"num_read_kb":43549,"num_write":459,"num_write_kb":20802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"100'251","reported_seq":3352,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.623645+0000","last_change":"2024-09-13T07:01:21.313310+0000","last_active":"2024-09-13T07:02:45.623645+0000","last_peered":"2024-09-13T07:02:45.623645+0000","last_clean":"2024-09-13T07:02:45.623645+0000","last_became_active":"2024-09-13T07:01:19.282520+0000","last_became_peered":"2024-09-13T07:01:19.282520+0000","last_unstale":"2024-09-13T07:02:45.623645+0000","last_undegraded":"2024-09-13T07:02:45.623645+0000","last_fullsized":"2024-09-13T07:02:45.623645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:41:44.801487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018248170000000001,"stat_sum":{"num_bytes":43034530,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":21,"num_read":3407,"num_read_kb":9442,"num_write":243,"num_write_kb":10221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"100'364","reported_seq":1955,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.652394+0000","last_change":"2024-09-13T07:01:21.313674+0000","last_active":"2024-09-13T07:02:45.652394+0000","last_peered":"2024-09-13T07:02:45.652394+0000","last_clean":"2024-09-13T07:02:45.652394+0000","last_became_active":"2024-09-13T07:01:19.281215+0000","last_became_peered":"2024-09-13T07:01:19.281215+0000","last_unstale":"2024-09-13T07:02:45.652394+0000","last_undegraded":"2024-09-13T07:02:45.652394+0000","last_fullsized":"2024-09-13T07:02:45.652394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:22.161596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019878309999999998,"stat_sum":{"num_bytes":33916175,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":2625,"num_read_kb":15600,"num_write":580,"num_write_kb":9064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"97'624","reported_seq":2089,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T07:02:44.087923+0000","last_change":"2024-09-13T07:01:21.314048+0000","last_active":"2024-09-13T07:02:44.087923+0000","last_peered":"2024-09-13T07:02:44.087923+0000","last_clean":"2024-09-13T07:02:44.087923+0000","last_became_active":"2024-09-13T07:01:19.281169+0000","last_became_peered":"2024-09-13T07:01:19.281169+0000","last_unstale":"2024-09-13T07:02:44.087923+0000","last_undegraded":"2024-09-13T07:02:44.087923+0000","last_fullsized":"2024-09-13T07:02:44.087923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:49:46.525500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023438180000000001,"stat_sum":{"num_bytes":41704972,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":22,"num_read":3898,"num_read_kb":17699,"num_write":1040,"num_write_kb":24228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"100'200","reported_seq":1530,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.636637+0000","last_change":"2024-09-13T07:02:12.324979+0000","last_active":"2024-09-13T07:02:45.636637+0000","last_peered":"2024-09-13T07:02:45.636637+0000","last_clean":"2024-09-13T07:02:45.636637+0000","last_became_active":"2024-09-13T07:02:12.247568+0000","last_became_peered":"2024-09-13T07:02:12.247568+0000","last_unstale":"2024-09-13T07:02:45.636637+0000","last_undegraded":"2024-09-13T07:02:45.636637+0000","last_fullsized":"2024-09-13T07:02:45.636637+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:04:50.472141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061928799999999996,"stat_sum":{"num_bytes":29443336,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":1228,"num_read_kb":5939,"num_write":216,"num_write_kb":5613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":4419876,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"100'284","reported_seq":6587,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.639939+0000","last_change":"2024-09-13T07:01:21.313825+0000","last_active":"2024-09-13T07:02:45.639939+0000","last_peered":"2024-09-13T07:02:45.639939+0000","last_clean":"2024-09-13T07:02:45.639939+0000","last_became_active":"2024-09-13T07:01:19.279714+0000","last_became_peered":"2024-09-13T07:01:19.279714+0000","last_unstale":"2024-09-13T07:02:45.639939+0000","last_undegraded":"2024-09-13T07:02:45.639939+0000","last_fullsized":"2024-09-13T07:02:45.639939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:06:09.717433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002154132,"stat_sum":{"num_bytes":55491207,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":7404,"num_read_kb":15271,"num_write":323,"num_write_kb":12698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"100'507","reported_seq":6959,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.600007+0000","last_change":"2024-09-13T07:01:50.078398+0000","last_active":"2024-09-13T07:02:45.600007+0000","last_peered":"2024-09-13T07:02:45.600007+0000","last_clean":"2024-09-13T07:02:45.600007+0000","last_became_active":"2024-09-13T07:01:50.078015+0000","last_became_peered":"2024-09-13T07:01:50.078015+0000","last_unstale":"2024-09-13T07:02:45.600007+0000","last_undegraded":"2024-09-13T07:02:45.600007+0000","last_fullsized":"2024-09-13T07:02:45.600007+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:52.203434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060210699999999995,"stat_sum":{"num_bytes":56707452,"num_objects":46,"num_object_clones":14,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":17,"num_read":7939,"num_read_kb":30377,"num_write":739,"num_write_kb":8867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":60382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"100'294","reported_seq":2416,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T07:02:45.087436+0000","last_change":"2024-09-13T07:01:21.313226+0000","last_active":"2024-09-13T07:02:45.087436+0000","last_peered":"2024-09-13T07:02:45.087436+0000","last_clean":"2024-09-13T07:02:45.087436+0000","last_became_active":"2024-09-13T07:01:19.279419+0000","last_became_peered":"2024-09-13T07:01:19.279419+0000","last_unstale":"2024-09-13T07:02:45.087436+0000","last_undegraded":"2024-09-13T07:02:45.087436+0000","last_fullsized":"2024-09-13T07:02:45.087436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:46:29.817757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001803165,"stat_sum":{"num_bytes":36987576,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":3050,"num_read_kb":21402,"num_write":343,"num_write_kb":17440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1334304785,"num_objects":1533,"num_object_clones":350,"num_object_copies":3066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":105,"num_objects_unfound":0,"num_objects_dirty":1533,"num_whiteouts":649,"num_read":162025,"num_read_kb":559362,"num_write":12879,"num_write_kb":304709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":734,"num_bytes_recovered":30894002,"num_keys_recovered":153,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":298405888,"data_stored":338842763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9974,"ondisk_log_size":9974,"up":64,"acting":59,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":120068,"kb_used_data":70900,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94251772,"statfs":{"total":96636764160,"available":96513814528,"internally_reserved":0,"allocated":72601600,"data_stored":82006382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":1.081}]},{"osd":1,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.878},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.84599999999999997}]},{"osd":2,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":1.149},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":1.05}]},{"osd":3,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.119}]},{"osd":4,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.51},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.4890000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.296}]},{"osd":6,"last update":"Fri Sep 13 07:02:03 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.45},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.4179999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69784,"kb_used_data":34564,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94302056,"statfs":{"total":96636764160,"available":96565305344,"internally_reserved":0,"allocated":35393536,"data_stored":40598183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.75},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.76800000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Sep 13 07:02:04 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.79100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84556,"kb_used_data":45240,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94287284,"statfs":{"total":96636764160,"available":96550178816,"internally_reserved":0,"allocated":46325760,"data_stored":50095808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.91800000000000004}]},{"osd":1,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.1449999999999996,"5min":6.1449999999999996,"15min":6.1449999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.032}]},{"osd":3,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.105},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":6.2030000000000003,"5min":6.2030000000000003,"15min":6.2030000000000003},"last":0.88}]},{"osd":4,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":1.208}]},{"osd":6,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":6.1100000000000003,"5min":6.1100000000000003,"15min":6.1100000000000003},"last":1.054},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":1.2450000000000001}]},{"osd":7,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.276},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":6.1749999999999998,"5min":6.1749999999999998,"15min":6.1749999999999998},"last":1.127}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":121108,"kb_used_data":76040,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94250732,"statfs":{"total":96636764160,"available":96512749568,"internally_reserved":0,"allocated":77864960,"data_stored":87581696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":1.022}]},{"osd":1,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.91800000000000004}]},{"osd":2,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.86799999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.95499999999999996}]},{"osd":5,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.0940000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.056},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":1.228}]},{"osd":7,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":1.165},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.1859999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77932,"kb_used_data":50916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94293908,"statfs":{"total":96636764160,"available":96556961792,"internally_reserved":0,"allocated":52137984,"data_stored":57921056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.86199999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.82499999999999996}]},{"osd":4,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.512}]},{"osd":5,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.3300000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.4319999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:02:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.28},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.47}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1175552,"data_stored":541491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.39600000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:02:01 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":6.867,"5min":6.867,"15min":6.867},"last":0.747},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Sep 13 07:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.84899999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Sep 13 07:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Sep 13 07:02:01 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.167},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.2250000000000001}]},{"osd":7,"last update":"Fri Sep 13 07:02:01 2024","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.70399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36368,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335472,"statfs":{"total":96636764160,"available":96599523328,"internally_reserved":0,"allocated":1175552,"data_stored":541491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69199999999999995}]},{"osd":2,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.499}]},{"osd":3,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":1.1240000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.1910000000000001},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.875}]},{"osd":5,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.1399999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.93500000000000005}]},{"osd":7,"last update":"Fri Sep 13 07:02:00 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.044},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.2390000000000001}]}]},{"osd":0,"up_from":0,"seq":55834574872,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9293824,"data_stored":10365051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9109504,"data_stored":11024733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2637824,"data_stored":3015948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":50950144,"data_stored":57374259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":76709888,"data_stored":87065088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45195264,"data_stored":49602562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":33603584,"data_stored":39456197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":70905856,"data_stored":80938925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:02:49.315 INFO:tasks.thrashosds.thrasher:Setting 3.d to [6, 2] 2024-09-13T07:02:49.315 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.d', '6', '2'] 2024-09-13T07:02:49.315 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.d 6 2 2024-09-13T07:02:49.324 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:49.324 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:49.334 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:49.377 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:49.376+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:49.438 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:49.436+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:49.450 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:49.450 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:49.460 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:49.524 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:02:49.539 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:49.538+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:49.573 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:49.573 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:49.583 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:49.618 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:02:49.639 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:49.636+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:49.647 INFO:teuthology.orchestra.run.smithi118.stdout:leaving image image_client.0-clone33 intact 2024-09-13T07:02:49.687 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:49.687 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:49.697 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:49.740 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:49.738+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:49.780 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-13T07:02:49.785 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:49.797 INFO:teuthology.orchestra.run.smithi118.stdout:3504 writesame 0x4fdae87 thru 0x4fea5ae data_size 0x16(0xf728 bytes) 2024-09-13T07:02:49.807 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:49.807 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:49.807 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:49.817 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:49.828 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:49.831 INFO:teuthology.orchestra.run.smithi118.stdout:3506 write 0xd565bc1 thru 0xd5734e0 (0xd920 bytes) 2024-09-13T07:02:49.841 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:49.840+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:49.847 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:49.848 INFO:teuthology.orchestra.run.smithi118.stdout:3507 writesame 0xe32bd36 thru 0xe330169 data_size 0x5a(0x4434 bytes) 2024-09-13T07:02:49.848 INFO:teuthology.orchestra.run.smithi118.stdout:3508 write 0xc6e8754 thru 0xc6f3d95 (0xb642 bytes) 2024-09-13T07:02:49.854 INFO:teuthology.orchestra.run.smithi118.stdout:3509 read 0x7a4e5e0 thru 0x7a5d6d4 (0xf0f5 bytes) 2024-09-13T07:02:49.854 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:49.856 INFO:teuthology.orchestra.run.smithi118.stdout:3512 trunc from 0xe33016a to 0xd642af0 2024-09-13T07:02:49.868 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:49.870 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:49.870 INFO:teuthology.orchestra.run.smithi118.stdout:3514 read 0x80be260 thru 0x80cdb24 (0xf8c5 bytes) 2024-09-13T07:02:49.870 INFO:teuthology.orchestra.run.smithi118.stdout:3521 punch from 0x78fd480 to 0x7903427, (0x5fa7 bytes) 2024-09-13T07:02:49.871 INFO:teuthology.orchestra.run.smithi118.stdout:3522 flatten 2024-09-13T07:02:49.900 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:49.933 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:49.942 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:49.938+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:49.952 INFO:teuthology.orchestra.run.smithi118.stdout:3523 write 0xe5a18d4 thru 0xe5b057f (0xecac bytes) 2024-09-13T07:02:49.952 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:49.954 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:49.954 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:49.954 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:49.954 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:49.954 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:49.954 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:49.954 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:49.954 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:49.957 INFO:teuthology.orchestra.run.smithi118.stdout:3525 write 0x387bd23 thru 0x38855fd (0x98db bytes) 2024-09-13T07:02:49.964 INFO:teuthology.orchestra.run.smithi118.stdout:3526 write 0xae249b8 thru 0xae28a00 (0x4049 bytes) 2024-09-13T07:02:49.983 INFO:teuthology.orchestra.run.smithi118.stdout:3527 trunc from 0xe5b0580 to 0x746570b 2024-09-13T07:02:49.992 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:49.992 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:49.992 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:50.003 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:50.004 INFO:teuthology.orchestra.run.smithi118.stdout:3529 writesame 0x63a4a35 thru 0x63b3cdc data_size 0x14(0xf2a8 bytes) 2024-09-13T07:02:50.038 INFO:teuthology.orchestra.run.smithi118.stdout:3530 writesame 0xcf93e1e thru 0xcf98c78 data_size 0x1(0x4e5b bytes) 2024-09-13T07:02:50.038 INFO:teuthology.orchestra.run.smithi118.stdout:3532 punch from 0x6f38df5 to 0x6f4145b, (0x8666 bytes) 2024-09-13T07:02:50.038 INFO:teuthology.orchestra.run.smithi118.stdout:3533 read 0x8826bc6 thru 0x8830c86 (0xa0c1 bytes) 2024-09-13T07:02:50.038 INFO:teuthology.orchestra.run.smithi118.stdout:3536 punch from 0xa93fa63 to 0xa93fee3, (0x480 bytes) 2024-09-13T07:02:50.038 INFO:teuthology.orchestra.run.smithi118.stdout:3537 punch from 0x1506748 to 0x150bc58, (0x5510 bytes) 2024-09-13T07:02:50.038 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:50.038 INFO:teuthology.orchestra.run.smithi118.stdout:3540 read 0x6fbd47e thru 0x6fcbcb5 (0xe838 bytes) 2024-09-13T07:02:50.038 INFO:teuthology.orchestra.run.smithi118.stdout:3541 write 0x77a283b thru 0x77ae731 (0xbef7 bytes) 2024-09-13T07:02:50.042 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:50.040+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:50.059 INFO:teuthology.orchestra.run.smithi118.stdout:3542 trunc from 0xcf98c79 to 0x2bb00db 2024-09-13T07:02:50.105 INFO:teuthology.orchestra.run.smithi118.stdout:3543 write 0xaeb1a9a thru 0xaeb299b (0xf02 bytes) 2024-09-13T07:02:50.111 INFO:teuthology.orchestra.run.smithi118.stdout:3544 write 0xbff207 thru 0xc04b60 (0x595a bytes) 2024-09-13T07:02:50.114 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:50.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:02:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.993713786999997, 2024-09-13T07:02:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:02:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.115 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:02:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:02:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:02:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:02:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:02:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:02:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:02:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:02:50.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:02:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:02:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:02:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:02:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:02:50.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:02:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:02:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:02:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:02:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:02:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:02:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:02:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:02:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.123 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129532 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:02:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4973109, 2024-09-13T07:02:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89889205500000002, 2024-09-13T07:02:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129532 2024-09-13T07:02:50.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:02:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:02:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:02:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614994+0000", 2024-09-13T07:02:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.044e-06 2024-09-13T07:02:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615000+0000", 2024-09-13T07:02:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9229999999999996e-06 2024-09-13T07:02:50.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615005+0000", 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615060+0000", 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4561000000000002e-05 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615069+0000", 2024-09-13T07:02:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1540000000000008e-06 2024-09-13T07:02:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513869+0000", 2024-09-13T07:02:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89879977700000002 2024-09-13T07:02:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513883+0000", 2024-09-13T07:02:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4361e-05 2024-09-13T07:02:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.128 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129533 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:02:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.497215567, 2024-09-13T07:02:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.899270177, 2024-09-13T07:02:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129533 2024-09-13T07:02:50.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:02:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615090+0000", 2024-09-13T07:02:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.856e-06 2024-09-13T07:02:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615096+0000", 2024-09-13T07:02:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.703e-06 2024-09-13T07:02:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615102+0000", 2024-09-13T07:02:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.378e-06 2024-09-13T07:02:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615140+0000", 2024-09-13T07:02:50.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7966999999999997e-05 2024-09-13T07:02:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615147+0000", 2024-09-13T07:02:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8800000000000002e-06 2024-09-13T07:02:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514342+0000", 2024-09-13T07:02:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919565300000004 2024-09-13T07:02:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514357+0000", 2024-09-13T07:02:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4355000000000001e-05 2024-09-13T07:02:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129534 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.497142669, 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89923080600000005, 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129534 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615162+0000", 2024-09-13T07:02:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3569999999999998e-06 2024-09-13T07:02:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615169+0000", 2024-09-13T07:02:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.455e-06 2024-09-13T07:02:50.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615175+0000", 2024-09-13T07:02:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3679999999999998e-06 2024-09-13T07:02:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615212+0000", 2024-09-13T07:02:50.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6733999999999998e-05 2024-09-13T07:02:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615218+0000", 2024-09-13T07:02:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7569999999999998e-06 2024-09-13T07:02:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514383+0000", 2024-09-13T07:02:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916408000000003 2024-09-13T07:02:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514390+0000", 2024-09-13T07:02:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5429999999999996e-06 2024-09-13T07:02:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.143 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129535 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:02:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4970692529999998, 2024-09-13T07:02:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919205499999999, 2024-09-13T07:02:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129535 2024-09-13T07:02:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.145 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:02:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:02:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615232+0000", 2024-09-13T07:02:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615236+0000", 2024-09-13T07:02:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:02:50.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615244+0000", 2024-09-13T07:02:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7369999999999995e-06 2024-09-13T07:02:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615250+0000", 2024-09-13T07:02:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2270000000000001e-06 2024-09-13T07:02:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:02:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4627000000000002e-05 2024-09-13T07:02:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:02:50.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5089999999999993e-06 2024-09-13T07:02:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514412+0000", 2024-09-13T07:02:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89907890599999996 2024-09-13T07:02:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514425+0000", 2024-09-13T07:02:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3138e-05 2024-09-13T07:02:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129536 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:02:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4970357760000002, 2024-09-13T07:02:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89918503500000002, 2024-09-13T07:02:50.154 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129536 2024-09-13T07:02:50.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:02:50.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:02:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:02:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615268+0000", 2024-09-13T07:02:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8370000000000001e-06 2024-09-13T07:02:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615288+0000", 2024-09-13T07:02:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9945999999999998e-05 2024-09-13T07:02:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615294+0000", 2024-09-13T07:02:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3340000000000001e-06 2024-09-13T07:02:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615374+0000", 2024-09-13T07:02:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9988999999999996e-05 2024-09-13T07:02:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615381+0000", 2024-09-13T07:02:50.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:02:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514444+0000", 2024-09-13T07:02:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89906347499999995 2024-09-13T07:02:50.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514451+0000", 2024-09-13T07:02:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7630000000000003e-06 2024-09-13T07:02:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129537 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.163 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:02:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4969872240000002, 2024-09-13T07:02:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916833399999996, 2024-09-13T07:02:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129537 2024-09-13T07:02:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:02:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:02:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615314+0000", 2024-09-13T07:02:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615317+0000", 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1410000000000001e-06 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3589999999999996e-06 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615329+0000", 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-13T07:02:50.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615408+0000", 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8678999999999997e-05 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615414+0000", 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514471+0000", 2024-09-13T07:02:50.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905719200000001 2024-09-13T07:02:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514483+0000", 2024-09-13T07:02:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1691999999999999e-05 2024-09-13T07:02:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129538 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.168 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4969686850000001, 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89917603999999995, 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129538 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:02:50.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615335+0000", 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4090000000000001e-06 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615346+0000", 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0866000000000001e-05 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615424+0000", 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7628999999999998e-05 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615459+0000", 2024-09-13T07:02:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5046999999999998e-05 2024-09-13T07:02:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615466+0000", 2024-09-13T07:02:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7610000000000002e-06 2024-09-13T07:02:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514503+0000", 2024-09-13T07:02:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89903750299999996 2024-09-13T07:02:50.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514509+0000", 2024-09-13T07:02:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2140000000000001e-06 2024-09-13T07:02:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129539 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:02:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.496823934, 2024-09-13T07:02:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905735799999997, 2024-09-13T07:02:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.174 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129539 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615477+0000", 2024-09-13T07:02:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615481+0000", 2024-09-13T07:02:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6490000000000001e-06 2024-09-13T07:02:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615488+0000", 2024-09-13T07:02:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4359999999999999e-06 2024-09-13T07:02:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615493+0000", 2024-09-13T07:02:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3120000000000001e-06 2024-09-13T07:02:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615529+0000", 2024-09-13T07:02:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6468999999999999e-05 2024-09-13T07:02:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615536+0000", 2024-09-13T07:02:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9789999999999996e-06 2024-09-13T07:02:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514529+0000", 2024-09-13T07:02:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89899276400000006 2024-09-13T07:02:50.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514535+0000", 2024-09-13T07:02:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2920000000000001e-06 2024-09-13T07:02:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129540 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:02:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4967529399999999, 2024-09-13T07:02:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89901438700000003, 2024-09-13T07:02:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129540 2024-09-13T07:02:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:02:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:02:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615548+0000", 2024-09-13T07:02:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615552+0000", 2024-09-13T07:02:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8349999999999997e-06 2024-09-13T07:02:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615559+0000", 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615564+0000", 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2360000000000004e-06 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615600+0000", 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6637000000000002e-05 2024-09-13T07:02:50.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615607+0000", 2024-09-13T07:02:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.669e-06 2024-09-13T07:02:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514557+0000", 2024-09-13T07:02:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89894939399999996 2024-09-13T07:02:50.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514563+0000", 2024-09-13T07:02:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8560000000000001e-06 2024-09-13T07:02:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129541 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:02:50.185 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4966803280000001, 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89896824600000003, 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129541 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:02:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615621+0000", 2024-09-13T07:02:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615625+0000", 2024-09-13T07:02:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-13T07:02:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615631+0000", 2024-09-13T07:02:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3890000000000002e-06 2024-09-13T07:02:50.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615636+0000", 2024-09-13T07:02:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3489999999999999e-06 2024-09-13T07:02:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615671+0000", 2024-09-13T07:02:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4570000000000003e-05 2024-09-13T07:02:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615678+0000", 2024-09-13T07:02:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8510000000000001e-06 2024-09-13T07:02:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514583+0000", 2024-09-13T07:02:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89890530599999996 2024-09-13T07:02:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514590+0000", 2024-09-13T07:02:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7469999999999999e-06 2024-09-13T07:02:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129542 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:02:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4966068379999999, 2024-09-13T07:02:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89892092400000001, 2024-09-13T07:02:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.191 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129542 2024-09-13T07:02:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:02:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:02:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:02:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615698+0000", 2024-09-13T07:02:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.664e-06 2024-09-13T07:02:50.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615705+0000", 2024-09-13T07:02:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:02:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615711+0000", 2024-09-13T07:02:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6470000000000003e-06 2024-09-13T07:02:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615752+0000", 2024-09-13T07:02:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0389999999999998e-05 2024-09-13T07:02:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615759+0000", 2024-09-13T07:02:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9230000000000002e-06 2024-09-13T07:02:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514609+0000", 2024-09-13T07:02:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89884984899999998 2024-09-13T07:02:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514616+0000", 2024-09-13T07:02:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7299999999999999e-06 2024-09-13T07:02:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129543 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:02:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4965279649999998, 2024-09-13T07:02:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89886872600000001, 2024-09-13T07:02:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129543 2024-09-13T07:02:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615773+0000", 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:50.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615777+0000", 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1860000000000002e-06 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615785+0000", 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2710000000000003e-06 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615790+0000", 2024-09-13T07:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3619999999999997e-06 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615828+0000", 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8059999999999998e-05 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615836+0000", 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8809999999999996e-06 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514636+0000", 2024-09-13T07:02:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89880009599999999 2024-09-13T07:02:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514643+0000", 2024-09-13T07:02:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:02:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.201 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129544 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:02:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4964524429999999, 2024-09-13T07:02:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89881970799999999, 2024-09-13T07:02:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129544 2024-09-13T07:02:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615853+0000", 2024-09-13T07:02:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8600000000000003e-06 2024-09-13T07:02:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615860+0000", 2024-09-13T07:02:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1709999999999996e-06 2024-09-13T07:02:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615867+0000", 2024-09-13T07:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7020000000000004e-06 2024-09-13T07:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615905+0000", 2024-09-13T07:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8265e-05 2024-09-13T07:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615913+0000", 2024-09-13T07:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-13T07:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514663+0000", 2024-09-13T07:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89874952500000005 2024-09-13T07:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514669+0000", 2024-09-13T07:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129545 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.49637427, 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89876792599999999, 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129545 2024-09-13T07:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615927+0000", 2024-09-13T07:02:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615931+0000", 2024-09-13T07:02:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8010000000000003e-06 2024-09-13T07:02:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615939+0000", 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6310000000000003e-06 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615947+0000", 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7090000000000002e-06 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615985+0000", 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8389000000000001e-05 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615993+0000", 2024-09-13T07:02:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2209999999999997e-06 2024-09-13T07:02:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514689+0000", 2024-09-13T07:02:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89869535199999995 2024-09-13T07:02:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514696+0000", 2024-09-13T07:02:50.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8179999999999998e-06 2024-09-13T07:02:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129546 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:02:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4962941819999998, 2024-09-13T07:02:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89872448199999999, 2024-09-13T07:02:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.212 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129546 2024-09-13T07:02:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616007+0000", 2024-09-13T07:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616011+0000", 2024-09-13T07:02:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1659999999999996e-06 2024-09-13T07:02:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616018+0000", 2024-09-13T07:02:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0829999999999998e-06 2024-09-13T07:02:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616024+0000", 2024-09-13T07:02:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8170000000000004e-06 2024-09-13T07:02:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616063+0000", 2024-09-13T07:02:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8708000000000003e-05 2024-09-13T07:02:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616070+0000", 2024-09-13T07:02:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6299999999999998e-06 2024-09-13T07:02:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514721+0000", 2024-09-13T07:02:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89865028800000002 2024-09-13T07:02:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514732+0000", 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1452e-05 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129547 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4962188539999999, 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89868529799999997, 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.218 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129547 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:02:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616086+0000", 2024-09-13T07:02:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.394e-06 2024-09-13T07:02:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616093+0000", 2024-09-13T07:02:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1130000000000003e-06 2024-09-13T07:02:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616100+0000", 2024-09-13T07:02:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1520000000000003e-06 2024-09-13T07:02:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616138+0000", 2024-09-13T07:02:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8084000000000003e-05 2024-09-13T07:02:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616146+0000", 2024-09-13T07:02:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3010000000000001e-06 2024-09-13T07:02:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:02:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89861259299999996 2024-09-13T07:02:50.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514768+0000", 2024-09-13T07:02:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0175e-05 2024-09-13T07:02:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.223 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129548 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:02:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4961347950000001, 2024-09-13T07:02:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89862847899999998, 2024-09-13T07:02:50.224 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129548 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:02:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616166+0000", 2024-09-13T07:02:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616170+0000", 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8619999999999996e-06 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616177+0000", 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0450000000000003e-06 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616183+0000", 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.139e-06 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616221+0000", 2024-09-13T07:02:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7320000000000002e-05 2024-09-13T07:02:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616228+0000", 2024-09-13T07:02:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2980000000000003e-06 2024-09-13T07:02:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514789+0000", 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89856044599999996 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514796+0000", 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0809999999999996e-06 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129549 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:02:50.229 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.496061332, 2024-09-13T07:02:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89859381900000002, 2024-09-13T07:02:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129549 2024-09-13T07:02:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:02:50.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616240+0000", 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616244+0000", 2024-09-13T07:02:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4149999999999999e-06 2024-09-13T07:02:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616251+0000", 2024-09-13T07:02:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-13T07:02:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616256+0000", 2024-09-13T07:02:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3299999999999998e-06 2024-09-13T07:02:50.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616302+0000", 2024-09-13T07:02:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5646999999999999e-05 2024-09-13T07:02:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616309+0000", 2024-09-13T07:02:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5530000000000004e-06 2024-09-13T07:02:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514825+0000", 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89851528400000003 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514834+0000", 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8349999999999999e-06 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129550 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:02:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.495976953, 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89854296199999995, 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129550 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:02:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616324+0000", 2024-09-13T07:02:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:50.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616328+0000", 2024-09-13T07:02:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7479999999999999e-06 2024-09-13T07:02:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616335+0000", 2024-09-13T07:02:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9280000000000004e-06 2024-09-13T07:02:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616342+0000", 2024-09-13T07:02:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2690000000000002e-06 2024-09-13T07:02:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616380+0000", 2024-09-13T07:02:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:02:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616387+0000", 2024-09-13T07:02:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.075e-06 2024-09-13T07:02:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514861+0000", 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89847450799999995 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514868+0000", 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6050000000000003e-06 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.239 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:50.241 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:50.241 INFO:teuthology.orchestra.run.smithi118.stdout:3546 punch from 0x537f863 to 0x538aeb5, (0xb652 bytes) 2024-09-13T07:02:50.241 INFO:teuthology.orchestra.run.smithi118.stdout:3547 writesame 0x8d8f4a4 thru 0x8d951b1 data_size 0x2b(0x5d0e bytes) 2024-09-13T07:02:50.241 INFO:teuthology.orchestra.run.smithi118.stdout:3548 write 0x1536f7 thru 0x159363 (0x5c6d bytes) 2024-09-13T07:02:50.242 INFO:teuthology.orchestra.run.smithi118.stdout:3549 clone 35 order 23 su 4194304 sc 3 2024-09-13T07:02:50.242 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:50.240+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:50.242 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:50.342 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:50.340+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:50.356 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:50.357 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:50.366 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:50.443 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:50.440+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:50.473 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:50.474 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:50.483 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:50.535 INFO:teuthology.orchestra.run.smithi118.stderr:set 3.d pg_upmap mapping to [6,2] 2024-09-13T07:02:50.537 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:02:50.545 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:50.544+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:50.596 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.535580132000007, 2024-09-13T07:02:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:02:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:02:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:02:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:02:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:02:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:02:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:02:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:02:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:02:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:02:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:02:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:02:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:02:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:02:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:02:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:02:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:02:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93979 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977820819000002, 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042253801000000001, 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93979 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616683+0000", 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2820000000000001e-06 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616691+0000", 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0779999999999996e-06 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616698+0000", 2024-09-13T07:02:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658799+0000", 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042100763999999999 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658869+0000", 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0065000000000001e-05 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658934+0000", 2024-09-13T07:02:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5071000000000001e-05 2024-09-13T07:02:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:02:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977796254000001, 2024-09-13T07:02:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:02:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:02:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:02:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:02:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:02:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:02:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:02:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:02:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977773287000002, 2024-09-13T07:02:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:02:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:02:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:02:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:02:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:02:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:02:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:02:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:02:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:02:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:02:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:02:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:02:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:02:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:02:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:02:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977756109000001, 2024-09-13T07:02:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:02:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:02:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:02:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:02:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:02:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:02:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:02:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977742244000002, 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:02:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:02:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:02:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:02:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:02:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:02:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:02:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:02:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:02:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:02:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:02:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:02:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:02:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:02:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:02:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:02:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977729099000001, 2024-09-13T07:02:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:02:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:02:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:02:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:02:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:02:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:02:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:02:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:02:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:02:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:02:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:02:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:02:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:02:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:02:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:02:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:02:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:02:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977715891999999, 2024-09-13T07:02:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:02:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:02:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:02:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:02:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:02:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:02:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:02:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:02:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:02:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:02:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:02:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:02:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:02:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:02:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977701825, 2024-09-13T07:02:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:02:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:02:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977681811, 2024-09-13T07:02:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:02:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:02:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:02:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977662061, 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:02:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977637255000001, 2024-09-13T07:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:02:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:02:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:02:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:02:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:02:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977612990000001, 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:02:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:02:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:02:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:02:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:02:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:02:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:02:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:02:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:02:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:02:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:02:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:02:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977571257000001, 2024-09-13T07:02:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:02:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:02:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:02:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:02:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:02:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:02:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:02:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:02:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:02:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.977534968000001, 2024-09-13T07:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:02:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:02:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:02:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:02:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:02:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5344316360000008, 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:02:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:02:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5344147540000002, 2024-09-13T07:02:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:02:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:02:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:02:50.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:02:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:02:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:02:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:02:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:02:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:02:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:02:50.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126633 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2408448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061112+0000", 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.533388789, 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042234086999999997, 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.676 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126633 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061112+0000", 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061112+0000", 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061112+0000", 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061115+0000", 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6569999999999999e-06 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061125+0000", 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0224000000000001e-05 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061131+0000", 2024-09-13T07:02:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7130000000000002e-06 2024-09-13T07:02:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103179+0000", 2024-09-13T07:02:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042047582 2024-09-13T07:02:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103252+0000", 2024-09-13T07:02:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2822000000000002e-05 2024-09-13T07:02:50.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103346+0000", 2024-09-13T07:02:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4539999999999999e-05 2024-09-13T07:02:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:02:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:02:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5333641970000009, 2024-09-13T07:02:50.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:02:50.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:02:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:02:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:02:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:02:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:02:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:02:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:02:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:02:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:02:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:02:50.684 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:02:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:02:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5316896480000004, 2024-09-13T07:02:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:02:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:50.685 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:02:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:02:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:02:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:02:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:02:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:02:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:02:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:02:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:02:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:02:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:02:50.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:02:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:02:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:02:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:02:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:02:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.689 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:50.690 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:50.690 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:50.691 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:50.692 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:50.688+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:50.792 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:50.790+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:50.804 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:50.804 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:50.814 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:50.893 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:50.892+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:50.963 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:50.963 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:50.963 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:50.963 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:50.963 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:50.972 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:50.994 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:50.992+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:51.082 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:02:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.026239652000001, 2024-09-13T07:02:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:02:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:02:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:02:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:02:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:02:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:02:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:02:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:02:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:02:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.868358418, 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:02:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:02:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:02:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:02:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:02:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:02:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:02:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:02:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:02:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:02:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:02:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:02:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:02:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:02:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:02:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:02:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:02:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:02:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.842588706999997, 2024-09-13T07:02:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:02:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:02:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:02:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:02:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:02:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:02:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:02:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:02:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:02:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:02:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:02:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:02:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:02:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:02:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:02:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:02:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.821437541000002, 2024-09-13T07:02:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:02:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:02:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:02:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:02:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:02:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:02:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:02:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:02:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:02:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:02:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:02:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:02:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:02:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:02:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:02:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:02:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:02:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:02:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:02:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.704392777999999, 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:02:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:02:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:02:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:02:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:02:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:02:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:02:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:02:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:02:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:02:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:02:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:02:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:02:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94596 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:02:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.419416299000002, 2024-09-13T07:02:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044330268999999999, 2024-09-13T07:02:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94596 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660995+0000", 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660999+0000", 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2770000000000001e-06 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661007+0000", 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0029999999999994e-06 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661012+0000", 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1279999999999999e-06 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705253+0000", 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044241603999999997 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705293+0000", 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9684999999999997e-05 2024-09-13T07:02:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705326+0000", 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2932000000000003e-05 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.419395532999999, 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:02:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:02:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94598 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.419224812, 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044365860999999999, 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94598 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661034+0000", 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9998465 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661192+0000", 2024-09-13T07:02:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015813000000000001 2024-09-13T07:02:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661199+0000", 2024-09-13T07:02:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5689999999999999e-06 2024-09-13T07:02:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:02:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705452+0000", 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044246273000000003 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705501+0000", 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8748000000000002e-05 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705553+0000", 2024-09-13T07:02:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2241999999999999e-05 2024-09-13T07:02:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107060 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1003520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:02:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.167702522999999, 2024-09-13T07:02:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044341754999999997, 2024-09-13T07:02:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107060 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912709+0000", 2024-09-13T07:02:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912712+0000", 2024-09-13T07:02:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6000000000000001e-06 2024-09-13T07:02:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912717+0000", 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3689999999999998e-06 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912722+0000", 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7219999999999999e-06 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956876+0000", 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044154469000000002 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956957+0000", 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1231999999999997e-05 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957051+0000", 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3858999999999997e-05 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.167686395, 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:02:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:02:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:02:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:02:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:02:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:02:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:02:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:02:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:02:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:02:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:02:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:02:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.167668361, 2024-09-13T07:02:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:02:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:02:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:02:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:02:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:02:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.167652280999999, 2024-09-13T07:02:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:02:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:02:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:02:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.167636604999998, 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:02:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:02:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:02:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:02:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:02:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.167613682999999, 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:02:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:02:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:02:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:02:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:02:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:02:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.167597585999999, 2024-09-13T07:02:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:02:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:02:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:02:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:02:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:02:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:02:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:02:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107067 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.167037569000001, 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04441055, 2024-09-13T07:02:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107067 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:02:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912832+0000", 2024-09-13T07:02:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9994574 2024-09-13T07:02:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913379+0000", 2024-09-13T07:02:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00054719999999999997 2024-09-13T07:02:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913386+0000", 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4549999999999998e-06 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913393+0000", 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6259999999999997e-06 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957776+0000", 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044383143999999999 2024-09-13T07:02:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957785+0000", 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9220000000000003e-06 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107068 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.167013138000002, 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044410502999999997, 2024-09-13T07:02:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107068 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913401+0000", 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913407+0000", 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7520000000000002e-06 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913412+0000", 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8550000000000001e-06 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957802+0000", 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044389606999999998 2024-09-13T07:02:51.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957809+0000", 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9160000000000001e-06 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.166995973999999, 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:02:51.164 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.166953198000002, 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.168 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:51.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.169 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:02:51.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.169 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:02:51.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0099354280000004, 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:51.172 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:02:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:02:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:02:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:02:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:02:51.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:02:51.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:51.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:51.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:51.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:02:51.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:02:51.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.176 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:51.176 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:51.176 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:51.178 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:51.174+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:51.178 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:51.279 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:51.278+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:51.294 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:51.294 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:51.294 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:51.294 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:51.304 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:51.380 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:51.378+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:51.409 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:51.409 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:51.410 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:51.410 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:51.410 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:51.419 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:51.480 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:51.476+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:51.530 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:51.530 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:51.530 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:51.530 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:51.530 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.530 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:51.530 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:51.530 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 86.022888241000004, 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.532 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.532 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:51.532 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.532 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.532 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:02:51.532 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:51.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.532 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.532 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:02:51.533 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:02:51.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.533 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.533 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.533 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:02:51.533 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:02:51.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.533 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.533 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.533 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:02:51.533 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:02:51.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.533 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.534 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.534 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:02:51.534 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:02:51.534 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.534 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.534 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.534 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:02:51.534 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:02:51.534 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.534 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.534 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:51.534 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:02:51.534 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:51.535 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:02:51.536 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:02:51.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.536 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.536 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:02:51.536 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:02:51.536 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.536 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.536 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.536 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:51.536 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:51.537 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.998060530999993, 2024-09-13T07:02:51.537 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:02:51.537 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.537 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.537 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.537 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.537 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.537 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:02:51.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.537 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.537 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.537 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.537 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:51.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:51.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:02:51.538 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:51.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.539 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.539 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:02:51.539 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:02:51.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.539 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.539 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:02:51.539 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:02:51.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.539 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.539 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:02:51.539 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:51.540 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.541 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:51.542 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.459388707000002, 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:51.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.544 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:51.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.544 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:02:51.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:51.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:02:51.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:02:51.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:02:51.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:02:51.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:02:51.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:51.546 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:51.548 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:51.549 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.429763514000001, 2024-09-13T07:02:51.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:02:51.549 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.549 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.549 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.549 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.549 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.549 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:02:51.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.549 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:51.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.550 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:51.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.550 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:02:51.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:51.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.551 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:02:51.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:02:51.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.551 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:02:51.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:02:51.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.551 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:02:51.552 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:02:51.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.552 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.552 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:02:51.552 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:02:51.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.552 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.552 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:02:51.552 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:02:51.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:02:51.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:02:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:02:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:02:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:02:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.897049882999994, 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:51.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:51.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:51.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:51.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.557 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:02:51.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:02:51.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.557 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:02:51.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:02:51.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:02:51.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:02:51.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:02:51.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:02:51.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:02:51.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:02:51.559 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.559 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:51.559 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:02:51.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:02:51.559 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.559 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:51.559 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:02:51.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:02:51.559 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:02:51.560 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:02:51.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:51.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:02:51.560 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:02:51.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:02:51.560 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:02:51.561 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.561 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.561 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.561 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:51.561 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:51.561 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.896967427000007, 2024-09-13T07:02:51.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:02:51.561 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.561 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.561 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.561 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:02:51.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:51.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:02:51.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:02:51.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:02:51.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:02:51.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:51.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:02:51.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:02:51.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:51.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:02:51.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:02:51.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:02:51.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:02:51.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:51.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:02:51.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:02:51.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.896935616999997, 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.567 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.568 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.568 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.568 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:02:51.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.568 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:51.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:02:51.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:02:51.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:02:51.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:02:51.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:02:51.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:02:51.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:02:51.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:02:51.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:51.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:02:51.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:02:51.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:02:51.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:02:51.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:02:51.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:02:51.574 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.574 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.574 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.574 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:51.574 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:51.574 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.896880459000002, 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:51.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:02:51.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:51.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:02:51.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:02:51.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:02:51.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:02:51.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:02:51.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:02:51.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:02:51.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:02:51.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:02:51.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:02:51.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:51.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:02:51.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:02:51.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:02:51.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:02:51.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:51.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:02:51.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:02:51.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:51.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:02:51.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:02:51.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:02:51.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:02:51.580 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.580 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.580 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.896835179999997, 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:02:51.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:02:51.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:02:51.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:02:51.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:02:51.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:51.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:51.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:02:51.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:02:51.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:02:51.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:02:51.586 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.586 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.586 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.586 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:51.586 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:51.587 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.896796645999999, 2024-09-13T07:02:51.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:02:51.587 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.587 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.587 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.587 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.587 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.587 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:02:51.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.587 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:51.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:51.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:02:51.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:51.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:02:51.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:02:51.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:02:51.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:02:51.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:02:51.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:02:51.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:02:51.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:02:51.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:02:51.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:02:51.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:51.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:02:51.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:02:51.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:51.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:02:51.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:02:51.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:02:51.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:02:51.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:51.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:02:51.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:02:51.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:02:51.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:02:51.592 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.592 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.896735027000005, 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:02:51.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:02:51.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:51.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:02:51.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:02:51.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:02:51.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:02:51.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:02:51.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:02:51.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:02:51.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:51.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:02:51.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:02:51.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:02:51.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:02:51.598 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.598 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.598 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.882512344000006, 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.599 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:51.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:51.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:02:51.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:02:51.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:02:51.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:02:51.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:02:51.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:02:51.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:51.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:02:51.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:02:51.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:51.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:02:51.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:02:51.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:02:51.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:02:51.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:51.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:02:51.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:02:51.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:02:51.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:02:51.604 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.605 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.605 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.605 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:51.605 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:51.605 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.870633823999995, 2024-09-13T07:02:51.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:02:51.605 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.605 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:51.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.605 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.606 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:02:51.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.606 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:51.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:51.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:02:51.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:02:51.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:02:51.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:02:51.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:02:51.609 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.609 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.609 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.609 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:51.609 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.870076763, 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:51.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:02:51.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:02:51.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:02:51.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:02:51.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:02:51.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:02:51.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:02:51.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:02:51.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:02:51.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:02:51.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:02:51.613 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.614 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.614 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.614 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:51.614 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:51.614 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.870054608000004, 2024-09-13T07:02:51.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:02:51.614 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.614 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:51.614 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.614 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.614 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:02:51.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:02:51.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:02:51.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:02:51.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:02:51.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:02:51.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:02:51.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:02:51.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:02:51.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:02:51.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:02:51.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:02:51.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:02:51.618 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.618 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.618 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.619 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:51.619 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:51.619 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.870034891000003, 2024-09-13T07:02:51.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:02:51.619 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.619 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:51.619 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.619 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.619 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.619 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:02:51.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.619 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:51.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:51.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:51.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:02:51.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:02:51.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:02:51.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:02:51.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:02:51.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:02:51.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:02:51.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:02:51.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:02:51.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:02:51.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.870017560999997, 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:02:51.623 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.624 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:51.624 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.624 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.624 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.624 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:02:51.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.624 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:51.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:51.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:02:51.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:02:51.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:02:51.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:02:51.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:02:51.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:02:51.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:02:51.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:02:51.627 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.627 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.869721898999998, 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.628 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:51.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:51.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:02:51.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:02:51.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:02:51.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:02:51.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:02:51.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:02:51.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:02:51.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:02:51.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:02:51.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:02:51.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:02:51.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:02:51.632 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.632 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.632 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.632 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:02:51.632 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:51.632 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.308519079999996, 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:51.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:02:51.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:51.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:51.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:02:51.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:02:51.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:51.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:02:51.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:02:51.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:51.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:02:51.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.276440176999998, 2024-09-13T07:02:51.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:02:51.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:02:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:02:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:02:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:02:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:02:51.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:02:51.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:51.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:51.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:51.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:02:51.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:02:51.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.642 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:51.642 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:51.642 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:51.644 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:51.644 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:51.642+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:51.745 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:51.744+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:51.803 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:51.803 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:51.803 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:51.803 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:51.813 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:51.846 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:51.842+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:51.922 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:51.922 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:51.922 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:51.931 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:51.946 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:51.944+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:51.969 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:02:52.042 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:52.042 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:52.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:52.042 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:52.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.042 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:02:52.042 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.547828472000006, 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:52.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:02:52.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:02:52.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:02:52.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:02:52.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:02:52.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:02:52.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:02:52.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.061344908000002, 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:02:52.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:02:52.051 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.061274978999997, 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.052 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:02:52.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:02:52.056 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.056 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.056 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.056 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.056 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:52.056 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.061259106999998, 2024-09-13T07:02:52.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:02:52.056 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:52.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:02:52.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:02:52.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:02:52.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:02:52.060 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.060 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.060 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.061188649999998, 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:02:52.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:02:52.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057836612000003, 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.065 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:02:52.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:02:52.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:02:52.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:02:52.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:02:52.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057820311, 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.069 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.072 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057800526000001, 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.076 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057784253000001, 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:02:52.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:52.080 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057766428000001, 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:02:52.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:02:52.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:02:52.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:02:52.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:02:52.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:02:52.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.084 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:52.085 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057750335000001, 2024-09-13T07:02:52.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:02:52.085 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.085 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.085 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:02:52.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.085 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:02:52.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:02:52.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:02:52.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:02:52.088 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.088 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.088 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.088 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057729148999996, 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:52.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:52.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:02:52.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:02:52.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:02:52.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:02:52.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:02:52.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:02:52.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.093 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.093 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:52.093 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057713116000002, 2024-09-13T07:02:52.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:02:52.093 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.093 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.093 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.093 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.093 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.093 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:02:52.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.093 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:02:52.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.096 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057663877000003, 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.097 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:02:52.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:02:52.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:02:52.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:02:52.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:02:52.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.100 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057645817000001, 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:02:52.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.102 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:52.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:52.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:52.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:02:52.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:02:52.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:02:52.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057629302000002, 2024-09-13T07:02:52.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:02:52.106 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.106 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.106 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.106 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.106 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.106 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:02:52.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.106 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:52.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:02:52.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:52.109 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057610937, 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:02:52.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057594838999997, 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:02:52.113 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:02:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:02:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:02:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:02:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:02:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057578904000003, 2024-09-13T07:02:52.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:02:52.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:02:52.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:02:52.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:02:52.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:02:52.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:02:52.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 54.057562406999999, 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:02:52.121 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:02:52.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:52.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:52.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:52.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:02:52.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:02:52.125 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.125 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.125 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.125 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:52.125 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:52.125 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:52.127 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:52.127 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:52.127 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:52.128 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:52.126+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:52.146 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:52.168 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:52.190 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:52.211 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:52.227 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:52.224+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:52.236 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:52.238 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:52.239 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:52.249 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:52.258 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:52.260 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:52.260 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:52.260 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:52.260 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:52.260 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:52.260 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:52.260 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:52.260 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:52.328 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:52.326+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:52.352 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:52.352 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:52.362 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:52.429 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:52.426+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:52.472 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:52.472 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:52.482 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:52.530 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:52.528+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:52.551 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:02:52.611 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:52.612 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:52.621 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:52.631 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:52.628+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:52.732 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:52.728+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:52.736 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:52.736 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:52.746 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:52.833 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:52.832+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:52.892 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:52.892 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:52.902 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:52.946 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:52.946+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:53.015 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:53.016 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:53.026 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:53.034 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:53.032+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:53.135 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:53.134+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:53.137 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-13T07:02:53.140 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:53.140 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:53.151 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:53.236 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:53.234+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:53.305 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:53.305 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:53.315 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:53.316 INFO:teuthology.orchestra.run.smithi118.stdout:3551 trunc from 0xaeb299c to 0xd01600b 2024-09-13T07:02:53.337 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:53.336+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:53.341 INFO:teuthology.orchestra.run.smithi118.stdout:3552 trunc from 0xd01600b to 0xa1104f0 2024-09-13T07:02:53.359 INFO:teuthology.orchestra.run.smithi118.stdout:3553 punch from 0x3826084 to 0x3827bf4, (0x1b70 bytes) 2024-09-13T07:02:53.380 INFO:teuthology.orchestra.run.smithi118.stdout:3554 write 0xe2ef4db thru 0xe2ef57b (0xa1 bytes) 2024-09-13T07:02:53.386 INFO:teuthology.orchestra.run.smithi118.stdout:3555 read 0xd5b172e thru 0xd5b3a51 (0x2324 bytes) 2024-09-13T07:02:53.386 INFO:teuthology.orchestra.run.smithi118.stdout:3556 punch from 0xd0c1535 to 0xd0c1eeb, (0x9b6 bytes) 2024-09-13T07:02:53.386 INFO:teuthology.orchestra.run.smithi118.stdout:3557 write 0x24fe735 thru 0x2508295 (0x9b61 bytes) 2024-09-13T07:02:53.414 INFO:teuthology.orchestra.run.smithi118.stdout:3558 write 0x7f6680b thru 0x7f71845 (0xb03b bytes) 2024-09-13T07:02:53.422 INFO:teuthology.orchestra.run.smithi118.stdout:3559 write 0x701245c thru 0x701e514 (0xc0b9 bytes) 2024-09-13T07:02:53.429 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:53.429 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:53.438 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:53.436+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:53.439 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:53.539 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:53.536+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:53.544 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:53.544 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:53.554 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:53.560 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:02:53.638 INFO:teuthology.orchestra.run.smithi118.stdout:3560 write 0x1c8352f thru 0x1c858d6 (0x23a8 bytes) 2024-09-13T07:02:53.640 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:53.638+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:53.653 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:53.653 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:53.663 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:53.741 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:53.740+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:53.772 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:53.772 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:53.782 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:53.842 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:53.842+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:53.893 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:53.893 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:53.903 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:53.947 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:53.946+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:54.045 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:54.042+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:54.056 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:54.056 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:54.066 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:54.146 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:54.144+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:54.175 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:54.175 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:54.185 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:54.246 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:54.242+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:54.271 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:02:54.293 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:54.293 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:54.303 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:54.351 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:54.350+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:54.412 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:54.412 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:54.422 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:54.425 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:54.443 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:54.448 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:54.444+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:54.461 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:54.480 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:54.503 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:54.526 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:54.531 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:54.531 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:54.532 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:54.532 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:54.541 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:54.549 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:54.550 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:54.548+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:54.573 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:54.573 INFO:teuthology.orchestra.run.smithi118.stdout:3561 write 0x21afb9c thru 0x21b979d (0x9c02 bytes) 2024-09-13T07:02:54.575 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:54.575 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:54.576 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:54.576 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:54.576 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:54.576 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:54.576 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:54.576 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:54.582 INFO:teuthology.orchestra.run.smithi118.stdout:3562 read 0xce122d1 thru 0xce21f64 (0xfc94 bytes) 2024-09-13T07:02:54.582 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:54.582 INFO:teuthology.orchestra.run.smithi118.stdout:3564 trunc from 0xe2ef57c to 0xe2ac206 2024-09-13T07:02:54.592 INFO:teuthology.orchestra.run.smithi118.stdout:3565 read 0x38f08ef thru 0x38f9e06 (0x9518 bytes) 2024-09-13T07:02:54.593 INFO:teuthology.orchestra.run.smithi118.stdout:3566 write 0x7b59487 thru 0x7b64225 (0xad9f bytes) 2024-09-13T07:02:54.599 INFO:teuthology.orchestra.run.smithi118.stdout:3567 write 0x8fe904 thru 0x9092c7 (0xa9c4 bytes) 2024-09-13T07:02:54.631 INFO:teuthology.orchestra.run.smithi118.stdout:3570 write 0xc183585 thru 0xc1908eb (0xd367 bytes) 2024-09-13T07:02:54.639 INFO:teuthology.orchestra.run.smithi118.stdout:3571 writesame 0x6fb8554 thru 0x6fbd714 data_size 0x1(0x51c1 bytes) 2024-09-13T07:02:54.649 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:54.648+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:54.654 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:54.654 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:54.654 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:54.654 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:54.654 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:54.660 INFO:teuthology.orchestra.run.smithi118.stdout:3572 punch from 0x14d4b9 to 0x1556d7, (0x821e bytes) 2024-09-13T07:02:54.664 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:54.671 INFO:teuthology.orchestra.run.smithi118.stdout:3573 write 0xe2d27fb thru 0xe2d4de8 (0x25ee bytes) 2024-09-13T07:02:54.673 INFO:teuthology.orchestra.run.smithi118.stdout:3574 read 0x1a443d9 thru 0x1a47f56 (0x3b7e bytes) 2024-09-13T07:02:54.674 INFO:teuthology.orchestra.run.smithi118.stdout:3575 punch from 0x12fc79 to 0x134e76, (0x51fd bytes) 2024-09-13T07:02:54.674 INFO:teuthology.orchestra.run.smithi118.stdout:3576 write 0x4169090 thru 0x4178791 (0xf702 bytes) 2024-09-13T07:02:54.694 INFO:teuthology.orchestra.run.smithi118.stdout:3577 write 0xeda6d6b thru 0xedb5b87 (0xee1d bytes) 2024-09-13T07:02:54.697 INFO:teuthology.orchestra.run.smithi118.stdout:3579 writesame 0x1d82a14 thru 0x1d8adbc data_size 0x2d(0x83a9 bytes) 2024-09-13T07:02:54.700 INFO:teuthology.orchestra.run.smithi118.stdout:3580 read 0xfe7b8 thru 0x108eb4 (0xa6fd bytes) 2024-09-13T07:02:54.701 INFO:teuthology.orchestra.run.smithi118.stdout:3581 write 0x50d55ec thru 0x50d604f (0xa64 bytes) 2024-09-13T07:02:54.708 INFO:teuthology.orchestra.run.smithi118.stdout:3583 write 0xe14731b thru 0xe148031 (0xd17 bytes) 2024-09-13T07:02:54.711 INFO:teuthology.orchestra.run.smithi118.stdout:3584 read 0x31c76fe thru 0x31d14c1 (0x9dc4 bytes) 2024-09-13T07:02:54.718 INFO:teuthology.orchestra.run.smithi118.stdout:3585 trunc from 0xedb5b88 to 0xb3497d0 2024-09-13T07:02:54.738 INFO:teuthology.orchestra.run.smithi118.stdout:3586 read 0xa0953f3 thru 0xa09a889 (0x5497 bytes) 2024-09-13T07:02:54.739 INFO:teuthology.orchestra.run.smithi118.stdout:3588 punch from 0x32d3325 to 0x32db213, (0x7eee bytes) 2024-09-13T07:02:54.739 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:54.739 INFO:teuthology.orchestra.run.smithi118.stdout:3590 write 0x61b8567 thru 0x61b995c (0x13f6 bytes) 2024-09-13T07:02:54.742 INFO:teuthology.orchestra.run.smithi118.stdout:3592 write 0xa8dc1eb thru 0xa8ded16 (0x2b2c bytes) 2024-09-13T07:02:54.748 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:54.748 INFO:teuthology.orchestra.run.smithi118.stdout:3594 read 0x34ddbde thru 0x34dea8e (0xeb1 bytes) 2024-09-13T07:02:54.749 INFO:teuthology.orchestra.run.smithi118.stdout:3595 punch from 0x896dffc to 0x897c4ee, (0xe4f2 bytes) 2024-09-13T07:02:54.749 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:54.749 INFO:teuthology.orchestra.run.smithi118.stdout:3597 punch from 0x37ad38 to 0x37c08f, (0x1357 bytes) 2024-09-13T07:02:54.749 INFO:teuthology.orchestra.run.smithi118.stdout:3598 write 0x1f4d191 thru 0x1f50743 (0x35b3 bytes) 2024-09-13T07:02:54.750 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:54.748+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:54.752 INFO:teuthology.orchestra.run.smithi118.stdout:3599 write 0x1005186 thru 0x100ecc4 (0x9b3f bytes) 2024-09-13T07:02:54.780 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:54.780 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:54.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:54.781 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:54.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.781 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:54.781 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:54.781 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 89.274175353999993, 2024-09-13T07:02:54.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:02:54.781 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.781 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:54.781 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.781 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.781 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.781 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:02:54.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:02:54.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:02:54.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:02:54.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:02:54.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:02:54.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:02:54.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:02:54.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:02:54.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:54.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:02:54.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:02:54.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:54.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:02:54.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:02:54.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:54.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:02:54.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:02:54.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:02:54.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:02:54.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.787 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.788 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:54.788 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:54.788 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 89.249347643999997, 2024-09-13T07:02:54.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:02:54.788 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.788 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:54.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.789 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:02:54.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.789 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:54.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:02:54.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:02:54.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:02:54.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:02:54.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:54.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:02:54.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:02:54.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:54.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:02:54.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:02:54.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:02:54.793 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 88.710675820000006, 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.794 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:02:54.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.795 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:54.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:02:54.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:02:54.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:02:54.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:02:54.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:02:54.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:02:54.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:02:54.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:02:54.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:02:54.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:02:54.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:54.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:02:54.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:02:54.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:02:54.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:02:54.800 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.800 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.800 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.800 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:02:54.800 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 88.681050627000005, 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:54.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:02:54.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:02:54.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:02:54.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:02:54.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:02:54.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:02:54.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:02:54.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:02:54.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 88.148336995999998, 2024-09-13T07:02:54.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:02:54.807 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.807 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:54.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.807 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:02:54.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.808 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:54.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:54.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:02:54.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:02:54.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:02:54.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:54.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:02:54.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:02:54.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:54.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:02:54.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:02:54.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.812 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.813 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.813 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:54.813 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:54.813 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 88.148254539999996, 2024-09-13T07:02:54.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:02:54.813 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.813 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:54.813 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.813 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.813 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.813 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:02:54.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:02:54.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:02:54.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:02:54.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:02:54.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:02:54.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:54.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:02:54.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:02:54.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:02:54.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:02:54.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.819 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.819 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:54.819 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:54.819 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 88.148222730000001, 2024-09-13T07:02:54.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:02:54.819 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:02:54.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:02:54.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:02:54.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:54.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:02:54.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:02:54.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:54.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:02:54.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:02:54.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 88.148167572000006, 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:54.825 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.826 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:02:54.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.826 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:54.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:02:54.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:02:54.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:54.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:02:54.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:02:54.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:54.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:02:54.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 88.148122293, 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.831 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:02:54.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.836 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 88.148083759000002, 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:02:54.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.838 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:54.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:02:54.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:02:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:54.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:02:54.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:02:54.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:54.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:02:54.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:02:54.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:54.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.842 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 88.148022139999995, 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.843 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:02:54.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:54.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:54.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:02:54.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:02:54.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:02:54.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:02:54.848 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.848 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.848 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.848 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 88.133799456999995, 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:02:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:02:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:02:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:02:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:02:54.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:02:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:02:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:02:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:02:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:02:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:54.854 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.121920936999999, 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:54.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:02:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:02:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:02:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:02:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:02:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:02:54.858 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.121363876000004, 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.859 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:02:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:02:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:02:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:02:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:02:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:02:54.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.121341720999993, 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:02:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:02:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:02:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:02:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:02:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:02:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:02:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.121322004000007, 2024-09-13T07:02:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:02:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:54.868 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:54.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:02:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:02:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:02:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:02:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.872 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.121304674000001, 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:02:54.876 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.121009012000002, 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.877 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:02:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:02:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:02:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:02:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:02:54.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.559806193, 2024-09-13T07:02:54.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:02:54.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:02:54.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:54.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:02:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:02:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:02:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:02:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:02:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:02:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.527727290000001, 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:54.887 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:02:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:02:54.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:02:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:02:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:54.891 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:54.891 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:54.893 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:54.893 INFO:teuthology.orchestra.run.smithi118.stdout:3600 writesame 0xb47cbad thru 0xb484903 data_size 0x1(0x7d57 bytes) 2024-09-13T07:02:54.893 INFO:teuthology.orchestra.run.smithi118.stdout:3601 trunc from 0xb484904 to 0x3ae0c5 2024-09-13T07:02:54.894 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:54.890+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:54.905 INFO:teuthology.orchestra.run.smithi118.stdout:3602 write 0x52f6705 thru 0x52ff00a (0x8906 bytes) 2024-09-13T07:02:54.912 INFO:teuthology.orchestra.run.smithi118.stdout:3603 read 0x29e68f0 thru 0x29f54fe (0xec0f bytes) 2024-09-13T07:02:54.913 INFO:teuthology.orchestra.run.smithi118.stdout:3604 trunc from 0x52ff00b to 0x55bade3 2024-09-13T07:02:54.925 INFO:teuthology.orchestra.run.smithi118.stdout:3605 writesame 0x87b235 thru 0x886117 data_size 0x1(0xaee3 bytes) 2024-09-13T07:02:54.993 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:54.990+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:54.997 INFO:teuthology.orchestra.run.smithi118.stdout:3607 write 0xdab6a31 thru 0xdabaef2 (0x44c2 bytes) 2024-09-13T07:02:55.004 INFO:teuthology.orchestra.run.smithi118.stdout:3608 write 0x57796af thru 0x578632f (0xcc81 bytes) 2024-09-13T07:02:55.010 INFO:teuthology.orchestra.run.smithi118.stdout:3609 read 0x6c9c81 thru 0x6cd9eb (0x3d6b bytes) 2024-09-13T07:02:55.010 INFO:teuthology.orchestra.run.smithi118.stdout:3611 clone 36 order 23 su 1048576 sc 6 2024-09-13T07:02:55.026 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:55.026 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:55.026 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:55.026 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:55.037 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:55.094 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:55.092+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:55.193 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:55.194 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:55.194 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:55.196 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:55.192+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:55.203 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:55.296 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:55.292+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:55.315 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:02:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.821244441000005, 2024-09-13T07:02:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:02:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:02:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:02:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:02:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:02:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:02:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:02:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:02:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:02:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:02:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:02:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:02:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:02:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:02:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:02:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:02:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:02:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:02:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:02:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:02:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:02:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:02:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.334760877000001, 2024-09-13T07:02:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:02:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:02:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:02:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:02:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:02:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:02:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:02:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:02:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:02:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.334690948000002, 2024-09-13T07:02:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:02:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:02:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:02:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:02:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:02:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:02:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:02:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:02:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:02:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:02:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:02:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:02:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:02:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:02:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.334675076000003, 2024-09-13T07:02:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:02:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:02:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:02:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:02:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:02:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:02:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:02:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:02:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.334604618999997, 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:02:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.331252581000001, 2024-09-13T07:02:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:02:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:02:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:02:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:02:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:02:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:02:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:02:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:02:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:02:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:02:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:02:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:02:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:02:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.331236279999999, 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:02:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:02:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:02:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:02:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:02:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:02:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:02:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:02:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.331216495, 2024-09-13T07:02:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:02:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:02:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:02:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:02:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.331200222, 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:02:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:02:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.331182396999999, 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:02:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:02:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:02:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:02:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:02:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:02:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:02:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:02:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.331166304, 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:02:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:02:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:02:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:02:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:02:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:02:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.331145118000002, 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:02:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:02:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.331129085000001, 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:55.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:02:55.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:02:55.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:02:55.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.331079846000002, 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:55.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:02:55.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:02:55.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.378 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.379 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.379 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:55.379 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.331061785999999, 2024-09-13T07:02:55.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:02:55.379 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.379 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.379 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.379 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.379 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.379 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:02:55.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:02:55.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:02:55.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:02:55.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:02:55.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:02:55.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:02:55.382 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.331045271000001, 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.383 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:02:55.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:02:55.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:02:55.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:02:55.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:02:55.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:02:55.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.331026905999998, 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:02:55.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:02:55.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:02:55.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:02:55.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:02:55.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.331010808000002, 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.391 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:02:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:02:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:02:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.330994873000002, 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.395 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:55.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:02:55.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:02:55.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:02:55.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 57.330978375999997, 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.399 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:02:55.400 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.400 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.400 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.400 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:02:55.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.400 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:55.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:02:55.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:02:55.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:02:55.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:02:55.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:02:55.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:02:55.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:02:55.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:02:55.403 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.403 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.403 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.403 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.403 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.403 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:55.405 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:55.405 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:55.402+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:55.505 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:55.502+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:55.520 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:55.520 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:55.520 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:55.520 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:55.530 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:55.550 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7, 2] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:02:55.550 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:02:55.550 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-13T07:02:55.550 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T07:02:55.550 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T07:02:55.550 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-13T07:02:55.574 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:02:55.606 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:55.606+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:55.671 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:55.671 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T07:02:55.671 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:55.671 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:55.672 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:55.672 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:55.683 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:55.688 INFO:teuthology.orchestra.run.smithi118.stdout:leaving image image_client.0-clone35 intact 2024-09-13T07:02:55.707 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:55.706+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:55.808 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:55.806+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:55.819 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:02:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 89.760922335000004, 2024-09-13T07:02:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:02:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:02:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:02:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:02:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:02:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:02:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:02:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "MOSDPGPush(3.9 100/99 [PushOp(3:90c0de62:::rbd_data.1194e1a844f8.0000000000000013:1b, version: 82'103, data_included: [0~49152,65536~163840,245760~81920,344064~196608,557056~49152,622592~16384,655360~49152,720896~98304,835584~49152,901120~16384,933888~49152,999424~196608,1212416~32768], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:90c0de62:::rbd_data.1194e1a844f8.0000000000000013:1b@82'103, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 1b=[]:{1b=[1b]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1245184, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T07:02:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:46.054720+0000", 2024-09-13T07:02:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.7627933680000005, 2024-09-13T07:02:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0054811130000000001, 2024-09-13T07:02:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.054720+0000", 2024-09-13T07:02:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.054720+0000", 2024-09-13T07:02:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.054719+0000", 2024-09-13T07:02:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.057056+0000", 2024-09-13T07:02:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023371949999999998 2024-09-13T07:02:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.057100+0000", 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3791999999999997e-05 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.057136+0000", 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.587e-05 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.057283+0000", 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000147773 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.057306+0000", 2024-09-13T07:02:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2186000000000001e-05 2024-09-13T07:02:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.060201+0000", 2024-09-13T07:02:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028957309999999999 2024-09-13T07:02:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133453 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1310720~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874059+0000", 2024-09-13T07:02:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9434546939999997, 2024-09-13T07:02:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055990880000000003, 2024-09-13T07:02:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133453 2024-09-13T07:02:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874059+0000", 2024-09-13T07:02:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874059+0000", 2024-09-13T07:02:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874034+0000", 2024-09-13T07:02:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999747 2024-09-13T07:02:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874064+0000", 2024-09-13T07:02:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0392999999999999e-05 2024-09-13T07:02:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874079+0000", 2024-09-13T07:02:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.49e-05 2024-09-13T07:02:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874091+0000", 2024-09-13T07:02:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1986e-05 2024-09-13T07:02:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.878899+0000", 2024-09-13T07:02:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048082860000000002 2024-09-13T07:02:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.878984+0000", 2024-09-13T07:02:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4950999999999994e-05 2024-09-13T07:02:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.879658+0000", 2024-09-13T07:02:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00067367399999999997 2024-09-13T07:02:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133462 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 786432~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874168+0000", 2024-09-13T07:02:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.943345699, 2024-09-13T07:02:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0063862329999999998, 2024-09-13T07:02:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133462 2024-09-13T07:02:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874168+0000", 2024-09-13T07:02:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874168+0000", 2024-09-13T07:02:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874167+0000", 2024-09-13T07:02:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874172+0000", 2024-09-13T07:02:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0119999999999996e-06 2024-09-13T07:02:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874193+0000", 2024-09-13T07:02:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0936000000000001e-05 2024-09-13T07:02:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874204+0000", 2024-09-13T07:02:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1189000000000001e-05 2024-09-13T07:02:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.879696+0000", 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0054921500000000003 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.879779+0000", 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2557000000000003e-05 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.880554+0000", 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00077517799999999996 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133466 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874214+0000", 2024-09-13T07:02:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9432994170000004, 2024-09-13T07:02:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078938729999999992, 2024-09-13T07:02:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133466 2024-09-13T07:02:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874214+0000", 2024-09-13T07:02:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874214+0000", 2024-09-13T07:02:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874213+0000", 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874219+0000", 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.254e-06 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874232+0000", 2024-09-13T07:02:55.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3874e-05 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874244+0000", 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1491000000000001e-05 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.880629+0000", 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0063846939999999998 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.880878+0000", 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000249724 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882108+0000", 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001229653 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133467 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874255+0000", 2024-09-13T07:02:55.850 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9432583780000003, 2024-09-13T07:02:55.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085814259999999996, 2024-09-13T07:02:55.851 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.851 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.851 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133467 2024-09-13T07:02:55.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.851 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874255+0000", 2024-09-13T07:02:55.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874255+0000", 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874254+0000", 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874260+0000", 2024-09-13T07:02:55.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4430000000000002e-06 2024-09-13T07:02:55.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874284+0000", 2024-09-13T07:02:55.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4145e-05 2024-09-13T07:02:55.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874295+0000", 2024-09-13T07:02:55.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0857e-05 2024-09-13T07:02:55.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882136+0000", 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078415900000000007 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882218+0000", 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1866999999999994e-05 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882837+0000", 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061833000000000005 2024-09-13T07:02:55.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.855 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.855 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.855 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133476 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.855 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:02:55.855 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.943162472, 2024-09-13T07:02:55.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0096071349999999993, 2024-09-13T07:02:55.855 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.855 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.856 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133476 2024-09-13T07:02:55.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.856 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:02:55.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:02:55.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874350+0000", 2024-09-13T07:02:55.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:55.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874356+0000", 2024-09-13T07:02:55.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5999999999999997e-06 2024-09-13T07:02:55.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874369+0000", 2024-09-13T07:02:55.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3125000000000001e-05 2024-09-13T07:02:55.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874380+0000", 2024-09-13T07:02:55.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1396999999999999e-05 2024-09-13T07:02:55.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882884+0000", 2024-09-13T07:02:55.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.008503666 2024-09-13T07:02:55.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883050+0000", 2024-09-13T07:02:55.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000165926 2024-09-13T07:02:55.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883958+0000", 2024-09-13T07:02:55.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00090819200000000005 2024-09-13T07:02:55.860 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.860 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.860 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.860 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133477 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1310720~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.860 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:02:55.860 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9431232529999996, 2024-09-13T07:02:55.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010531547, 2024-09-13T07:02:55.861 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.861 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.861 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.861 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.861 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.861 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133477 2024-09-13T07:02:55.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.861 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:02:55.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:02:55.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:02:55.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:55.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874394+0000", 2024-09-13T07:02:55.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7829999999999998e-06 2024-09-13T07:02:55.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874407+0000", 2024-09-13T07:02:55.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3050999999999999e-05 2024-09-13T07:02:55.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874418+0000", 2024-09-13T07:02:55.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0974e-05 2024-09-13T07:02:55.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883998+0000", 2024-09-13T07:02:55.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095798310000000005 2024-09-13T07:02:55.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884148+0000", 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014957499999999999 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884922+0000", 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000774127 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133484 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.865 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9430400990000001, 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011150092, 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133484 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:02:55.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874472+0000", 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874479+0000", 2024-09-13T07:02:55.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5700000000000004e-06 2024-09-13T07:02:55.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874497+0000", 2024-09-13T07:02:55.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7385999999999999e-05 2024-09-13T07:02:55.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874510+0000", 2024-09-13T07:02:55.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3628999999999999e-05 2024-09-13T07:02:55.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884968+0000", 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010458053 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885119+0000", 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000150932 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885624+0000", 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050432500000000002 2024-09-13T07:02:55.869 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133485 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9429884069999996, 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011776926, 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.870 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133485 2024-09-13T07:02:55.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.871 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:02:55.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:02:55.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874524+0000", 2024-09-13T07:02:55.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:55.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874530+0000", 2024-09-13T07:02:55.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8599999999999998e-06 2024-09-13T07:02:55.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874545+0000", 2024-09-13T07:02:55.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5186e-05 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874558+0000", 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3040000000000001e-05 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885700+0000", 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011141514999999999 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885859+0000", 2024-09-13T07:02:55.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015918599999999999 2024-09-13T07:02:55.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886302+0000", 2024-09-13T07:02:55.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044316899999999998 2024-09-13T07:02:55.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.874 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.874 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133494 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.874 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9428526100000001, 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012188055999999999, 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133494 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:02:55.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:02:55.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874660+0000", 2024-09-13T07:02:55.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:55.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874666+0000", 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7300000000000002e-06 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874701+0000", 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5095999999999997e-05 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874714+0000", 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2833000000000001e-05 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886333+0000", 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011619078 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886405+0000", 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2525000000000001e-05 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886849+0000", 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000443697 2024-09-13T07:02:55.878 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133498 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.942787848, 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012651943000000001, 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.879 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133498 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874725+0000", 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874731+0000", 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0159999999999997e-06 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874788+0000", 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7021000000000002e-05 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874797+0000", 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3430000000000002e-06 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886880+0000", 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012083287 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886959+0000", 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8582000000000001e-05 2024-09-13T07:02:55.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887378+0000", 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041888400000000001 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9427059780000002, 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:02:55.883 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.884 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.884 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:02:55.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.884 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:02:55.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:02:55.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:02:55.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:55.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:02:55.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:02:55.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:02:55.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:02:55.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:02:55.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:02:55.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:02:55.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:02:55.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:02:55.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:02:55.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:02:55.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9426565900000003, 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.888 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:02:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:02:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:02:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:02:55.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:02:55.892 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9426338830000001, 2024-09-13T07:02:55.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:02:55.893 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.893 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.893 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.893 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.893 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.893 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:02:55.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.893 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:02:55.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:02:55.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:02:55.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:02:55.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:02:55.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:02:55.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:02:55.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:02:55.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:02:55.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:02:55.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:02:55.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:02:55.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9425896900000001, 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:02:55.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:02:55.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:02:55.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:02:55.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:02:55.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:02:55.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:02:55.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:02:55.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:02:55.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:02:55.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:02:55.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:02:55.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:02:55.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.901 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133516 3.0 3:07bbab57:::rbd_data.11946d7ee8e1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9317545090000001, 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010861226999999999, 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133516 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.902 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:02:55.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:02:55.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885757+0000", 2024-09-13T07:02:55.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:55.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886801+0000", 2024-09-13T07:02:55.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010439550000000001 2024-09-13T07:02:55.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886827+0000", 2024-09-13T07:02:55.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5803e-05 2024-09-13T07:02:55.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886839+0000", 2024-09-13T07:02:55.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2136e-05 2024-09-13T07:02:55.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886894+0000", 2024-09-13T07:02:55.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4434000000000001e-05 2024-09-13T07:02:55.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887043+0000", 2024-09-13T07:02:55.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014903200000000001 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887376+0000", 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00033327399999999999 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.894299+0000", 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069232620000000003 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:55.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896556+0000", 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022571309999999999 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896599+0000", 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2620999999999997e-05 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896620+0000", 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1367999999999999e-05 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.907 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.908 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133521 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 2097152~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:55.908 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.886882+0000", 2024-09-13T07:02:55.908 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9306316429999999, 2024-09-13T07:02:55.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00551499, 2024-09-13T07:02:55.908 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.908 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:55.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:55.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:55.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:55.908 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133521 2024-09-13T07:02:55.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.909 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886882+0000", 2024-09-13T07:02:55.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886882+0000", 2024-09-13T07:02:55.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886882+0000", 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886884+0000", 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5100000000000001e-06 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886889+0000", 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1279999999999999e-06 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886893+0000", 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2560000000000004e-06 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.890804+0000", 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039109669999999996 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.891901+0000", 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010970120000000001 2024-09-13T07:02:55.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.892397+0000", 2024-09-13T07:02:55.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049547800000000004 2024-09-13T07:02:55.912 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.912 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.912 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.912 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:02:55.912 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:02:55.912 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.9116106009999996, 2024-09-13T07:02:55.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:02:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:02:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:02:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:02:55.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:02:55.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:02:55.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:02:55.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:02:55.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:02:55.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:02:55.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:02:55.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:55.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:02:55.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:02:55.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:02:55.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:02:55.917 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.917 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.917 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.917 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133627 3.1f e104/101 3:fe6b0ce8:::rbd_data.11946d7ee8e1.000000000000000d:head v 104'181, mlcod=0'0)", 2024-09-13T07:02:55.917 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.921853+0000", 2024-09-13T07:02:55.917 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.8956602220000001, 2024-09-13T07:02:55.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055765900000000002, 2024-09-13T07:02:55.917 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.921853+0000", 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.921853+0000", 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.921853+0000", 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.924842+0000", 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029892650000000001 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.924852+0000", 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3379999999999994e-06 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:55.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.924857+0000", 2024-09-13T07:02:55.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9960000000000001e-06 2024-09-13T07:02:55.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:55.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.924902+0000", 2024-09-13T07:02:55.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5126000000000001e-05 2024-09-13T07:02:55.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:55.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.924924+0000", 2024-09-13T07:02:55.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1807999999999999e-05 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.927392+0000", 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024684160000000002 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.927430+0000", 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8021000000000001e-05 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.921 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:55.922 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:55.922 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:55.923 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:55.923 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-13T07:02:55.924 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:55.922+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:56.024 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:56.020+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:56.057 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:56.057 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:56.057 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:56.057 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:56.067 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:56.125 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:56.124+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:56.187 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:56.187 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:56.187 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:56.187 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:56.187 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:56.198 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:56.225 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:56.226+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:56.257 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:56.257 INFO:teuthology.orchestra.run.smithi118.stdout:3613 write 0x5a05eae thru 0x5a138ee (0xda41 bytes) 2024-09-13T07:02:56.275 INFO:teuthology.orchestra.run.smithi118.stdout:3614 punch from 0x8aab7bf to 0x8ab2977, (0x71b8 bytes) 2024-09-13T07:02:56.275 INFO:teuthology.orchestra.run.smithi118.stdout:3615 writesame 0xbbdb9a8 thru 0xbbdc4f0 data_size 0x9(0xb49 bytes) 2024-09-13T07:02:56.281 INFO:teuthology.orchestra.run.smithi118.stdout:3616 read 0xcaeefd4 thru 0xcaf25a4 (0x35d1 bytes) 2024-09-13T07:02:56.281 INFO:teuthology.orchestra.run.smithi118.stdout:3617 write 0x4efa3e0 thru 0x4f05963 (0xb584 bytes) 2024-09-13T07:02:56.289 INFO:teuthology.orchestra.run.smithi118.stdout:3618 write 0x882904e thru 0x882ce76 (0x3e29 bytes) 2024-09-13T07:02:56.295 INFO:teuthology.orchestra.run.smithi118.stdout:3619 write 0x6be1d4e thru 0x6be99bd (0x7c70 bytes) 2024-09-13T07:02:56.301 INFO:teuthology.orchestra.run.smithi118.stdout:3620 read 0x4b8e2bc thru 0x4b9b5c8 (0xd30d bytes) 2024-09-13T07:02:56.303 INFO:teuthology.orchestra.run.smithi118.stdout:3621 punch from 0x91b3ee1 to 0x91b73b6, (0x34d5 bytes) 2024-09-13T07:02:56.303 INFO:teuthology.orchestra.run.smithi118.stdout:3622 writesame 0x80f9da6 thru 0x80fdb76 data_size 0x19(0x3dd1 bytes) 2024-09-13T07:02:56.304 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.304 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.182096872999999, 2024-09-13T07:02:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:02:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:02:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.308 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.182076395999999, 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.309 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:02:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87036 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 6963200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175392330000001, 2024-09-13T07:02:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042468831999999998, 2024-09-13T07:02:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87036 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127964+0000", 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7799999999999999e-06 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127967+0000", 2024-09-13T07:02:56.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9730000000000002e-06 2024-09-13T07:02:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127971+0000", 2024-09-13T07:02:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4349999999999999e-06 2024-09-13T07:02:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170202+0000", 2024-09-13T07:02:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042230940000000002 2024-09-13T07:02:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170293+0000", 2024-09-13T07:02:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0858000000000003e-05 2024-09-13T07:02:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170431+0000", 2024-09-13T07:02:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013821699999999999 2024-09-13T07:02:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175379792999998, 2024-09-13T07:02:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:02:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.318 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:02:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:02:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:02:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:02:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:02:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:02:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.322 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175367723999997, 2024-09-13T07:02:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:02:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:02:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.323 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:56.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:02:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:02:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:02:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:02:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:02:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:02:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:02:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:02:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:02:56.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175355281000002, 2024-09-13T07:02:56.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:02:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:02:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:02:56.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:02:56.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175337683000002, 2024-09-13T07:02:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:02:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:02:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:56.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:02:56.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175326980000001, 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.336 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:02:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:02:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:02:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:02:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:02:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:02:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:02:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:02:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:02:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:02:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:02:56.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:02:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175283047000001, 2024-09-13T07:02:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:02:56.341 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:02:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:02:56.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:02:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:02:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:02:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:02:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:02:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.345 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175263260999998, 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:02:56.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:02:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:02:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:02:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:02:56.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:02:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:02:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:02:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:02:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:02:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:02:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:02:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:02:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.350 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175247511000002, 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:56.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:02:56.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:02:56.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:02:56.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:02:56.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:02:56.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:02:56.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:02:56.354 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175236259000002, 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:02:56.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:02:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:02:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:02:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175225660000002, 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:02:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175216167000002, 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:02:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:02:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:02:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:02:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:02:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:02:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:02:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:02:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175206623000001, 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.17517402, 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:02:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.175160996999999, 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:02:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:02:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:02:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:02:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:02:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:02:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:02:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:02:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:02:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.171810012000002, 2024-09-13T07:02:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:02:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:02:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:02:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:02:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:02:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:02:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.171789734999997, 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:02:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:02:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:02:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:02:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 34.171769533999999, 2024-09-13T07:02:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:02:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:02:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:56.394 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:56.394 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:56.395 INFO:teuthology.orchestra.run.smithi118.stdout:3626 write 0xa719b88 thru 0xa71c87e (0x2cf7 bytes) 2024-09-13T07:02:56.395 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:56.397 INFO:teuthology.orchestra.run.smithi118.stdout:3627 punch from 0x7edb934 to 0x7ee1b01, (0x61cd bytes) 2024-09-13T07:02:56.397 INFO:teuthology.orchestra.run.smithi118.stdout:3628 read 0x8330951 thru 0x83385c7 (0x7c77 bytes) 2024-09-13T07:02:56.397 INFO:teuthology.orchestra.run.smithi118.stdout:3629 writesame 0x84eda98 thru 0x84fcec5 data_size 0x5f(0xf42e bytes) 2024-09-13T07:02:56.397 INFO:teuthology.orchestra.run.smithi118.stdout:3630 read 0xc07863a thru 0xc0833a3 (0xad6a bytes) 2024-09-13T07:02:56.397 INFO:teuthology.orchestra.run.smithi118.stdout:3631 writesame 0xdbe1702 thru 0xdbe61a7 data_size 0x4e(0x4aa6 bytes) 2024-09-13T07:02:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:3632 read 0x7ccb731 thru 0x7cceb8e (0x345e bytes) 2024-09-13T07:02:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:3633 read 0x823317d thru 0x82333ad (0x231 bytes) 2024-09-13T07:02:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:3636 write 0x6d2c1db thru 0x6d32be4 (0x6a0a bytes) 2024-09-13T07:02:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:3637 punch from 0x8f55dc4 to 0x8f619a5, (0xbbe1 bytes) 2024-09-13T07:02:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:3639 writesame 0x43c1bcc thru 0x43c1e00 data_size 0x1(0x235 bytes) 2024-09-13T07:02:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:3646 writesame 0x2f3a5a thru 0x2fcc75 data_size 0x12(0x921c bytes) 2024-09-13T07:02:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:3647 read 0xc448f88 thru 0xc454f61 (0xbfda bytes) 2024-09-13T07:02:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:3648 trunc from 0xdbe61a8 to 0xe2368ee 2024-09-13T07:02:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:3649 read 0xacf7f99 thru 0xad03bed (0xbc55 bytes) 2024-09-13T07:02:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:3650 write 0xdaf0680 thru 0xdaf4a35 (0x43b6 bytes) 2024-09-13T07:02:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:3652 read 0x1bba61b thru 0x1bc7224 (0xcc0a bytes) 2024-09-13T07:02:56.398 INFO:teuthology.orchestra.run.smithi118.stdout:3653 clone 37 order 22 su 65536 sc 3 2024-09-13T07:02:56.399 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:56.394+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:56.496 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:56.494+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:56.515 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:56.516 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:56.516 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:02:56.516 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:56.526 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:56.577 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:02:56.589 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:02:56.596 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:56.592+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:56.640 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:56.641 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:56.650 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:02:56.698 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:56.696+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:56.704 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone36 from 0xe2368ee (overlap 0xdabaef3) to 0x15df7e5 2024-09-13T07:02:56.739 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:56.761 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:56.769 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:02:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:02:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:02:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167611599000001, 2024-09-13T07:02:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:02:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:02:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:02:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:02:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:02:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:02:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:02:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:02:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:02:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:02:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:02:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167580498, 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:02:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:02:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:02:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:02:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:02:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:02:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:02:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:02:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:02:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:02:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:02:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:02:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167558184000001, 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:02:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:02:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:02:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:02:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:02:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:02:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:02:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167535152999999, 2024-09-13T07:02:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:02:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:02:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:02:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:02:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:02:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:02:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:02:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:02:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:02:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:02:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:02:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:02:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:02:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:02:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:02:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:02:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:02:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:02:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167510603, 2024-09-13T07:02:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:02:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:02:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:02:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167474894, 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167450215000001, 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:02:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:02:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:02:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:02:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:02:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:02:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:02:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:02:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167416941000001, 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:02:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:02:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:02:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:02:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:02:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:02:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167390152999999, 2024-09-13T07:02:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:02:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:02:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:02:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:02:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:02:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:02:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:02:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:02:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:02:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:02:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167365599, 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:02:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:02:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:02:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:02:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:02:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:02:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:02:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:02:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:02:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:02:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:02:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167335830000001, 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:02:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:02:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:02:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:02:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:02:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:02:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:02:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:02:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:02:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:02:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:02:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:02:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167275987, 2024-09-13T07:02:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:02:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:02:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:02:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167209137, 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:02:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:02:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:02:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:02:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:02:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:02:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:02:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:02:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167176635000001, 2024-09-13T07:02:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:02:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:02:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:02:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:02:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:02:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:02:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:02:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:02:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:02:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:02:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:02:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:02:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:02:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:02:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:02:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:02:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:02:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167142014, 2024-09-13T07:02:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:02:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:02:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:02:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:02:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:02:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:02:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:02:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:02:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:02:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:02:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:02:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:02:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:02:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167091783, 2024-09-13T07:02:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:02:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:02:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:02:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:02:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:02:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:02:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:02:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:02:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:02:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:02:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:02:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167043596999999, 2024-09-13T07:02:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:02:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:02:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:02:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:02:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:02:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:02:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:02:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:02:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:02:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:02:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:02:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:02:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:02:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:02:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:02:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:02:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.167016959, 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:02:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:02:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:02:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:02:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:02:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:02:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:02:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:02:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:02:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.166970124000001, 2024-09-13T07:02:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:02:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:02:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:02:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.163615707, 2024-09-13T07:02:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:02:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:02:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:02:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:02:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:02:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:02:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:02:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:56.869 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:56.870 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:56.870 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:56.870 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:56.870 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:56.871 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:02:56.871 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:56.866+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:56.872 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-13T07:02:56.873 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:56.902 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:56.904 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:56.904 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:56.905 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:56.905 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:56.905 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:56.905 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:56.905 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:56.905 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:56.972 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:56.970+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:56.982 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:56.982 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:56.993 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:02:57.072 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:57.068+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:57.146 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:57.146 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:57.156 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:02:57.173 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:57.170+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:57.264 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:57.264 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:57.265 INFO:teuthology.orchestra.run.smithi118.stdout:3656 write 0xc9f8ec0 thru 0xca08d88 (0xfec9 bytes) 2024-09-13T07:02:57.274 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:02:57.274 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:57.270+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:57.289 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:57.289 INFO:teuthology.orchestra.run.smithi118.stdout:3659 write 0x38a7dd7 thru 0x38b276c (0xa996 bytes) 2024-09-13T07:02:57.300 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:57.300 INFO:teuthology.orchestra.run.smithi118.stdout:3661 write 0xaa19431 thru 0xaa1ad8c (0x195c bytes) 2024-09-13T07:02:57.309 INFO:teuthology.orchestra.run.smithi118.stdout:3662 writesame 0x8b71136 thru 0x8b7472f data_size 0x15(0x35fa bytes) 2024-09-13T07:02:57.323 INFO:teuthology.orchestra.run.smithi118.stdout:3665 write 0x49c023a thru 0x49c92dd (0x90a4 bytes) 2024-09-13T07:02:57.361 INFO:teuthology.orchestra.run.smithi118.stdout:3669 trunc from 0xe2368ee to 0x707c29f 2024-09-13T07:02:57.374 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:57.372+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:57.382 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:57.383 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:57.394 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:02:57.402 INFO:teuthology.orchestra.run.smithi118.stdout:3670 write 0xf81441 thru 0xf85cd6 (0x4896 bytes) 2024-09-13T07:02:57.416 INFO:teuthology.orchestra.run.smithi118.stdout:3671 punch from 0x64f0371 to 0x64ff53a, (0xf1c9 bytes) 2024-09-13T07:02:57.438 INFO:teuthology.orchestra.run.smithi118.stdout:3672 write 0xa9da9c8 thru 0xa9e6327 (0xb960 bytes) 2024-09-13T07:02:57.446 INFO:teuthology.orchestra.run.smithi118.stdout:3673 read 0x51fe202 thru 0x520da7d (0xf87c bytes) 2024-09-13T07:02:57.459 INFO:teuthology.orchestra.run.smithi118.stdout:3674 trunc from 0xa9e6328 to 0x48be099 2024-09-13T07:02:57.475 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:57.472+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:57.499 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:57.499 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:57.508 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:02:57.525 INFO:teuthology.orchestra.run.smithi118.stdout:3676 writesame 0xde5c46c thru 0xde6929d data_size 0x1(0xce32 bytes) 2024-09-13T07:02:57.525 INFO:teuthology.orchestra.run.smithi118.stdout:3677 write 0x87b0d67 thru 0x87b589f (0x4b39 bytes) 2024-09-13T07:02:57.531 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:57.531 INFO:teuthology.orchestra.run.smithi118.stdout:3680 write 0x4dafaa0 thru 0x4dba434 (0xa995 bytes) 2024-09-13T07:02:57.534 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:57.534 INFO:teuthology.orchestra.run.smithi118.stdout:3682 read 0x6425e99 thru 0x64341a6 (0xe30e bytes) 2024-09-13T07:02:57.534 INFO:teuthology.orchestra.run.smithi118.stdout:3683 read 0x6e4ad33 thru 0x6e4ea61 (0x3d2f bytes) 2024-09-13T07:02:57.534 INFO:teuthology.orchestra.run.smithi118.stdout:3684 read 0x107bc48 thru 0x108038d (0x4746 bytes) 2024-09-13T07:02:57.535 INFO:teuthology.orchestra.run.smithi118.stdout:3686 read 0x2923b1e thru 0x292c66f (0x8b52 bytes) 2024-09-13T07:02:57.535 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:57.535 INFO:teuthology.orchestra.run.smithi118.stdout:3689 write 0x7b3d7c8 thru 0x7b4b745 (0xdf7e bytes) 2024-09-13T07:02:57.542 INFO:teuthology.orchestra.run.smithi118.stdout:3690 write 0x7aef1df thru 0x7af6b27 (0x7949 bytes) 2024-09-13T07:02:57.562 INFO:teuthology.orchestra.run.smithi118.stdout:3691 writesame 0xe644cf7 thru 0xe65485e data_size 0x14(0xfb68 bytes) 2024-09-13T07:02:57.562 INFO:teuthology.orchestra.run.smithi118.stdout:3695 read 0x3a64b03 thru 0x3a700ce (0xb5cc bytes) 2024-09-13T07:02:57.562 INFO:teuthology.orchestra.run.smithi118.stdout:3696 writesame 0xe16f8e9 thru 0xe17e0ff data_size 0x11(0xe817 bytes) 2024-09-13T07:02:57.573 INFO:teuthology.orchestra.run.smithi118.stdout:3697 writesame 0xb9f8f31 thru 0xb9f9c98 data_size 0x2c(0xd68 bytes) 2024-09-13T07:02:57.576 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:57.574+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:57.580 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:57.580 INFO:teuthology.orchestra.run.smithi118.stdout:3700 writesame 0x7610c7c thru 0x7620891 data_size 0x2(0xfc16 bytes) 2024-09-13T07:02:57.609 INFO:teuthology.orchestra.run.smithi118.stdout:3702 read 0x9b044e1 thru 0x9b0df0a (0x9a2a bytes) 2024-09-13T07:02:57.609 INFO:teuthology.orchestra.run.smithi118.stdout:3703 write 0x7f07f01 thru 0x7f16c0e (0xed0e bytes) 2024-09-13T07:02:57.612 INFO:teuthology.orchestra.run.smithi118.stdout:3704 punch from 0xcf0672 to 0xcfb74d, (0xb0db bytes) 2024-09-13T07:02:57.612 INFO:teuthology.orchestra.run.smithi118.stdout:3706 writesame 0x2640372 thru 0x2643948 data_size 0xb(0x35d7 bytes) 2024-09-13T07:02:57.622 INFO:teuthology.orchestra.run.smithi118.stdout:3707 writesame 0x342d3c8 thru 0x342e60d data_size 0x1(0x1246 bytes) 2024-09-13T07:02:57.627 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:57.627 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:57.627 INFO:teuthology.orchestra.run.smithi118.stdout:3708 writesame 0xcff3f88 thru 0xcff91e0 data_size 0x1(0x5259 bytes) 2024-09-13T07:02:57.637 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:02:57.648 INFO:teuthology.orchestra.run.smithi118.stdout:3709 write 0x3f2313f thru 0x3f2fad9 (0xc99b bytes) 2024-09-13T07:02:57.660 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:57.660 INFO:teuthology.orchestra.run.smithi118.stdout:3711 write 0xb0128e5 thru 0xb01e850 (0xbf6c bytes) 2024-09-13T07:02:57.677 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:57.676+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:57.687 INFO:teuthology.orchestra.run.smithi118.stdout:3713 trunc from 0xe65485f to 0x2d72321 2024-09-13T07:02:57.740 INFO:teuthology.orchestra.run.smithi118.stdout:3714 write 0x56218f3 thru 0x5623a05 (0x2113 bytes) 2024-09-13T07:02:57.743 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:57.743 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:57.747 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:57.747 INFO:teuthology.orchestra.run.smithi118.stdout:3716 write 0x14d9dd6 thru 0x14ddccb (0x3ef6 bytes) 2024-09-13T07:02:57.754 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:02:57.778 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:57.776+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:57.788 INFO:teuthology.orchestra.run.smithi118.stdout:3717 write 0xd504d9a thru 0xd50b923 (0x6b8a bytes) 2024-09-13T07:02:57.795 INFO:teuthology.orchestra.run.smithi118.stdout:3719 write 0x9dd734 thru 0x9e2d60 (0x562d bytes) 2024-09-13T07:02:57.831 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:57.831 INFO:teuthology.orchestra.run.smithi118.stdout:3722 write 0x1eccd78 thru 0x1ed6dfb (0xa084 bytes) 2024-09-13T07:02:57.845 INFO:teuthology.orchestra.run.smithi118.stdout:3724 writesame 0x427ea48 thru 0x4284c82 data_size 0x1(0x623b bytes) 2024-09-13T07:02:57.871 INFO:teuthology.orchestra.run.smithi118.stdout:3725 punch from 0x5bb9219 to 0x5bbd490, (0x4277 bytes) 2024-09-13T07:02:57.871 INFO:teuthology.orchestra.run.smithi118.stdout:3727 read 0x6a7cc70 thru 0x6a7e318 (0x16a9 bytes) 2024-09-13T07:02:57.871 INFO:teuthology.orchestra.run.smithi118.stdout:3728 read 0x390f103 thru 0x391e3f3 (0xf2f1 bytes) 2024-09-13T07:02:57.871 INFO:teuthology.orchestra.run.smithi118.stdout:3729 write 0xbfd6d56 thru 0xbfda5fb (0x38a6 bytes) 2024-09-13T07:02:57.876 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:57.876 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:57.877 INFO:teuthology.orchestra.run.smithi118.stdout:3730 trunc from 0xd50b924 to 0xdc493af 2024-09-13T07:02:57.879 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:57.878+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:57.886 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:02:57.912 INFO:teuthology.orchestra.run.smithi118.stdout:3731 trunc from 0xdc493af to 0x44aaee2 2024-09-13T07:02:57.947 INFO:teuthology.orchestra.run.smithi118.stdout:3732 write 0x53968f3 thru 0x53a4774 (0xde82 bytes) 2024-09-13T07:02:57.955 INFO:teuthology.orchestra.run.smithi118.stdout:3733 read 0x3fbd466 thru 0x3fbe521 (0x10bc bytes) 2024-09-13T07:02:57.956 INFO:teuthology.orchestra.run.smithi118.stdout:3735 writesame 0x182415d thru 0x1831670 data_size 0x1a(0xd514 bytes) 2024-09-13T07:02:57.980 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:57.978+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:57.984 INFO:teuthology.orchestra.run.smithi118.stdout:3736 trunc from 0x53a4775 to 0xabebd85 2024-09-13T07:02:57.995 INFO:teuthology.orchestra.run.smithi118.stdout:3737 punch from 0x6f5b076 to 0x6f62025, (0x6faf bytes) 2024-09-13T07:02:57.995 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:57.995 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:57.995 INFO:teuthology.orchestra.run.smithi118.stdout:3741 read 0x74b65cd thru 0x74bf60a (0x903e bytes) 2024-09-13T07:02:57.995 INFO:teuthology.orchestra.run.smithi118.stdout:3742 read 0x73c0160 thru 0x73ccb82 (0xca23 bytes) 2024-09-13T07:02:57.996 INFO:teuthology.orchestra.run.smithi118.stdout:3743 write 0x7dea37a thru 0x7decea1 (0x2b28 bytes) 2024-09-13T07:02:57.996 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:57.996 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:58.001 INFO:teuthology.orchestra.run.smithi118.stdout:3744 read 0x62c95e5 thru 0x62d1a76 (0x8492 bytes) 2024-09-13T07:02:58.005 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:02:58.015 INFO:teuthology.orchestra.run.smithi118.stdout:3745 write 0xc52770a thru 0xc529ebb (0x27b2 bytes) 2024-09-13T07:02:58.022 INFO:teuthology.orchestra.run.smithi118.stdout:3746 read 0x87afa23 thru 0x87b5eec (0x64ca bytes) 2024-09-13T07:02:58.023 INFO:teuthology.orchestra.run.smithi118.stdout:3747 punch from 0x52be16f to 0x52c9954, (0xb7e5 bytes) 2024-09-13T07:02:58.023 INFO:teuthology.orchestra.run.smithi118.stdout:3748 write 0x1323317 thru 0x13317df (0xe4c9 bytes) 2024-09-13T07:02:58.047 INFO:teuthology.orchestra.run.smithi118.stdout:3749 write 0xd24ea85 thru 0xd251682 (0x2bfe bytes) 2024-09-13T07:02:58.055 INFO:teuthology.orchestra.run.smithi118.stdout:3750 writesame 0x7b07dc4 thru 0x7b08fe7 data_size 0x24(0x1224 bytes) 2024-09-13T07:02:58.061 INFO:teuthology.orchestra.run.smithi118.stdout:3751 write 0xc5ec113 thru 0xc5ec374 (0x262 bytes) 2024-09-13T07:02:58.064 INFO:teuthology.orchestra.run.smithi118.stdout:3752 write 0xa9d2ba8 thru 0xa9d94ee (0x6947 bytes) 2024-09-13T07:02:58.071 INFO:teuthology.orchestra.run.smithi118.stdout:3753 read 0x4639f80 thru 0x4644e18 (0xae99 bytes) 2024-09-13T07:02:58.072 INFO:teuthology.orchestra.run.smithi118.stdout:3755 punch from 0x381e42a to 0x382696d, (0x8543 bytes) 2024-09-13T07:02:58.072 INFO:teuthology.orchestra.run.smithi118.stdout:3756 writesame 0xa277177 thru 0xa28283e data_size 0x4(0xb6c8 bytes) 2024-09-13T07:02:58.081 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:58.080+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:58.087 INFO:teuthology.orchestra.run.smithi118.stdout:3757 write 0x19bb4e thru 0x1a4a78 (0x8f2b bytes) 2024-09-13T07:02:58.093 INFO:teuthology.orchestra.run.smithi118.stdout:3759 trunc from 0xd251683 to 0xe428f37 2024-09-13T07:02:58.104 INFO:teuthology.orchestra.run.smithi118.stdout:3760 read 0x63f1d99 thru 0x63f9316 (0x757e bytes) 2024-09-13T07:02:58.105 INFO:teuthology.orchestra.run.smithi118.stdout:3761 writesame 0x604dbe3 thru 0x605b7cd data_size 0x1(0xdbeb bytes) 2024-09-13T07:02:58.122 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:58.122 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:58.131 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:02:58.151 INFO:teuthology.orchestra.run.smithi118.stdout:3763 punch from 0xdffd2ba to 0xdffd2df, (0x25 bytes) 2024-09-13T07:02:58.151 INFO:teuthology.orchestra.run.smithi118.stdout:3764 read 0xd53fc9 thru 0xd609ea (0xca22 bytes) 2024-09-13T07:02:58.151 INFO:teuthology.orchestra.run.smithi118.stdout:3766 read 0x6d2c538 thru 0x6d3225f (0x5d28 bytes) 2024-09-13T07:02:58.152 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:58.153 INFO:teuthology.orchestra.run.smithi118.stdout:3769 write 0x2971998 thru 0x297e6c3 (0xcd2c bytes) 2024-09-13T07:02:58.156 INFO:teuthology.orchestra.run.smithi118.stdout:3770 read 0x2ad61bf thru 0x2ae5d31 (0xfb73 bytes) 2024-09-13T07:02:58.157 INFO:teuthology.orchestra.run.smithi118.stdout:3771 writesame 0x6a159c6 thru 0x6a1aadd data_size 0x3c(0x5118 bytes) 2024-09-13T07:02:58.160 INFO:teuthology.orchestra.run.smithi118.stdout:3772 punch from 0x164b27d to 0x1656081, (0xae04 bytes) 2024-09-13T07:02:58.160 INFO:teuthology.orchestra.run.smithi118.stdout:3773 read 0xbd42104 thru 0xbd4869a (0x6597 bytes) 2024-09-13T07:02:58.160 INFO:teuthology.orchestra.run.smithi118.stdout:3774 read 0x4df011e thru 0x4df84c3 (0x83a6 bytes) 2024-09-13T07:02:58.161 INFO:teuthology.orchestra.run.smithi118.stdout:3775 writesame 0x608d75e thru 0x609b796 data_size 0x1(0xe039 bytes) 2024-09-13T07:02:58.181 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:58.180+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:58.207 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:58.207 INFO:teuthology.orchestra.run.smithi118.stdout:3778 write 0x9403217 thru 0x941082e (0xd618 bytes) 2024-09-13T07:02:58.215 INFO:teuthology.orchestra.run.smithi118.stdout:3779 writesame 0x7983b2f thru 0x7990684 data_size 0x11(0xcb56 bytes) 2024-09-13T07:02:58.221 INFO:teuthology.orchestra.run.smithi118.stdout:3780 write 0x4e04ee3 thru 0x4e0bbaf (0x6ccd bytes) 2024-09-13T07:02:58.243 INFO:teuthology.orchestra.run.smithi118.stdout:3781 trunc from 0xe428f37 to 0x348504d 2024-09-13T07:02:58.257 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:58.257 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:58.268 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:02:58.282 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:02:58.282+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:02:58.289 INFO:teuthology.orchestra.run.smithi118.stdout:3782 write 0xd0ab26b thru 0xd0b4233 (0x8fc9 bytes) 2024-09-13T07:02:58.297 INFO:teuthology.orchestra.run.smithi118.stdout:3783 punch from 0x61717e7 to 0x6171c1e, (0x437 bytes) 2024-09-13T07:02:58.298 INFO:teuthology.orchestra.run.smithi118.stdout:3784 trunc from 0xd0b4234 to 0xcc92467 2024-09-13T07:02:58.309 INFO:teuthology.orchestra.run.smithi118.stdout:3787 read 0xf56b7d thru 0xf64be8 (0xe06c bytes) 2024-09-13T07:02:58.310 INFO:teuthology.orchestra.run.smithi118.stdout:3788 punch from 0xb5bdf11 to 0xb5c8234, (0xa323 bytes) 2024-09-13T07:02:58.311 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:58.330 INFO:teuthology.orchestra.run.smithi118.stdout:3790 trunc from 0xcc92467 to 0xdff790 2024-09-13T07:02:58.372 INFO:teuthology.orchestra.run.smithi118.stdout:3792 writesame 0x784e1e6 thru 0x7851b34 data_size 0x1(0x394f bytes) 2024-09-13T07:02:58.373 INFO:teuthology.orchestra.run.smithi118.stdout:3793 punch from 0x50f78ce to 0x50f7e72, (0x5a4 bytes) 2024-09-13T07:02:58.373 INFO:teuthology.orchestra.run.smithi118.stdout:3795 write 0x26794d6 thru 0x267b736 (0x2261 bytes) 2024-09-13T07:02:58.383 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:58.382+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:58.394 INFO:teuthology.orchestra.run.smithi118.stdout:3797 write 0xabcb654 thru 0xabd4e70 (0x981d bytes) 2024-09-13T07:02:58.400 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:58.400 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:58.402 INFO:teuthology.orchestra.run.smithi118.stdout:3798 read 0x9831c2 thru 0x988204 (0x5043 bytes) 2024-09-13T07:02:58.403 INFO:teuthology.orchestra.run.smithi118.stdout:3799 read 0x70abf6f thru 0x70b4d21 (0x8db3 bytes) 2024-09-13T07:02:58.403 INFO:teuthology.orchestra.run.smithi118.stdout:3801 read 0x9d289cd thru 0x9d2c463 (0x3a97 bytes) 2024-09-13T07:02:58.403 INFO:teuthology.orchestra.run.smithi118.stdout:3802 punch from 0x8cde282 to 0x8ce523d, (0x6fbb bytes) 2024-09-13T07:02:58.411 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:02:58.416 INFO:teuthology.orchestra.run.smithi118.stdout:3804 trunc from 0xabd4e71 to 0x417caf5 2024-09-13T07:02:58.435 INFO:teuthology.orchestra.run.smithi118.stdout:3805 read 0x2f6813c thru 0x2f754ba (0xd37f bytes) 2024-09-13T07:02:58.436 INFO:teuthology.orchestra.run.smithi118.stdout:3806 punch from 0x1f1016f to 0x1f10f22, (0xdb3 bytes) 2024-09-13T07:02:58.436 INFO:teuthology.orchestra.run.smithi118.stdout:3807 writesame 0x14e8aa9 thru 0x14f79d3 data_size 0x3(0xef2b bytes) 2024-09-13T07:02:58.471 INFO:teuthology.orchestra.run.smithi118.stdout:3809 write 0x8f3d9bd thru 0x8f42d38 (0x537c bytes) 2024-09-13T07:02:58.484 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:58.480+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:58.497 INFO:teuthology.orchestra.run.smithi118.stdout:3810 write 0xd65a3fc thru 0xd66286c (0x8471 bytes) 2024-09-13T07:02:58.506 INFO:teuthology.orchestra.run.smithi118.stdout:3812 read 0x3009ba2 thru 0x3012712 (0x8b71 bytes) 2024-09-13T07:02:58.506 INFO:teuthology.orchestra.run.smithi118.stdout:3813 read 0x23180cd thru 0x23212d3 (0x9207 bytes) 2024-09-13T07:02:58.506 INFO:teuthology.orchestra.run.smithi118.stdout:3815 read 0x72ab40a thru 0x72b75ab (0xc1a2 bytes) 2024-09-13T07:02:58.518 INFO:teuthology.orchestra.run.smithi118.stdout:3816 write 0xd6d1dd7 thru 0xd6d2ff7 (0x1221 bytes) 2024-09-13T07:02:58.524 INFO:teuthology.orchestra.run.smithi118.stdout:3817 write 0x829284e thru 0x82992ad (0x6a60 bytes) 2024-09-13T07:02:58.528 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:58.528 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:58.530 INFO:teuthology.orchestra.run.smithi118.stdout:3819 read 0x9b053d7 thru 0x9b0a6c6 (0x52f0 bytes) 2024-09-13T07:02:58.539 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:02:58.543 INFO:teuthology.orchestra.run.smithi118.stdout:3820 write 0xda5886b thru 0xda5929e (0xa34 bytes) 2024-09-13T07:02:58.551 INFO:teuthology.orchestra.run.smithi118.stdout:3822 write 0x1afd9fc thru 0x1b086c4 (0xacc9 bytes) 2024-09-13T07:02:58.560 INFO:teuthology.orchestra.run.smithi118.stdout:3823 read 0x7ff64c thru 0x801848 (0x21fd bytes) 2024-09-13T07:02:58.561 INFO:teuthology.orchestra.run.smithi118.stdout:3825 writesame 0x334a7d thru 0x335cb2 data_size 0x15(0x1236 bytes) 2024-09-13T07:02:58.585 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:58.582+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:58.591 INFO:teuthology.orchestra.run.smithi118.stdout:3827 write 0x4385540 thru 0x438fa0e (0xa4cf bytes) 2024-09-13T07:02:58.591 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:02:58.597 INFO:teuthology.orchestra.run.smithi118.stdout:3829 write 0x895552b thru 0x89561c0 (0xc96 bytes) 2024-09-13T07:02:58.603 INFO:teuthology.orchestra.run.smithi118.stdout:3831 punch from 0x3df8533 to 0x3dfeb0c, (0x65d9 bytes) 2024-09-13T07:02:58.603 INFO:teuthology.orchestra.run.smithi118.stdout:3832 punch from 0xc8eb75e to 0xc8f231d, (0x6bbf bytes) 2024-09-13T07:02:58.604 INFO:teuthology.orchestra.run.smithi118.stdout:3833 trunc from 0xda5929f to 0xd8eeaa0 2024-09-13T07:02:58.616 INFO:teuthology.orchestra.run.smithi118.stdout:3834 writesame 0x6d921fa thru 0x6d9f152 data_size 0x1(0xcf59 bytes) 2024-09-13T07:02:58.661 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:58.661 INFO:teuthology.orchestra.run.smithi118.stdout:3838 write 0xa265cf5 thru 0xa26d7f2 (0x7afe bytes) 2024-09-13T07:02:58.663 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:58.663 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:58.668 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:58.669 INFO:teuthology.orchestra.run.smithi118.stdout:3842 write 0x76530c5 thru 0x7658c4f (0x5b8b bytes) 2024-09-13T07:02:58.673 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:02:58.674 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:58.675 INFO:teuthology.orchestra.run.smithi118.stdout:3844 read 0x3032199 thru 0x3034f39 (0x2da1 bytes) 2024-09-13T07:02:58.675 INFO:teuthology.orchestra.run.smithi118.stdout:3846 punch from 0xace93ec to 0xaced09f, (0x3cb3 bytes) 2024-09-13T07:02:58.675 INFO:teuthology.orchestra.run.smithi118.stdout:3847 read 0x271e4ea thru 0x27258a0 (0x73b7 bytes) 2024-09-13T07:02:58.675 INFO:teuthology.orchestra.run.smithi118.stdout:3848 read 0x13e9e50 thru 0x13f01e3 (0x6394 bytes) 2024-09-13T07:02:58.676 INFO:teuthology.orchestra.run.smithi118.stdout:3850 read 0x4f43d2b thru 0x4f512fc (0xd5d2 bytes) 2024-09-13T07:02:58.685 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:02:58.682+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:02:58.693 INFO:teuthology.orchestra.run.smithi118.stdout:3851 trunc from 0xd8eeaa0 to 0x4a3e9aa 2024-09-13T07:02:58.713 INFO:teuthology.orchestra.run.smithi118.stdout:3854 write 0x2581503 thru 0x258b280 (0x9d7e bytes) 2024-09-13T07:02:58.715 INFO:teuthology.orchestra.run.smithi118.stdout:3855 read 0x1594153 thru 0x15a15d0 (0xd47e bytes) 2024-09-13T07:02:58.737 INFO:teuthology.orchestra.run.smithi118.stdout:3856 writesame 0xaa22a8f thru 0xaa2f775 data_size 0xf(0xcce7 bytes) 2024-09-13T07:02:58.737 INFO:teuthology.orchestra.run.smithi118.stdout:3859 read 0x2b0bf8b thru 0x2b18262 (0xc2d8 bytes) 2024-09-13T07:02:58.737 INFO:teuthology.orchestra.run.smithi118.stdout:3860 writesame 0x8efc8d0 thru 0x8f02323 data_size 0x52(0x5a54 bytes) 2024-09-13T07:02:58.745 INFO:teuthology.orchestra.run.smithi118.stdout:3862 punch from 0x9ffee61 to 0xa002167, (0x3306 bytes) 2024-09-13T07:02:58.750 INFO:teuthology.orchestra.run.smithi118.stdout:3864 trunc from 0xaa2f776 to 0xdf9aef3 2024-09-13T07:02:58.780 INFO:teuthology.orchestra.run.smithi118.stdout:3866 trunc from 0xdf9aef3 to 0x28b1cee 2024-09-13T07:02:58.785 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:58.785 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:58.786 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:58.784+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:58.796 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:02:58.813 INFO:teuthology.orchestra.run.smithi118.stdout:3867 write 0x328b50c thru 0x3296e3d (0xb932 bytes) 2024-09-13T07:02:58.847 INFO:teuthology.orchestra.run.smithi118.stdout:3868 write 0xd8dec6c thru 0xd8e335d (0x46f2 bytes) 2024-09-13T07:02:58.855 INFO:teuthology.orchestra.run.smithi118.stdout:3869 write 0x64b6743 thru 0x64c44ae (0xdd6c bytes) 2024-09-13T07:02:58.862 INFO:teuthology.orchestra.run.smithi118.stdout:3870 read 0x8dbc26d thru 0x8dbc790 (0x524 bytes) 2024-09-13T07:02:58.862 INFO:teuthology.orchestra.run.smithi118.stdout:3871 punch from 0xc1d2380 to 0xc1da52c, (0x81ac bytes) 2024-09-13T07:02:58.867 INFO:teuthology.orchestra.run.smithi118.stdout:3872 trunc from 0xd8e335e to 0xac0d206 2024-09-13T07:02:58.886 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:58.886 INFO:teuthology.orchestra.run.smithi118.stdout:3875 read 0x8d03267 thru 0x8d10ba6 (0xd940 bytes) 2024-09-13T07:02:58.886 INFO:teuthology.orchestra.run.smithi118.stdout:3876 read 0x60e8b7b thru 0x60e921f (0x6a5 bytes) 2024-09-13T07:02:58.886 INFO:teuthology.orchestra.run.smithi118.stdout:3877 writesame 0x567baa3 thru 0x56856ef data_size 0x1(0x9c4d bytes) 2024-09-13T07:02:58.887 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:02:58.886+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:02:58.911 INFO:teuthology.orchestra.run.smithi118.stdout:3878 punch from 0x611e0ee to 0x611e556, (0x468 bytes) 2024-09-13T07:02:58.911 INFO:teuthology.orchestra.run.smithi118.stdout:3879 read 0x4e7ace9 thru 0x4e82b50 (0x7e68 bytes) 2024-09-13T07:02:58.918 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:02:58.926 INFO:teuthology.orchestra.run.smithi118.stdout:3880 writesame 0xd8856b8 thru 0xd8857c1 data_size 0x7(0x10a bytes) 2024-09-13T07:02:58.926 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:02:58.928 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:58.928 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:58.928 INFO:teuthology.orchestra.run.smithi118.stdout:3882 trunc from 0xd8857c2 to 0xe7cbe5d 2024-09-13T07:02:58.937 INFO:teuthology.orchestra.run.smithi118.stdout:3884 read 0xccf6d13 thru 0xccfa735 (0x3a23 bytes) 2024-09-13T07:02:58.937 INFO:teuthology.orchestra.run.smithi118.stdout:3887 write 0x86f6a54 thru 0x86fab19 (0x40c6 bytes) 2024-09-13T07:02:58.941 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:02:58.942 INFO:teuthology.orchestra.run.smithi118.stdout:3888 clone 38 order 23 su 1048576 sc 4 2024-09-13T07:02:58.988 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:58.986+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:59.042 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:02:59.043 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:02:59.053 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:02:59.056 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:02:59.086 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:02:59.089 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:02:59.084+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:02:59.120 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:02:59.140 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:02:59.161 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:02:59.163 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:59.164 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:59.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:59.164 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:59.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.164 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:02:59.164 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:59.164 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 93.106208975000001, 2024-09-13T07:02:59.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:02:59.164 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.164 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:59.164 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:59.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:02:59.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:02:59.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:59.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:02:59.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:02:59.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:02:59.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:02:59.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:02:59.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:02:59.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:02:59.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:02:59.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:02:59.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:02:59.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:59.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:02:59.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:02:59.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:02:59.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:02:59.169 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.169 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.169 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.170 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "MOSDPGPush(3.9 100/99 [PushOp(3:90c0de62:::rbd_data.1194e1a844f8.0000000000000013:1b, version: 82'103, data_included: [0~49152,65536~163840,245760~81920,344064~196608,557056~49152,622592~16384,655360~49152,720896~98304,835584~49152,901120~16384,933888~49152,999424~196608,1212416~32768], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:90c0de62:::rbd_data.1194e1a844f8.0000000000000013:1b@82'103, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 1b=[]:{1b=[1b]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1245184, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T07:02:59.170 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:46.054720+0000", 2024-09-13T07:02:59.170 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.108080008, 2024-09-13T07:02:59.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0054811130000000001, 2024-09-13T07:02:59.170 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.170 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.170 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.054720+0000", 2024-09-13T07:02:59.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.054720+0000", 2024-09-13T07:02:59.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.054719+0000", 2024-09-13T07:02:59.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:02:59.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.057056+0000", 2024-09-13T07:02:59.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023371949999999998 2024-09-13T07:02:59.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.057100+0000", 2024-09-13T07:02:59.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3791999999999997e-05 2024-09-13T07:02:59.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.057136+0000", 2024-09-13T07:02:59.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.587e-05 2024-09-13T07:02:59.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.057283+0000", 2024-09-13T07:02:59.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000147773 2024-09-13T07:02:59.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.057306+0000", 2024-09-13T07:02:59.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2186000000000001e-05 2024-09-13T07:02:59.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:46.060201+0000", 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028957309999999999 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133453 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1310720~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874059+0000", 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2887413340000009, 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055990880000000003, 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.174 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133453 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874059+0000", 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874059+0000", 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874034+0000", 2024-09-13T07:02:59.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999747 2024-09-13T07:02:59.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874064+0000", 2024-09-13T07:02:59.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0392999999999999e-05 2024-09-13T07:02:59.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874079+0000", 2024-09-13T07:02:59.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.49e-05 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874091+0000", 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1986e-05 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.878899+0000", 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0048082860000000002 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.878984+0000", 2024-09-13T07:02:59.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4950999999999994e-05 2024-09-13T07:02:59.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.879658+0000", 2024-09-13T07:02:59.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00067367399999999997 2024-09-13T07:02:59.178 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.178 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.178 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.178 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133462 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 786432~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.179 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874168+0000", 2024-09-13T07:02:59.179 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2886323389999994, 2024-09-13T07:02:59.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0063862329999999998, 2024-09-13T07:02:59.179 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.179 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.179 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.179 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.179 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.179 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133462 2024-09-13T07:02:59.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.179 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874168+0000", 2024-09-13T07:02:59.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874168+0000", 2024-09-13T07:02:59.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874167+0000", 2024-09-13T07:02:59.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:59.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874172+0000", 2024-09-13T07:02:59.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0119999999999996e-06 2024-09-13T07:02:59.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874193+0000", 2024-09-13T07:02:59.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0936000000000001e-05 2024-09-13T07:02:59.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874204+0000", 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1189000000000001e-05 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.879696+0000", 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0054921500000000003 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.879779+0000", 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2557000000000003e-05 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.880554+0000", 2024-09-13T07:02:59.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00077517799999999996 2024-09-13T07:02:59.183 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.183 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.183 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.183 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133466 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.183 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874214+0000", 2024-09-13T07:02:59.183 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2885860569999998, 2024-09-13T07:02:59.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078938729999999992, 2024-09-13T07:02:59.183 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.184 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.184 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.184 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.184 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.184 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133466 2024-09-13T07:02:59.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.184 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874214+0000", 2024-09-13T07:02:59.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874214+0000", 2024-09-13T07:02:59.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874213+0000", 2024-09-13T07:02:59.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:59.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874219+0000", 2024-09-13T07:02:59.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.254e-06 2024-09-13T07:02:59.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874232+0000", 2024-09-13T07:02:59.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3874e-05 2024-09-13T07:02:59.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874244+0000", 2024-09-13T07:02:59.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1491000000000001e-05 2024-09-13T07:02:59.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.880629+0000", 2024-09-13T07:02:59.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0063846939999999998 2024-09-13T07:02:59.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.880878+0000", 2024-09-13T07:02:59.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000249724 2024-09-13T07:02:59.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882108+0000", 2024-09-13T07:02:59.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001229653 2024-09-13T07:02:59.190 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.191 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.191 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.191 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133467 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.191 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874255+0000", 2024-09-13T07:02:59.191 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2885450180000007, 2024-09-13T07:02:59.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085814259999999996, 2024-09-13T07:02:59.192 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.192 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.192 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.192 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.192 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.193 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133467 2024-09-13T07:02:59.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.193 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874255+0000", 2024-09-13T07:02:59.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874255+0000", 2024-09-13T07:02:59.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874254+0000", 2024-09-13T07:02:59.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:59.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874260+0000", 2024-09-13T07:02:59.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4430000000000002e-06 2024-09-13T07:02:59.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874284+0000", 2024-09-13T07:02:59.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4145e-05 2024-09-13T07:02:59.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874295+0000", 2024-09-13T07:02:59.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0857e-05 2024-09-13T07:02:59.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882136+0000", 2024-09-13T07:02:59.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078415900000000007 2024-09-13T07:02:59.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882218+0000", 2024-09-13T07:02:59.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1866999999999994e-05 2024-09-13T07:02:59.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882837+0000", 2024-09-13T07:02:59.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061833000000000005 2024-09-13T07:02:59.200 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.200 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.201 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.201 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133476 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.201 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:02:59.201 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2884491120000003, 2024-09-13T07:02:59.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0096071349999999993, 2024-09-13T07:02:59.202 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.202 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.202 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.203 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133476 2024-09-13T07:02:59.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.203 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:02:59.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:02:59.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874350+0000", 2024-09-13T07:02:59.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:59.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874356+0000", 2024-09-13T07:02:59.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5999999999999997e-06 2024-09-13T07:02:59.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874369+0000", 2024-09-13T07:02:59.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3125000000000001e-05 2024-09-13T07:02:59.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874380+0000", 2024-09-13T07:02:59.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1396999999999999e-05 2024-09-13T07:02:59.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882884+0000", 2024-09-13T07:02:59.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.008503666 2024-09-13T07:02:59.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883050+0000", 2024-09-13T07:02:59.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000165926 2024-09-13T07:02:59.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883958+0000", 2024-09-13T07:02:59.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00090819200000000005 2024-09-13T07:02:59.208 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.208 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.209 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.209 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133477 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1310720~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.209 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:02:59.209 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2884098930000007, 2024-09-13T07:02:59.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010531547, 2024-09-13T07:02:59.209 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.209 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.209 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.209 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.209 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.209 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133477 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:02:59.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:59.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874394+0000", 2024-09-13T07:02:59.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7829999999999998e-06 2024-09-13T07:02:59.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874407+0000", 2024-09-13T07:02:59.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3050999999999999e-05 2024-09-13T07:02:59.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874418+0000", 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0974e-05 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883998+0000", 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095798310000000005 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884148+0000", 2024-09-13T07:02:59.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014957499999999999 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884922+0000", 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000774127 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133484 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2883267390000004, 2024-09-13T07:02:59.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011150092, 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133484 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874472+0000", 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874479+0000", 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5700000000000004e-06 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874497+0000", 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7385999999999999e-05 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874510+0000", 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3628999999999999e-05 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884968+0000", 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010458053 2024-09-13T07:02:59.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885119+0000", 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000150932 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885624+0000", 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050432500000000002 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133485 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2882750470000008, 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011776926, 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133485 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874524+0000", 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874530+0000", 2024-09-13T07:02:59.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8599999999999998e-06 2024-09-13T07:02:59.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874545+0000", 2024-09-13T07:02:59.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5186e-05 2024-09-13T07:02:59.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874558+0000", 2024-09-13T07:02:59.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3040000000000001e-05 2024-09-13T07:02:59.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885700+0000", 2024-09-13T07:02:59.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011141514999999999 2024-09-13T07:02:59.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885859+0000", 2024-09-13T07:02:59.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015918599999999999 2024-09-13T07:02:59.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886302+0000", 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044316899999999998 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133494 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2881392500000004, 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012188055999999999, 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.222 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133494 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874660+0000", 2024-09-13T07:02:59.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:59.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874666+0000", 2024-09-13T07:02:59.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7300000000000002e-06 2024-09-13T07:02:59.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874701+0000", 2024-09-13T07:02:59.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5095999999999997e-05 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874714+0000", 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2833000000000001e-05 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886333+0000", 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011619078 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886405+0000", 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2525000000000001e-05 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886849+0000", 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000443697 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133498 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:02:59.226 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2880744879999995, 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012651943000000001, 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133498 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874725+0000", 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874731+0000", 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0159999999999997e-06 2024-09-13T07:02:59.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874788+0000", 2024-09-13T07:02:59.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7021000000000002e-05 2024-09-13T07:02:59.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874797+0000", 2024-09-13T07:02:59.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3430000000000002e-06 2024-09-13T07:02:59.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886880+0000", 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012083287 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886959+0000", 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8582000000000001e-05 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887378+0000", 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041888400000000001 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2879926180000005, 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:02:59.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:02:59.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:02:59.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2879432299999998, 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.235 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.236 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.236 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.236 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:02:59.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.236 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:02:59.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:02:59.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:02:59.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:02:59.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:02:59.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:02:59.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:02:59.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.239 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.240 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:02:59.240 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2879205230000004, 2024-09-13T07:02:59.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:02:59.240 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.240 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.240 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.240 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.240 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.240 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:02:59.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.240 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:02:59.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:02:59.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:02:59.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:02:59.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:02:59.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2878763299999996, 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.244 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.245 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.245 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:02:59.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.245 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:02:59.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:02:59.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:02:59.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:02:59.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:02:59.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:02:59.248 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.248 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.248 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.248 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133516 3.0 3:07bbab57:::rbd_data.11946d7ee8e1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2770411490000004, 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010861226999999999, 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133516 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885757+0000", 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886801+0000", 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010439550000000001 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886827+0000", 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5803e-05 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886839+0000", 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2136e-05 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886894+0000", 2024-09-13T07:02:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4434000000000001e-05 2024-09-13T07:02:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887043+0000", 2024-09-13T07:02:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014903200000000001 2024-09-13T07:02:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:02:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887376+0000", 2024-09-13T07:02:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00033327399999999999 2024-09-13T07:02:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.894299+0000", 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069232620000000003 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896556+0000", 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022571309999999999 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896599+0000", 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2620999999999997e-05 2024-09-13T07:02:59.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896620+0000", 2024-09-13T07:02:59.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1367999999999999e-05 2024-09-13T07:02:59.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.254 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.254 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133521 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 2097152~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:02:59.254 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.886882+0000", 2024-09-13T07:02:59.254 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2759182829999993, 2024-09-13T07:02:59.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00551499, 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133521 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886882+0000", 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886882+0000", 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886882+0000", 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886884+0000", 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5100000000000001e-06 2024-09-13T07:02:59.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886889+0000", 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1279999999999999e-06 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886893+0000", 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2560000000000004e-06 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.890804+0000", 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0039109669999999996 2024-09-13T07:02:59.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.891901+0000", 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010970120000000001 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.892397+0000", 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049547800000000004 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2568972410000008, 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:02:59.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:02:59.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:02:59.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:02:59.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:02:59.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:02:59.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:02:59.263 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.263 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.263 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.263 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133627 3.1f e104/101 3:fe6b0ce8:::rbd_data.11946d7ee8e1.000000000000000d:head v 104'181, mlcod=0'0)", 2024-09-13T07:02:59.263 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.921853+0000", 2024-09-13T07:02:59.263 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2409468619999995, 2024-09-13T07:02:59.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055765900000000002, 2024-09-13T07:02:59.263 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.263 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:02:59.263 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.921853+0000", 2024-09-13T07:02:59.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.921853+0000", 2024-09-13T07:02:59.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.921853+0000", 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.924842+0000", 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029892650000000001 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.924852+0000", 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3379999999999994e-06 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.924857+0000", 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9960000000000001e-06 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.924902+0000", 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5126000000000001e-05 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.924924+0000", 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1807999999999999e-05 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:02:59.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.927392+0000", 2024-09-13T07:02:59.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024684160000000002 2024-09-13T07:02:59.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.927430+0000", 2024-09-13T07:02:59.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8021000000000001e-05 2024-09-13T07:02:59.267 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.267 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.267 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.267 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.267 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.268 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:59.269 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:02:59.270 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:02:59.270 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:02:59.270 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:02:59.271 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:02:59.271 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:02:59.271 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:02:59.271 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:02:59.271 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:02:59.271 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:02:59.271 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:02:59.271 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:02:59.272 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:59.266+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:59.370 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:02:59.368+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:02:59.381 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:59.381 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:02:59.381 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:59.391 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:02:59.471 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:59.468+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:59.544 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:59.544 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:02:59.544 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:02:59.544 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:02:59.544 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:59.554 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:02:59.572 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:59.568+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:59.657 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:02:59.658 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:02:59.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:02:59.658 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:02:59.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.658 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.658 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:59.658 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.536696049, 2024-09-13T07:02:59.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:02:59.658 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.658 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.658 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.658 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.659 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.659 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:02:59.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.659 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:59.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:02:59.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:02:59.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:02:59.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:02:59.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:02:59.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:02:59.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.536675572, 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:59.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:59.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:02:59.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:02:59.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:02:59.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:02:59.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:02:59.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:02:59.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:02:59.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:02:59.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:02:59.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:02:59.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:02:59.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87036 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 6963200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529991506000002, 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042468831999999998, 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.667 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.668 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.668 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.668 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.668 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87036 2024-09-13T07:02:59.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.668 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:59.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127964+0000", 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7799999999999999e-06 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127967+0000", 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9730000000000002e-06 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127971+0000", 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4349999999999999e-06 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170202+0000", 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042230940000000002 2024-09-13T07:02:59.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170293+0000", 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0858000000000003e-05 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170431+0000", 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013821699999999999 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.672 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.672 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:59.672 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529978968999998, 2024-09-13T07:02:59.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:02:59.672 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.672 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.672 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.672 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.672 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.672 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:02:59.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.672 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:59.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:59.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:02:59.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:59.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:02:59.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:02:59.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:02:59.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:02:59.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:02:59.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:02:59.676 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.676 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.676 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.676 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.676 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:59.676 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529966899999998, 2024-09-13T07:02:59.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:02:59.676 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.677 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.677 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.677 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.677 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.677 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:02:59.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.677 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:59.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:59.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:02:59.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:59.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:02:59.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:02:59.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:02:59.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:02:59.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:02:59.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:02:59.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:02:59.680 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.680 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529954457000002, 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:02:59.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529936859000003, 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.685 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.686 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.686 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.686 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.686 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:02:59.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.686 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:59.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:02:59.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:02:59.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:02:59.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:02:59.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:02:59.689 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.689 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.689 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529926156000002, 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.690 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:02:59.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:02:59.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:02:59.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:02:59.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:02:59.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:02:59.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:02:59.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:02:59.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:02:59.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:02:59.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:02:59.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529882223000001, 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.694 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:02:59.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:02:59.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:02:59.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:02:59.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:02:59.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:02:59.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:02:59.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:02:59.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:02:59.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:02:59.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:02:59.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:02:59.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:02:59.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:02:59.698 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.698 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.698 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529862436999998, 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.699 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:59.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:59.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:02:59.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:02:59.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:02:59.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:02:59.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:02:59.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:02:59.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529846687000003, 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:02:59.703 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:02:59.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:02:59.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:02:59.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:02:59.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:02:59.707 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.707 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.707 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529835435000003, 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:59.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:02:59.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:02:59.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:59.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:02:59.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:02:59.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:02:59.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:02:59.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:02:59.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:02:59.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:02:59.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:02:59.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:02:59.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:02:59.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:02:59.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:02:59.712 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.712 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.712 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.712 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.712 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:59.712 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529824836000003, 2024-09-13T07:02:59.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:02:59.713 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.713 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.713 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:02:59.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.713 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:59.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:02:59.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:02:59.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:02:59.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:02:59.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:02:59.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:02:59.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.716 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.717 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.717 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.717 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:59.717 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529815343000003, 2024-09-13T07:02:59.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:02:59.717 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.717 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.717 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.717 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.717 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.717 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:02:59.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.718 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:59.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:02:59.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:02:59.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:59.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:02:59.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:02:59.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:02:59.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:02:59.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:02:59.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:02:59.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:02:59.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:02:59.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:59.721 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529805799000002, 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:02:59.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:02:59.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:02:59.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:02:59.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:02:59.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529773196000001, 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:02:59.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:02:59.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:02:59.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:02:59.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:02:59.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:02:59.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:02:59.730 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.730 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.730 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.730 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.730 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:59.730 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.529760173, 2024-09-13T07:02:59.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:02:59.730 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.730 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.730 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.730 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:02:59.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:02:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:02:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:02:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:02:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:59.735 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.526409188000002, 2024-09-13T07:02:59.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:02:59.735 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.735 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.735 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.735 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.735 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.735 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:02:59.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.735 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:02:59.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:02:59.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:02:59.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:02:59.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:02:59.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:02:59.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:02:59.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:02:59.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:02:59.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:02:59.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.526388910999998, 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.739 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.740 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.740 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.740 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:02:59.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.740 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:59.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:02:59.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:02:59.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:02:59.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:02:59.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:02:59.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:02:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.743 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.52636871, 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:59.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:02:59.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:02:59.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:02:59.747 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:02:59.749 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:02:59.750 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:02:59.750 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:02:59.748+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:02:59.847 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone37 from 0xe7cbe5d (overlap 0xdff790) to 0x599639 2024-09-13T07:02:59.849 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:02:59.848+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:02:59.867 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:59.867 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:02:59.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.868 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:136735 3.1a 3:5db55a3f:::rbd_object_map.11944fbea4a:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-13T07:02:59.868 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:59.867209+0000", 2024-09-13T07:02:59.868 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001801214, 2024-09-13T07:02:59.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018217680000000001, 2024-09-13T07:02:59.868 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:59.868 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for sub ops", 2024-09-13T07:02:59.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:59.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:59.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.868 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 136735 2024-09-13T07:02:59.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.869 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:59.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:59.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867209+0000", 2024-09-13T07:02:59.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:59.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:59.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867209+0000", 2024-09-13T07:02:59.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:59.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:59.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867208+0000", 2024-09-13T07:02:59.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:02:59.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:59.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867213+0000", 2024-09-13T07:02:59.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6469999999999997e-06 2024-09-13T07:02:59.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:59.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867222+0000", 2024-09-13T07:02:59.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1330000000000005e-06 2024-09-13T07:02:59.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867227+0000", 2024-09-13T07:02:59.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2129999999999999e-06 2024-09-13T07:02:59.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:59.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867261+0000", 2024-09-13T07:02:59.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3717999999999998e-05 2024-09-13T07:02:59.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:59.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867354+0000", 2024-09-13T07:02:59.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2405999999999996e-05 2024-09-13T07:02:59.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-13T07:02:59.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867610+0000", 2024-09-13T07:02:59.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025661199999999998 2024-09-13T07:02:59.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:59.872 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:59.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:59.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.872 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:136736 3.1a 3:5db55a3f:::rbd_object_map.11944fbea4a:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-13T07:02:59.873 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:59.867246+0000", 2024-09-13T07:02:59.873 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0017645460000000001, 2024-09-13T07:02:59.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001829619, 2024-09-13T07:02:59.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:59.873 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for sub ops", 2024-09-13T07:02:59.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:59.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:59.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.873 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 136736 2024-09-13T07:02:59.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.873 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:59.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867246+0000", 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867246+0000", 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867245+0000", 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867250+0000", 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6779999999999999e-06 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867258+0000", 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7959999999999996e-06 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867263+0000", 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4310000000000002e-06 2024-09-13T07:02:59.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:02:59.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.868182+0000", 2024-09-13T07:02:59.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00091894000000000003 2024-09-13T07:02:59.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:02:59.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.868313+0000", 2024-09-13T07:02:59.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013092400000000001 2024-09-13T07:02:59.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-13T07:02:59.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.868737+0000", 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00042437399999999999 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:136737 3.1a 3.fc5aadba (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:59.867273+0000", 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0017373320000000001, 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018379749999999999, 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:02:59.877 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 136737 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867273+0000", 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867273+0000", 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:02:59.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867273+0000", 2024-09-13T07:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867276+0000", 2024-09-13T07:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7689999999999998e-06 2024-09-13T07:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867291+0000", 2024-09-13T07:02:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4824000000000001e-05 2024-09-13T07:02:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:02:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:02:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:02:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:59.867296+0000", 2024-09-13T07:02:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3530000000000002e-06 2024-09-13T07:02:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:02:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:02:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-13T07:02:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 3 2024-09-13T07:02:59.881 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:59.881 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:02:59.950 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:02:59.946+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:02:59.991 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:02:59.991 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:02:59.991 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:02:59.991 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:02:59.991 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:02:59.998 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:00.034 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-13T07:03:00.051 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:00.048+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:00.073 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.073 INFO:teuthology.orchestra.run.smithi118.stdout:3890 punch from 0x646646e to 0x6473ff2, (0xdb84 bytes) 2024-09-13T07:03:00.080 INFO:teuthology.orchestra.run.smithi118.stdout:3891 writesame 0x62a288d thru 0x62a4438 data_size 0x2e(0x1bac bytes) 2024-09-13T07:03:00.090 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.090 INFO:teuthology.orchestra.run.smithi118.stdout:3895 punch from 0xb2b793f to 0xb2be5b4, (0x6c75 bytes) 2024-09-13T07:03:00.090 INFO:teuthology.orchestra.run.smithi118.stdout:3897 read 0x674a2fc thru 0x675a2c4 (0xffc9 bytes) 2024-09-13T07:03:00.091 INFO:teuthology.orchestra.run.smithi118.stdout:3898 read 0xd1b1cb4 thru 0xd1bcc9e (0xafeb bytes) 2024-09-13T07:03:00.091 INFO:teuthology.orchestra.run.smithi118.stdout:3899 punch from 0x15d123e to 0x15d9efc, (0x8cbe bytes) 2024-09-13T07:03:00.091 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.111 INFO:teuthology.orchestra.run.smithi118.stdout:3903 trunc from 0xe7cbe5d to 0x3d2cb12 2024-09-13T07:03:00.111 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509494069, 2024-09-13T07:03:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:03:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:03:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:03:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:03:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:03:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:03:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:03:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:03:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509462967999999, 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:03:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:03:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:03:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:03:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509440654, 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:03:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:03:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:03:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509417623000001, 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:03:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:03:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509393073, 2024-09-13T07:03:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:03:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:03:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:03:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:03:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:03:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:03:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509357364, 2024-09-13T07:03:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:03:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:03:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:03:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:03:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:03:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:03:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:03:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:03:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:03:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:03:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:03:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509332685, 2024-09-13T07:03:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:03:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:03:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:03:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:03:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:03:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:03:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:03:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:03:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:03:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:03:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:03:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509299411000001, 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:03:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:03:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:03:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:03:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509272622999999, 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:03:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509248069, 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:03:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:03:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:03:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:03:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:03:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:03:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509218300000001, 2024-09-13T07:03:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:03:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:03:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:03:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:03:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:03:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:03:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:03:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:03:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509158457, 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:03:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:03:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:03:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:03:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:03:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:03:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:03:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:03:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.173 INFO:teuthology.orchestra.run.smithi118.stdout:3905 punch from 0x21ade56 to 0x21aed68, (0xf12 bytes) 2024-09-13T07:03:00.173 INFO:teuthology.orchestra.run.smithi118.stdout:3906 write 0x7775f65 thru 0x777cf3a (0x6fd6 bytes) 2024-09-13T07:03:00.173 INFO:teuthology.orchestra.run.smithi118.stdout:3907 read 0xdddefa thru 0xde4d25 (0x6e2c bytes) 2024-09-13T07:03:00.173 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.173 INFO:teuthology.orchestra.run.smithi118.stdout:3909 read 0x2d86bb7 thru 0x2d88440 (0x188a bytes) 2024-09-13T07:03:00.173 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.174 INFO:teuthology.orchestra.run.smithi118.stdout:3912 writesame 0x1e69d0d thru 0x1e72a0a data_size 0x1(0x8cfe bytes) 2024-09-13T07:03:00.174 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:00.168+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:03:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:03:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509091607, 2024-09-13T07:03:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:03:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:03:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:03:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:03:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:03:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:03:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:03:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:03:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:03:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:03:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:03:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509059105, 2024-09-13T07:03:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:03:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:03:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:03:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:03:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.509024483999999, 2024-09-13T07:03:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:03:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:03:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:03:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:03:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:03:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:03:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:03:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:03:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:03:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:03:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:03:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:03:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:03:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:03:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:03:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:03:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:03:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.508974253, 2024-09-13T07:03:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:03:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:03:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.508926067000001, 2024-09-13T07:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:03:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:03:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:03:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:03:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:03:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:03:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:03:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:03:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:03:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.508899429, 2024-09-13T07:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:03:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:03:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:03:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:03:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:03:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:03:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:03:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:03:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:03:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.508852594, 2024-09-13T07:03:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:03:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:03:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:03:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:03:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:03:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:03:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:03:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:03:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:03:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.505498177, 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:03:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:03:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:03:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:03:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:03:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:03:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:03:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:03:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:03:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:03:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.214 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:00.215 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:03:00.215 INFO:teuthology.orchestra.run.smithi118.stdout:3913 write 0x186cd91 thru 0x1875a40 (0x8cb0 bytes) 2024-09-13T07:03:00.255 INFO:teuthology.orchestra.run.smithi118.stdout:3914 write 0xac51564 thru 0xac5db87 (0xc624 bytes) 2024-09-13T07:03:00.262 INFO:teuthology.orchestra.run.smithi118.stdout:3916 write 0x5d22f3 thru 0x5e00b5 (0xddc3 bytes) 2024-09-13T07:03:00.274 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:00.272+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:00.282 INFO:teuthology.orchestra.run.smithi118.stdout:3917 write 0x54cf966 thru 0x54d1b14 (0x21af bytes) 2024-09-13T07:03:00.289 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.289 INFO:teuthology.orchestra.run.smithi118.stdout:3919 writesame 0x1053742 thru 0x105e4fa data_size 0xd(0xadb9 bytes) 2024-09-13T07:03:00.294 INFO:teuthology.orchestra.run.smithi118.stdout:3920 punch from 0x9795600 to 0x979596c, (0x36c bytes) 2024-09-13T07:03:00.306 INFO:teuthology.orchestra.run.smithi118.stdout:3921 write 0xaf8e472 thru 0xaf918c5 (0x3454 bytes) 2024-09-13T07:03:00.312 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.312 INFO:teuthology.orchestra.run.smithi118.stdout:3923 punch from 0x1598a2 to 0x15cf12, (0x3670 bytes) 2024-09-13T07:03:00.327 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:00.327 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:00.330 INFO:teuthology.orchestra.run.smithi118.stdout:3925 trunc from 0xaf918c6 to 0x2b08da6 2024-09-13T07:03:00.338 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:03:00.356 INFO:teuthology.orchestra.run.smithi118.stdout:3926 punch from 0xc5444a to 0xc60a9a, (0xc650 bytes) 2024-09-13T07:03:00.356 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.357 INFO:teuthology.orchestra.run.smithi118.stdout:3928 read 0x22663 thru 0x25889 (0x3227 bytes) 2024-09-13T07:03:00.357 INFO:teuthology.orchestra.run.smithi118.stdout:3929 write 0x1b3da6 thru 0x1b957b (0x57d6 bytes) 2024-09-13T07:03:00.360 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.374 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:00.374+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:00.375 INFO:teuthology.orchestra.run.smithi118.stdout:3931 write 0x45f5164 thru 0x45fb9bc (0x6859 bytes) 2024-09-13T07:03:00.395 INFO:teuthology.orchestra.run.smithi118.stdout:3932 trunc from 0x45fb9bd to 0xb7b83a0 2024-09-13T07:03:00.407 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.407 INFO:teuthology.orchestra.run.smithi118.stdout:3935 read 0x8e59e96 thru 0x8e61691 (0x77fc bytes) 2024-09-13T07:03:00.407 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.408 INFO:teuthology.orchestra.run.smithi118.stdout:3937 write 0x6b0ad15 thru 0x6b161ab (0xb497 bytes) 2024-09-13T07:03:00.413 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.414 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.414 INFO:teuthology.orchestra.run.smithi118.stdout:3940 read 0x6240d63 thru 0x6244355 (0x35f3 bytes) 2024-09-13T07:03:00.425 INFO:teuthology.orchestra.run.smithi118.stdout:3941 trunc from 0xb7b83a0 to 0x602ee79 2024-09-13T07:03:00.441 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:00.442 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:00.442 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:00.442 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:00.442 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:00.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:03:00.462 INFO:teuthology.orchestra.run.smithi118.stdout:3943 writesame 0x863d894 thru 0x863dee9 data_size 0x1(0x656 bytes) 2024-09-13T07:03:00.462 INFO:teuthology.orchestra.run.smithi118.stdout:3945 punch from 0x25fb595 to 0x2603df6, (0x8861 bytes) 2024-09-13T07:03:00.462 INFO:teuthology.orchestra.run.smithi118.stdout:3951 write 0x6d17094 thru 0x6d1dfe1 (0x6f4e bytes) 2024-09-13T07:03:00.467 INFO:teuthology.orchestra.run.smithi118.stdout:3952 read 0x70ec2b2 thru 0x70f7d8f (0xbade bytes) 2024-09-13T07:03:00.495 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:00.494+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:00.497 INFO:teuthology.orchestra.run.smithi118.stdout:3953 writesame 0xed6c6ba thru 0xed74ba4 data_size 0x1(0x84eb bytes) 2024-09-13T07:03:00.497 INFO:teuthology.orchestra.run.smithi118.stdout:3954 read 0x75f21c thru 0x76a703 (0xb4e8 bytes) 2024-09-13T07:03:00.498 INFO:teuthology.orchestra.run.smithi118.stdout:3956 punch from 0x59af7f5 to 0x59b2414, (0x2c1f bytes) 2024-09-13T07:03:00.498 INFO:teuthology.orchestra.run.smithi118.stdout:3957 read 0x5691d06 thru 0x569a431 (0x872c bytes) 2024-09-13T07:03:00.511 INFO:teuthology.orchestra.run.smithi118.stdout:3960 trunc from 0xed74ba5 to 0x8b0cf44 2024-09-13T07:03:00.523 INFO:teuthology.orchestra.run.smithi118.stdout:3962 read 0x4d20a54 thru 0x4d2444d (0x39fa bytes) 2024-09-13T07:03:00.532 INFO:teuthology.orchestra.run.smithi118.stdout:3963 trunc from 0x8b0cf44 to 0x485344d 2024-09-13T07:03:00.557 INFO:teuthology.orchestra.run.smithi118.stdout:3964 read 0x58d1b7 thru 0x59ca80 (0xf8ca bytes) 2024-09-13T07:03:00.559 INFO:teuthology.orchestra.run.smithi118.stdout:3965 read 0x8740fb thru 0x8784f2 (0x43f8 bytes) 2024-09-13T07:03:00.575 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:00.572+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:00.577 INFO:teuthology.orchestra.run.smithi118.stdout:3966 writesame 0x5dde4db thru 0x5dec124 data_size 0x75(0xdc4a bytes) 2024-09-13T07:03:00.580 INFO:teuthology.orchestra.run.smithi118.stdout:3967 read 0x29bc2f4 thru 0x29bdaee (0x17fb bytes) 2024-09-13T07:03:00.581 INFO:teuthology.orchestra.run.smithi118.stdout:3968 punch from 0x2fd406d to 0x2fe26ae, (0xe641 bytes) 2024-09-13T07:03:00.581 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:00.581 INFO:teuthology.orchestra.run.smithi118.stdout:3970 punch from 0x41e341a to 0x41f2a38, (0xf61e bytes) 2024-09-13T07:03:00.581 INFO:teuthology.orchestra.run.smithi118.stdout:3972 read 0x3f6bfd2 thru 0x3f6cfbf (0xfee bytes) 2024-09-13T07:03:00.585 INFO:teuthology.orchestra.run.smithi118.stdout:3973 trunc from 0x5dec125 to 0x4544cfb 2024-09-13T07:03:00.600 INFO:teuthology.orchestra.run.smithi118.stdout:3974 punch from 0x232e9ec to 0x2335b67, (0x717b bytes) 2024-09-13T07:03:00.600 INFO:teuthology.orchestra.run.smithi118.stdout:3976 read 0x17fc027 thru 0x1800165 (0x413f bytes) 2024-09-13T07:03:00.601 INFO:teuthology.orchestra.run.smithi118.stdout:3977 read 0x12e413b thru 0x12e6f92 (0x2e58 bytes) 2024-09-13T07:03:00.602 INFO:teuthology.orchestra.run.smithi118.stdout:3978 read 0x8f6ce thru 0x980e5 (0x8a18 bytes) 2024-09-13T07:03:00.608 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:03:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.487705497999997, 2024-09-13T07:03:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:03:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:03:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:03:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:03:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:03:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:03:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:03:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:03:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:03:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:03:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:03:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:03:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129532 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.991302611, 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89889205500000002, 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129532 2024-09-13T07:03:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614994+0000", 2024-09-13T07:03:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.044e-06 2024-09-13T07:03:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615000+0000", 2024-09-13T07:03:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9229999999999996e-06 2024-09-13T07:03:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615005+0000", 2024-09-13T07:03:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:03:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615060+0000", 2024-09-13T07:03:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4561000000000002e-05 2024-09-13T07:03:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615069+0000", 2024-09-13T07:03:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1540000000000008e-06 2024-09-13T07:03:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513869+0000", 2024-09-13T07:03:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89879977700000002 2024-09-13T07:03:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513883+0000", 2024-09-13T07:03:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4361e-05 2024-09-13T07:03:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129533 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.991207277999999, 2024-09-13T07:03:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.899270177, 2024-09-13T07:03:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129533 2024-09-13T07:03:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615090+0000", 2024-09-13T07:03:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.856e-06 2024-09-13T07:03:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615096+0000", 2024-09-13T07:03:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.703e-06 2024-09-13T07:03:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615102+0000", 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.378e-06 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615140+0000", 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7966999999999997e-05 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615147+0000", 2024-09-13T07:03:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8800000000000002e-06 2024-09-13T07:03:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514342+0000", 2024-09-13T07:03:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919565300000004 2024-09-13T07:03:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514357+0000", 2024-09-13T07:03:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4355000000000001e-05 2024-09-13T07:03:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129534 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.99113438, 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89923080600000005, 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129534 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615162+0000", 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3569999999999998e-06 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615169+0000", 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.455e-06 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615175+0000", 2024-09-13T07:03:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3679999999999998e-06 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615212+0000", 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6733999999999998e-05 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615218+0000", 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7569999999999998e-06 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514383+0000", 2024-09-13T07:03:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916408000000003 2024-09-13T07:03:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514390+0000", 2024-09-13T07:03:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5429999999999996e-06 2024-09-13T07:03:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129535 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.991060964000001, 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919205499999999, 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129535 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615232+0000", 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615236+0000", 2024-09-13T07:03:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615244+0000", 2024-09-13T07:03:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7369999999999995e-06 2024-09-13T07:03:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615250+0000", 2024-09-13T07:03:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2270000000000001e-06 2024-09-13T07:03:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4627000000000002e-05 2024-09-13T07:03:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5089999999999993e-06 2024-09-13T07:03:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514412+0000", 2024-09-13T07:03:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89907890599999996 2024-09-13T07:03:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514425+0000", 2024-09-13T07:03:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3138e-05 2024-09-13T07:03:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129536 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.991027487, 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89918503500000002, 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129536 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615268+0000", 2024-09-13T07:03:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8370000000000001e-06 2024-09-13T07:03:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615288+0000", 2024-09-13T07:03:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9945999999999998e-05 2024-09-13T07:03:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615294+0000", 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3340000000000001e-06 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615374+0000", 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9988999999999996e-05 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615381+0000", 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514444+0000", 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89906347499999995 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514451+0000", 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7630000000000003e-06 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129537 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.990978934999999, 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916833399999996, 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129537 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615314+0000", 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615317+0000", 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1410000000000001e-06 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3589999999999996e-06 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615329+0000", 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615408+0000", 2024-09-13T07:03:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8678999999999997e-05 2024-09-13T07:03:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615414+0000", 2024-09-13T07:03:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:03:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514471+0000", 2024-09-13T07:03:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905719200000001 2024-09-13T07:03:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514483+0000", 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1691999999999999e-05 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129538 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.990960396, 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89917603999999995, 2024-09-13T07:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129538 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615335+0000", 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4090000000000001e-06 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615346+0000", 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0866000000000001e-05 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615424+0000", 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7628999999999998e-05 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615459+0000", 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5046999999999998e-05 2024-09-13T07:03:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615466+0000", 2024-09-13T07:03:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7610000000000002e-06 2024-09-13T07:03:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514503+0000", 2024-09-13T07:03:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89903750299999996 2024-09-13T07:03:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514509+0000", 2024-09-13T07:03:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2140000000000001e-06 2024-09-13T07:03:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129539 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.990815645, 2024-09-13T07:03:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905735799999997, 2024-09-13T07:03:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129539 2024-09-13T07:03:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615477+0000", 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615481+0000", 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6490000000000001e-06 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615488+0000", 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4359999999999999e-06 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615493+0000", 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3120000000000001e-06 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615529+0000", 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6468999999999999e-05 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615536+0000", 2024-09-13T07:03:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9789999999999996e-06 2024-09-13T07:03:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514529+0000", 2024-09-13T07:03:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89899276400000006 2024-09-13T07:03:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514535+0000", 2024-09-13T07:03:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2920000000000001e-06 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129540 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.990744651, 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89901438700000003, 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129540 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615548+0000", 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615552+0000", 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8349999999999997e-06 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615559+0000", 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615564+0000", 2024-09-13T07:03:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2360000000000004e-06 2024-09-13T07:03:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615600+0000", 2024-09-13T07:03:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6637000000000002e-05 2024-09-13T07:03:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615607+0000", 2024-09-13T07:03:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.669e-06 2024-09-13T07:03:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514557+0000", 2024-09-13T07:03:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89894939399999996 2024-09-13T07:03:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514563+0000", 2024-09-13T07:03:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8560000000000001e-06 2024-09-13T07:03:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129541 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.990672039, 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89896824600000003, 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129541 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615621+0000", 2024-09-13T07:03:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615625+0000", 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615631+0000", 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3890000000000002e-06 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615636+0000", 2024-09-13T07:03:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3489999999999999e-06 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615671+0000", 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4570000000000003e-05 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615678+0000", 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8510000000000001e-06 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514583+0000", 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89890530599999996 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514590+0000", 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7469999999999999e-06 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129542 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.990598549, 2024-09-13T07:03:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89892092400000001, 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129542 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615698+0000", 2024-09-13T07:03:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.664e-06 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615705+0000", 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615711+0000", 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6470000000000003e-06 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615752+0000", 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0389999999999998e-05 2024-09-13T07:03:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615759+0000", 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9230000000000002e-06 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514609+0000", 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89884984899999998 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514616+0000", 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7299999999999999e-06 2024-09-13T07:03:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129543 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.990519676, 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89886872600000001, 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129543 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615773+0000", 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615777+0000", 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1860000000000002e-06 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615785+0000", 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2710000000000003e-06 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615790+0000", 2024-09-13T07:03:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3619999999999997e-06 2024-09-13T07:03:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615828+0000", 2024-09-13T07:03:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8059999999999998e-05 2024-09-13T07:03:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615836+0000", 2024-09-13T07:03:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8809999999999996e-06 2024-09-13T07:03:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514636+0000", 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89880009599999999 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514643+0000", 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129544 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.990444154, 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89881970799999999, 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129544 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615853+0000", 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8600000000000003e-06 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615860+0000", 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1709999999999996e-06 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615867+0000", 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7020000000000004e-06 2024-09-13T07:03:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615905+0000", 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8265e-05 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615913+0000", 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514663+0000", 2024-09-13T07:03:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89874952500000005 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514669+0000", 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129545 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.990365981, 2024-09-13T07:03:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89876792599999999, 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129545 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615927+0000", 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615931+0000", 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8010000000000003e-06 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615939+0000", 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6310000000000003e-06 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615947+0000", 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7090000000000002e-06 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615985+0000", 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8389000000000001e-05 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615993+0000", 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2209999999999997e-06 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514689+0000", 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89869535199999995 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514696+0000", 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8179999999999998e-06 2024-09-13T07:03:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129546 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.990285892999999, 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89872448199999999, 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129546 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616007+0000", 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616011+0000", 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1659999999999996e-06 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616018+0000", 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0829999999999998e-06 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616024+0000", 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8170000000000004e-06 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616063+0000", 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8708000000000003e-05 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616070+0000", 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6299999999999998e-06 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514721+0000", 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89865028800000002 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514732+0000", 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1452e-05 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129547 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.990210565, 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89868529799999997, 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129547 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616086+0000", 2024-09-13T07:03:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.394e-06 2024-09-13T07:03:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616093+0000", 2024-09-13T07:03:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1130000000000003e-06 2024-09-13T07:03:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616100+0000", 2024-09-13T07:03:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1520000000000003e-06 2024-09-13T07:03:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616138+0000", 2024-09-13T07:03:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8084000000000003e-05 2024-09-13T07:03:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616146+0000", 2024-09-13T07:03:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3010000000000001e-06 2024-09-13T07:03:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89861259299999996 2024-09-13T07:03:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514768+0000", 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0175e-05 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129548 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.990126505999999, 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89862847899999998, 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129548 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616166+0000", 2024-09-13T07:03:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616170+0000", 2024-09-13T07:03:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8619999999999996e-06 2024-09-13T07:03:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616177+0000", 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0450000000000003e-06 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616183+0000", 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.139e-06 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616221+0000", 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7320000000000002e-05 2024-09-13T07:03:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616228+0000", 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2980000000000003e-06 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514789+0000", 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89856044599999996 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514796+0000", 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0809999999999996e-06 2024-09-13T07:03:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129549 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.990053043, 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89859381900000002, 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129549 2024-09-13T07:03:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616240+0000", 2024-09-13T07:03:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616244+0000", 2024-09-13T07:03:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4149999999999999e-06 2024-09-13T07:03:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616251+0000", 2024-09-13T07:03:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-13T07:03:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616256+0000", 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3299999999999998e-06 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616302+0000", 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5646999999999999e-05 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616309+0000", 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5530000000000004e-06 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514825+0000", 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89851528400000003 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514834+0000", 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8349999999999999e-06 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129550 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.989968663999999, 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89854296199999995, 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129550 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616324+0000", 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616328+0000", 2024-09-13T07:03:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7479999999999999e-06 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616335+0000", 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9280000000000004e-06 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616342+0000", 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2690000000000002e-06 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616380+0000", 2024-09-13T07:03:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:03:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616387+0000", 2024-09-13T07:03:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.075e-06 2024-09-13T07:03:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514861+0000", 2024-09-13T07:03:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89847450799999995 2024-09-13T07:03:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514868+0000", 2024-09-13T07:03:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6050000000000003e-06 2024-09-13T07:03:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:00.705 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:00.706 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7, 2] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:03:00.706 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:03:00.706 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-13T07:03:00.706 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-13T07:03:00.708 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:00.708 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:00.706+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:00.709 INFO:teuthology.orchestra.run.smithi118.stdout:3979 write 0x766cb96 thru 0x7670ef9 (0x4364 bytes) 2024-09-13T07:03:00.709 INFO:teuthology.orchestra.run.smithi118.stdout:3981 trunc from 0x7670efa to 0xd26f9c5 2024-09-13T07:03:00.709 INFO:teuthology.orchestra.run.smithi118.stdout:3982 write 0x9fa1f88 thru 0x9fa9d86 (0x7dff bytes) 2024-09-13T07:03:00.709 INFO:teuthology.orchestra.run.smithi118.stdout:3983 flatten 2024-09-13T07:03:00.709 INFO:teuthology.orchestra.run.smithi118.stdout:3984 write 0x6c873a4 thru 0x6c8b6f3 (0x4350 bytes) 2024-09-13T07:03:00.709 INFO:teuthology.orchestra.run.smithi118.stdout:3985 punch from 0x7301cc6 to 0x730cea8, (0xb1e2 bytes) 2024-09-13T07:03:00.709 INFO:teuthology.orchestra.run.smithi118.stdout:3986 write 0x9fd5029 thru 0x9fd861c (0x35f4 bytes) 2024-09-13T07:03:00.709 INFO:teuthology.orchestra.run.smithi118.stdout:3987 clone 39 order 19 su 131072 sc 11 2024-09-13T07:03:00.808 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:00.804+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:00.820 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:00.820 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:00.821 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:00.821 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:00.830 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:00.909 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:00.908+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:00.936 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:00.936 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:00.936 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:00.937 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:00.937 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:00.946 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:01.009 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:01.006+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:01.020 INFO:tasks.ceph.mon.a.smithi118.stderr:2024-09-13T07:03:01.016+0000 7fd1ec319700 -1 mon.a@0(leader).osd e112 definitely_dead 0 2024-09-13T07:03:01.055 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:01.055 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:01.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:01.055 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:01.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.055 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:01.055 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 94.994875321999999, 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:01.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:01.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:03:01.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:03:01.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:03:01.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:03:01.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:03:01.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:03:01.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:01.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:03:01.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:03:01.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:03:01.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93979 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.437116009, 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042253801000000001, 2024-09-13T07:03:01.061 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93979 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616683+0000", 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2820000000000001e-06 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616691+0000", 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0779999999999996e-06 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616698+0000", 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658799+0000", 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042100763999999999 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658869+0000", 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0065000000000001e-05 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658934+0000", 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5071000000000001e-05 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:01.065 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.437091444000004, 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:01.069 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.437068476999997, 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.437051298999997, 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:03:01.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:03:01.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:03:01.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:03:01.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:03:01.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:03:01.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:03:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:03:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:03:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:03:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:03:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:03:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.437037433999997, 2024-09-13T07:03:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:03:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.078 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:01.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:03:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:03:01.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:03:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:03:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:03:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:03:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:03:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:03:01.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.437024289, 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.083 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:03:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:03:01.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:03:01.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:03:01.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:03:01.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:03:01.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.437011081999998, 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.087 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:01.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:01.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:03:01.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:03:01.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:01.091 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.436997015000003, 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:03:01.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:03:01.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.436977001000002, 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.436957251000003, 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:03:01.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.436932444999997, 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.104 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.436908180000003, 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:03:01.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.436866447, 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.112 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:03:01.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.436830157999999, 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:03:01.116 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:03:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:03:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:03:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.993726826, 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.993709943999999, 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:03:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:03:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.992659387, 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:03:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:03:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:03:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:03:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:03:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:03:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:03:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:03:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:03:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:03:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:03:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.990984837999999, 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:03:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:03:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:03:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:03:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:03:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:03:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:03:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:03:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:03:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.470402448, 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:03:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:03:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:03:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:03:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.142 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:01.143 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:01.143 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:01.142+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:01.244 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:01.244+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:01.272 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:03:01.294 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:01.294 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:01.294 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:01.294 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:01.304 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:01.344 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:01.342+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:01.405 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:01.405 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:01.406 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:01.406 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:01.406 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:01.415 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:01.426 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:01.445 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:01.442+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:01.447 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:01.475 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:01.495 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:01.519 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:01.523 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:01.523 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:01.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:01.523 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:01.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.523 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:03:01.523 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:01.523 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 95.468950618999997, 2024-09-13T07:03:01.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:03:01.524 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.524 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:01.524 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:01.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:01.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:03:01.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:03:01.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:03:01.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:03:01.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:03:01.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:03:01.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.527 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 63.311069385000003, 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.528 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:03:01.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:03:01.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:03:01.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 63.285299674000001, 2024-09-13T07:03:01.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:03:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:03:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:03:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:03:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:03:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:03:01.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:03:01.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:03:01.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:01.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:03:01.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:03:01.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:01.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:03:01.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:03:01.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:01.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:03:01.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:03:01.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:01.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:03:01.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:03:01.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:03:01.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:03:01.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.539 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.539 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:01.539 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 63.264148507999998, 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:01.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:03:01.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:03:01.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:03:01.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:03:01.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:03:01.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:03:01.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:03:01.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 53.147103745000003, 2024-09-13T07:03:01.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:01.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:01.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:03:01.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:03:01.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:03:01.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:03:01.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:03:01.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:03:01.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:01.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:03:01.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:03:01.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:01.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:03:01.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:03:01.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:01.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:03:01.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94596 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.862127266000002, 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044330268999999999, 2024-09-13T07:03:01.551 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94596 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660995+0000", 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660999+0000", 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2770000000000001e-06 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661007+0000", 2024-09-13T07:03:01.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0029999999999994e-06 2024-09-13T07:03:01.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661012+0000", 2024-09-13T07:03:01.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1279999999999999e-06 2024-09-13T07:03:01.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705253+0000", 2024-09-13T07:03:01.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044241603999999997 2024-09-13T07:03:01.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705293+0000", 2024-09-13T07:03:01.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9684999999999997e-05 2024-09-13T07:03:01.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705326+0000", 2024-09-13T07:03:01.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2932000000000003e-05 2024-09-13T07:03:01.555 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.555 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.555 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.556 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.556 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:01.556 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.862106500000003, 2024-09-13T07:03:01.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:03:01.556 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.556 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.556 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.556 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.556 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.556 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:03:01.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.557 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:01.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:01.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:03:01.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:03:01.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:03:01.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:03:01.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:03:01.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:03:01.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94598 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.861935779, 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044365860999999999, 2024-09-13T07:03:01.560 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.561 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.561 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94598 2024-09-13T07:03:01.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.561 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:03:01.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661034+0000", 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9998465 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661192+0000", 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015813000000000001 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661199+0000", 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5689999999999999e-06 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705452+0000", 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044246273000000003 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705501+0000", 2024-09-13T07:03:01.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8748000000000002e-05 2024-09-13T07:03:01.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705553+0000", 2024-09-13T07:03:01.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2241999999999999e-05 2024-09-13T07:03:01.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.564 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.564 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107060 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1003520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.610413489999999, 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044341754999999997, 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107060 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:03:01.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:03:01.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912709+0000", 2024-09-13T07:03:01.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912712+0000", 2024-09-13T07:03:01.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6000000000000001e-06 2024-09-13T07:03:01.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912717+0000", 2024-09-13T07:03:01.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3689999999999998e-06 2024-09-13T07:03:01.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912722+0000", 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7219999999999999e-06 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956876+0000", 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044154469000000002 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956957+0000", 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1231999999999997e-05 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957051+0000", 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3858999999999997e-05 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.610397362, 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.569 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.570 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:03:01.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.570 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:01.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:03:01.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:03:01.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:03:01.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:03:01.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:03:01.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:03:01.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:03:01.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:03:01.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:03:01.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:03:01.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:03:01.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.610379328, 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.574 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:03:01.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:03:01.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:03:01.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:03:01.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:01.578 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.610363247999999, 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:01.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:03:01.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:03:01.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:03:01.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:03:01.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:03:01.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:03:01.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:03:01.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:03:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:03:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:03:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:03:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:03:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:03:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.610347571999998, 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.583 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:03:01.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.584 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:01.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:01.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:03:01.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.586 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.610324649999999, 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:01.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:01.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:01.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:03:01.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:03:01.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:03:01.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:03:01.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:03:01.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:03:01.590 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.590 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.610308552999999, 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.591 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:03:01.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:03:01.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:03:01.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:03:01.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:03:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:03:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:03:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:03:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:03:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:03:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107067 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.609748536000001, 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04441055, 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107067 2024-09-13T07:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.596 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:01.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:01.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912832+0000", 2024-09-13T07:03:01.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9994574 2024-09-13T07:03:01.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913379+0000", 2024-09-13T07:03:01.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00054719999999999997 2024-09-13T07:03:01.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913386+0000", 2024-09-13T07:03:01.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4549999999999998e-06 2024-09-13T07:03:01.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913393+0000", 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6259999999999997e-06 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957776+0000", 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044383143999999999 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957785+0000", 2024-09-13T07:03:01.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9220000000000003e-06 2024-09-13T07:03:01.599 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.599 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.599 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.599 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107068 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:01.599 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:01.599 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.609724105000002, 2024-09-13T07:03:01.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044410502999999997, 2024-09-13T07:03:01.599 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.599 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:01.599 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.599 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.600 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107068 2024-09-13T07:03:01.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.600 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:01.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:01.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913401+0000", 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913407+0000", 2024-09-13T07:03:01.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7520000000000002e-06 2024-09-13T07:03:01.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913412+0000", 2024-09-13T07:03:01.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8550000000000001e-06 2024-09-13T07:03:01.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957802+0000", 2024-09-13T07:03:01.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044389606999999998 2024-09-13T07:03:01.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957809+0000", 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9160000000000001e-06 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.609706940999999, 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.603 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:01.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.604 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:03:01.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.604 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:01.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.609664165000002, 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.607 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:03:01.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:03:01.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:03:01.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:03:01.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:03:01.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.452646394999999, 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:01.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:03:01.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:03:01.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:03:01.615 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.615 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.615 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.615 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:01.615 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:01.615 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:01.616 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:01.616 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:01.616 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:01.617 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:01.618 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:01.618 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:01.618 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:01.618 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:01.618 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:01.618 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:01.618 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:01.619 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:01.619 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:01.612+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:01.714 INFO:teuthology.orchestra.run.smithi118.stderr:marked in osd.0. 2024-09-13T07:03:01.717 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:01.714+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:01.729 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-13T07:03:01.742 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:01.742 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:01.751 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:03:01.752 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:01.819 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:01.816+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:01.908 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:01.908 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:01.919 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:01.916+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:01.922 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:02.021 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:02.018+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:02.068 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:02.068 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:02.078 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:02.122 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:02.118+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:02.178 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:02.178 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:02.188 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:02.223 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:02.220+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:02.297 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:02.298 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:02.307 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:02.349 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:02.348+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:02.435 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:02.424+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:02.461 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:02.461 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:02.471 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:02.526 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:02.524+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:02.577 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:02.577 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:02.587 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:02.626 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:02.626+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:02.732 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:03:02.732 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:02.728+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:02.750 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:02.750 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:02.760 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:02.829 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:02.826+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:02.910 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:02.910 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:02.920 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:02.929 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:02.926+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:03.028 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:03.028 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:03.030 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:03.030+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:03.036 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:03.131 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:03.130+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:03.142 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:03.142 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:03.152 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:03.232 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:03.228+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:03.262 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:03.262 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:03.272 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:03.333 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:03.332+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:03.390 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:03.390 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:03.400 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:03.434 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:03.430+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:03.510 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:03.510 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:03.520 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:03.535 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:03.532+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:03.619 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:03:03.624 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:03.624 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:03.634 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:03:03.665 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:03.664+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:03.737 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:03.734+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:03.750 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:03.750 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:03.760 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:03:03.766 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:03.788 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:03.808 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:03.829 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:03.838 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:03.836+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:03.852 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:03.873 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:03.898 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:03.915 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:03.915 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:03.915 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:03.915 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:03.915 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:03.920 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:03.922 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:03.922 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:03.922 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:03.922 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:03.922 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:03.922 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:03.922 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:03.922 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:03.925 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:03:03.938 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:03.934+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:04.038 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:04.039 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:04.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:04.039 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:04.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.039 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:03:04.039 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:04.039 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.918077385000004, 2024-09-13T07:03:04.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:03:04.039 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.039 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:04.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.040 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.040 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.040 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:03:04.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.040 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:04.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:03:04.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:03:04.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:03:04.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:03:04.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:03:04.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:03:04.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:03:04.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:03:04.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:03:04.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:03:04.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:03:04.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:03:04.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:04.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:03:04.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:03:04.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:04.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:03:04.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:03:04.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:04.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:03:04.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:03:04.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:04.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:03:04.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:03:04.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:03:04.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:03:04.046 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.046 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.046 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.047 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129532 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.047 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:04.047 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.421674498000002, 2024-09-13T07:03:04.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89889205500000002, 2024-09-13T07:03:04.047 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.047 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.047 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.047 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.047 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.047 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129532 2024-09-13T07:03:04.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.047 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:04.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:04.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614994+0000", 2024-09-13T07:03:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.044e-06 2024-09-13T07:03:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615000+0000", 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9229999999999996e-06 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615005+0000", 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615060+0000", 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4561000000000002e-05 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615069+0000", 2024-09-13T07:03:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1540000000000008e-06 2024-09-13T07:03:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513869+0000", 2024-09-13T07:03:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89879977700000002 2024-09-13T07:03:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513883+0000", 2024-09-13T07:03:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4361e-05 2024-09-13T07:03:04.051 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129533 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.421579165000001, 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.899270177, 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.052 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129533 2024-09-13T07:03:04.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.053 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:04.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:04.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:04.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615090+0000", 2024-09-13T07:03:04.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.856e-06 2024-09-13T07:03:04.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615096+0000", 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.703e-06 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615102+0000", 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.378e-06 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615140+0000", 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7966999999999997e-05 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615147+0000", 2024-09-13T07:03:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8800000000000002e-06 2024-09-13T07:03:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514342+0000", 2024-09-13T07:03:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919565300000004 2024-09-13T07:03:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514357+0000", 2024-09-13T07:03:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4355000000000001e-05 2024-09-13T07:03:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.059 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129534 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.421506267000002, 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89923080600000005, 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129534 2024-09-13T07:03:04.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:04.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615162+0000", 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3569999999999998e-06 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615169+0000", 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.455e-06 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615175+0000", 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3679999999999998e-06 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615212+0000", 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6733999999999998e-05 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615218+0000", 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7569999999999998e-06 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514383+0000", 2024-09-13T07:03:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916408000000003 2024-09-13T07:03:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514390+0000", 2024-09-13T07:03:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5429999999999996e-06 2024-09-13T07:03:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129535 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.421432850999999, 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919205499999999, 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129535 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615232+0000", 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615236+0000", 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615244+0000", 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7369999999999995e-06 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615250+0000", 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2270000000000001e-06 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:04.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4627000000000002e-05 2024-09-13T07:03:04.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:04.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5089999999999993e-06 2024-09-13T07:03:04.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514412+0000", 2024-09-13T07:03:04.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89907890599999996 2024-09-13T07:03:04.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514425+0000", 2024-09-13T07:03:04.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3138e-05 2024-09-13T07:03:04.069 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.069 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.069 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.069 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129536 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.069 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:04.069 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.421399374, 2024-09-13T07:03:04.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89918503500000002, 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129536 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615268+0000", 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8370000000000001e-06 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615288+0000", 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9945999999999998e-05 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615294+0000", 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3340000000000001e-06 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615374+0000", 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9988999999999996e-05 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615381+0000", 2024-09-13T07:03:04.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:03:04.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514444+0000", 2024-09-13T07:03:04.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89906347499999995 2024-09-13T07:03:04.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514451+0000", 2024-09-13T07:03:04.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7630000000000003e-06 2024-09-13T07:03:04.074 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.074 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.074 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.074 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129537 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.074 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:04.074 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.421350822000001, 2024-09-13T07:03:04.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916833399999996, 2024-09-13T07:03:04.074 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.074 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.074 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.075 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129537 2024-09-13T07:03:04.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.075 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:04.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:04.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615314+0000", 2024-09-13T07:03:04.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615317+0000", 2024-09-13T07:03:04.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1410000000000001e-06 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3589999999999996e-06 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615329+0000", 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615408+0000", 2024-09-13T07:03:04.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8678999999999997e-05 2024-09-13T07:03:04.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615414+0000", 2024-09-13T07:03:04.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:03:04.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514471+0000", 2024-09-13T07:03:04.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905719200000001 2024-09-13T07:03:04.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514483+0000", 2024-09-13T07:03:04.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1691999999999999e-05 2024-09-13T07:03:04.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.079 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.079 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129538 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.079 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:04.079 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.421332283000002, 2024-09-13T07:03:04.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89917603999999995, 2024-09-13T07:03:04.080 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.080 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.080 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.080 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.080 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.080 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129538 2024-09-13T07:03:04.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.080 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:04.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:04.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:04.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615335+0000", 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4090000000000001e-06 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615346+0000", 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0866000000000001e-05 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615424+0000", 2024-09-13T07:03:04.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7628999999999998e-05 2024-09-13T07:03:04.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615459+0000", 2024-09-13T07:03:04.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5046999999999998e-05 2024-09-13T07:03:04.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615466+0000", 2024-09-13T07:03:04.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7610000000000002e-06 2024-09-13T07:03:04.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514503+0000", 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89903750299999996 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514509+0000", 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2140000000000001e-06 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.085 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129539 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.085 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:04.085 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.421187532000001, 2024-09-13T07:03:04.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905735799999997, 2024-09-13T07:03:04.085 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.085 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.085 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.085 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.085 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.085 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129539 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615477+0000", 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615481+0000", 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6490000000000001e-06 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615488+0000", 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4359999999999999e-06 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615493+0000", 2024-09-13T07:03:04.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3120000000000001e-06 2024-09-13T07:03:04.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615529+0000", 2024-09-13T07:03:04.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6468999999999999e-05 2024-09-13T07:03:04.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615536+0000", 2024-09-13T07:03:04.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9789999999999996e-06 2024-09-13T07:03:04.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514529+0000", 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89899276400000006 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514535+0000", 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2920000000000001e-06 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.090 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129540 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.090 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:04.090 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.421116538, 2024-09-13T07:03:04.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89901438700000003, 2024-09-13T07:03:04.090 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.090 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.090 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129540 2024-09-13T07:03:04.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.091 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:04.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:04.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615548+0000", 2024-09-13T07:03:04.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:04.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615552+0000", 2024-09-13T07:03:04.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8349999999999997e-06 2024-09-13T07:03:04.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615559+0000", 2024-09-13T07:03:04.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-13T07:03:04.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615564+0000", 2024-09-13T07:03:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2360000000000004e-06 2024-09-13T07:03:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615600+0000", 2024-09-13T07:03:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6637000000000002e-05 2024-09-13T07:03:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615607+0000", 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.669e-06 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514557+0000", 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89894939399999996 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514563+0000", 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8560000000000001e-06 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129541 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.421043925999999, 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89896824600000003, 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129541 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.095 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:04.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:04.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615621+0000", 2024-09-13T07:03:04.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615625+0000", 2024-09-13T07:03:04.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615631+0000", 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3890000000000002e-06 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615636+0000", 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3489999999999999e-06 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615671+0000", 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4570000000000003e-05 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615678+0000", 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8510000000000001e-06 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514583+0000", 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89890530599999996 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514590+0000", 2024-09-13T07:03:04.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7469999999999999e-06 2024-09-13T07:03:04.100 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.100 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.100 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.100 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129542 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.100 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:04.100 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.420970436000001, 2024-09-13T07:03:04.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89892092400000001, 2024-09-13T07:03:04.100 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.100 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.101 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.101 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.101 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129542 2024-09-13T07:03:04.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.101 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:04.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615698+0000", 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.664e-06 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615705+0000", 2024-09-13T07:03:04.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:04.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615711+0000", 2024-09-13T07:03:04.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6470000000000003e-06 2024-09-13T07:03:04.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615752+0000", 2024-09-13T07:03:04.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0389999999999998e-05 2024-09-13T07:03:04.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615759+0000", 2024-09-13T07:03:04.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9230000000000002e-06 2024-09-13T07:03:04.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514609+0000", 2024-09-13T07:03:04.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89884984899999998 2024-09-13T07:03:04.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514616+0000", 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7299999999999999e-06 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129543 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.420891563000001, 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89886872600000001, 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.106 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129543 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615773+0000", 2024-09-13T07:03:04.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:04.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615777+0000", 2024-09-13T07:03:04.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1860000000000002e-06 2024-09-13T07:03:04.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615785+0000", 2024-09-13T07:03:04.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2710000000000003e-06 2024-09-13T07:03:04.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615790+0000", 2024-09-13T07:03:04.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3619999999999997e-06 2024-09-13T07:03:04.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615828+0000", 2024-09-13T07:03:04.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8059999999999998e-05 2024-09-13T07:03:04.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615836+0000", 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8809999999999996e-06 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514636+0000", 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89880009599999999 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514643+0000", 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:04.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129544 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.420816040999998, 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89881970799999999, 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129544 2024-09-13T07:03:04.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.112 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:04.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:04.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615853+0000", 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8600000000000003e-06 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615860+0000", 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1709999999999996e-06 2024-09-13T07:03:04.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615867+0000", 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7020000000000004e-06 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615905+0000", 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8265e-05 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615913+0000", 2024-09-13T07:03:04.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514663+0000", 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89874952500000005 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514669+0000", 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129545 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.420737868, 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89876792599999999, 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129545 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615927+0000", 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615931+0000", 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8010000000000003e-06 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615939+0000", 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6310000000000003e-06 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615947+0000", 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7090000000000002e-06 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615985+0000", 2024-09-13T07:03:04.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8389000000000001e-05 2024-09-13T07:03:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615993+0000", 2024-09-13T07:03:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2209999999999997e-06 2024-09-13T07:03:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514689+0000", 2024-09-13T07:03:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89869535199999995 2024-09-13T07:03:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514696+0000", 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8179999999999998e-06 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129546 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.420657779999999, 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89872448199999999, 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.120 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.121 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.121 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129546 2024-09-13T07:03:04.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.121 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:04.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:04.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616007+0000", 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616011+0000", 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1659999999999996e-06 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616018+0000", 2024-09-13T07:03:04.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0829999999999998e-06 2024-09-13T07:03:04.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616024+0000", 2024-09-13T07:03:04.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8170000000000004e-06 2024-09-13T07:03:04.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616063+0000", 2024-09-13T07:03:04.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8708000000000003e-05 2024-09-13T07:03:04.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616070+0000", 2024-09-13T07:03:04.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6299999999999998e-06 2024-09-13T07:03:04.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514721+0000", 2024-09-13T07:03:04.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89865028800000002 2024-09-13T07:03:04.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514732+0000", 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1452e-05 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129547 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.420582452000001, 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89868529799999997, 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.125 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.126 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129547 2024-09-13T07:03:04.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.126 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:04.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:04.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616086+0000", 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.394e-06 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616093+0000", 2024-09-13T07:03:04.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1130000000000003e-06 2024-09-13T07:03:04.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616100+0000", 2024-09-13T07:03:04.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1520000000000003e-06 2024-09-13T07:03:04.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616138+0000", 2024-09-13T07:03:04.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8084000000000003e-05 2024-09-13T07:03:04.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616146+0000", 2024-09-13T07:03:04.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3010000000000001e-06 2024-09-13T07:03:04.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:04.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89861259299999996 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514768+0000", 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0175e-05 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129548 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.420498392999999, 2024-09-13T07:03:04.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89862847899999998, 2024-09-13T07:03:04.131 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.131 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.131 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129548 2024-09-13T07:03:04.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.131 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:04.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:04.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616166+0000", 2024-09-13T07:03:04.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616170+0000", 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8619999999999996e-06 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616177+0000", 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0450000000000003e-06 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616183+0000", 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.139e-06 2024-09-13T07:03:04.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616221+0000", 2024-09-13T07:03:04.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7320000000000002e-05 2024-09-13T07:03:04.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616228+0000", 2024-09-13T07:03:04.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2980000000000003e-06 2024-09-13T07:03:04.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514789+0000", 2024-09-13T07:03:04.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89856044599999996 2024-09-13T07:03:04.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514796+0000", 2024-09-13T07:03:04.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0809999999999996e-06 2024-09-13T07:03:04.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.135 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.135 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129549 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.135 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:04.136 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.420424929999999, 2024-09-13T07:03:04.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89859381900000002, 2024-09-13T07:03:04.136 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.136 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.136 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129549 2024-09-13T07:03:04.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.136 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616240+0000", 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616244+0000", 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4149999999999999e-06 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616251+0000", 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616256+0000", 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3299999999999998e-06 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616302+0000", 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5646999999999999e-05 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616309+0000", 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5530000000000004e-06 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514825+0000", 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89851528400000003 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514834+0000", 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8349999999999999e-06 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129550 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.420340550999999, 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89854296199999995, 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.140 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129550 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:04.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616324+0000", 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616328+0000", 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7479999999999999e-06 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616335+0000", 2024-09-13T07:03:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9280000000000004e-06 2024-09-13T07:03:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616342+0000", 2024-09-13T07:03:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2690000000000002e-06 2024-09-13T07:03:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616380+0000", 2024-09-13T07:03:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:03:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616387+0000", 2024-09-13T07:03:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.075e-06 2024-09-13T07:03:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514861+0000", 2024-09-13T07:03:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89847450799999995 2024-09-13T07:03:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514868+0000", 2024-09-13T07:03:04.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6050000000000003e-06 2024-09-13T07:03:04.145 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.145 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.145 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.145 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.145 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.145 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:04.147 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:04.147 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:04.146+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:04.247 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:04.244+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:04.302 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:04.303 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:04.313 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:04.348 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:04.346+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:04.427 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:04.427 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:04.427 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:04.427 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:04.427 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:04.438 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:04.449 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:04.448+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:04.519 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-13T07:03:04.542 INFO:teuthology.orchestra.run.smithi118.stdout:3989 writesame 0x4b27272 thru 0x4b3647b data_size 0x45(0xf20a bytes) 2024-09-13T07:03:04.550 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.488922959999996, 2024-09-13T07:03:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:03:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:03:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:03:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:03:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:03:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:03:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:03:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:03:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:03:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:03:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:03:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:03:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:03:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:03:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:03:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:03:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:03:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:03:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:03:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:03:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:03:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:03:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93979 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.931163646999998, 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042253801000000001, 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93979 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616683+0000", 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2820000000000001e-06 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616691+0000", 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0779999999999996e-06 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616698+0000", 2024-09-13T07:03:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658799+0000", 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042100763999999999 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658869+0000", 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0065000000000001e-05 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658934+0000", 2024-09-13T07:03:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5071000000000001e-05 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.931139082000001, 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:03:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:03:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:03:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:03:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.931116115000002, 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:03:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:03:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:03:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:03:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:03:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:03:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:03:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:03:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.931098937000002, 2024-09-13T07:03:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:03:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:03:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:03:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:03:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:03:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.931085072000002, 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:03:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:03:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:03:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:03:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:03:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:03:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:03:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:03:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:03:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:03:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:03:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.931071926999998, 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:03:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:03:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:03:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:03:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.931058720000003, 2024-09-13T07:03:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:03:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.584 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:03:04.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.584 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:04.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:04.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:04.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:03:04.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:03:04.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:03:04.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:03:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:03:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:03:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:03:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.587 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.588 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.588 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.588 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:04.588 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.931044653000001, 2024-09-13T07:03:04.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:03:04.588 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.588 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:04.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:03:04.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:03:04.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.931024639, 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.592 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.593 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:03:04.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.593 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:04.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:03:04.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:03:04.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:03:04.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:03:04.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:03:04.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.597 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.597 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:04.597 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.931004889, 2024-09-13T07:03:04.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:03:04.597 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.597 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.597 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:03:04.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:03:04.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:03:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:03:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:03:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:03:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:03:04.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:03:04.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:03:04.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:03:04.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:03:04.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:03:04.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.930980083000001, 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.601 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.602 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:03:04.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.602 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:04.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:03:04.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:03:04.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:03:04.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:03:04.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:03:04.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.606 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.606 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:04.606 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.930955818000001, 2024-09-13T07:03:04.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:03:04.606 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.606 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.606 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:03:04.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.606 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:04.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:04.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.930914084999998, 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.610 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:03:04.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:03:04.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.614 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.614 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:04.614 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:04.614 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.930877795999997, 2024-09-13T07:03:04.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:03:04.614 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.614 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:04.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:03:04.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:03:04.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:03:04.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:03:04.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:03:04.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:03:04.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:03:04.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:03:04.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.487774464000001, 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:04.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:03:04.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:03:04.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:03:04.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:03:04.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:03:04.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:03:04.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:03:04.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.487757582, 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:03:04.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:04.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:03:04.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:03:04.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:03:04.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:03:04.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:03:04.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:03:04.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:03:04.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:03:04.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:03:04.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:03:04.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.486707025000001, 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:03:04.627 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:03:04.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:03:04.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:03:04.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:03:04.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:03:04.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:03:04.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:03:04.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:03:04.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:03:04.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:03:04.631 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.631 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.631 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.632 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:04.632 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:04.632 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.485032476000001, 2024-09-13T07:03:04.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:03:04.632 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.632 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:04.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.632 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:03:04.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.633 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:04.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:04.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:04.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:04.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:03:04.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:03:04.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:03:04.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:03:04.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:03:04.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:03:04.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:03:04.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.637 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:03:04.637 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:04.637 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.964450085999999, 2024-09-13T07:03:04.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:03:04.637 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:04.637 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:03:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:03:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:03:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:03:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:03:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:03:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:04.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:03:04.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:03:04.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:04.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:03:04.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:03:04.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:04.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:04.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:04.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:03:04.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:03:04.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.642 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:04.642 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:04.642 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:04.642 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:04.546+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:04.643 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:04.645 INFO:teuthology.orchestra.run.smithi118.stdout:3990 write 0x77dedfe thru 0x77e3ad5 (0x4cd8 bytes) 2024-09-13T07:03:04.645 INFO:teuthology.orchestra.run.smithi118.stdout:3991 write 0x4747297 thru 0x474c268 (0x4fd2 bytes) 2024-09-13T07:03:04.645 INFO:teuthology.orchestra.run.smithi118.stdout:3992 writesame 0x6be8540 thru 0x6bf3ce9 data_size 0x1(0xb7aa bytes) 2024-09-13T07:03:04.645 INFO:teuthology.orchestra.run.smithi118.stdout:3993 writesame 0x6925b8a thru 0x69303f9 data_size 0x50(0xa870 bytes) 2024-09-13T07:03:04.645 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:04.645 INFO:teuthology.orchestra.run.smithi118.stdout:3996 trunc from 0xd26f9c5 to 0xaacd496 2024-09-13T07:03:04.646 INFO:teuthology.orchestra.run.smithi118.stdout:3997 read 0x50f850e thru 0x50fe4e0 (0x5fd3 bytes) 2024-09-13T07:03:04.646 INFO:teuthology.orchestra.run.smithi118.stdout:4000 punch from 0x1893cac to 0x189df63, (0xa2b7 bytes) 2024-09-13T07:03:04.646 INFO:teuthology.orchestra.run.smithi118.stdout:4001 punch from 0x27cacc5 to 0x27d02ff, (0x563a bytes) 2024-09-13T07:03:04.646 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:04.646 INFO:teuthology.orchestra.run.smithi118.stdout:4005 read 0x69e356d thru 0x69ed3e8 (0x9e7c bytes) 2024-09-13T07:03:04.646 INFO:teuthology.orchestra.run.smithi118.stdout:4006 punch from 0x30bb6f5 to 0x30c67bf, (0xb0ca bytes) 2024-09-13T07:03:04.646 INFO:teuthology.orchestra.run.smithi118.stdout:4007 writesame 0x3200544 thru 0x320ca5e data_size 0x1(0xc51b bytes) 2024-09-13T07:03:04.650 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:04.648+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:04.683 INFO:teuthology.orchestra.run.smithi118.stdout:4008 writesame 0x9b84be5 thru 0x9b91e96 data_size 0x2(0xd2b2 bytes) 2024-09-13T07:03:04.725 INFO:teuthology.orchestra.run.smithi118.stdout:4010 write 0xb3ba3c2 thru 0xb3bfd71 (0x59b0 bytes) 2024-09-13T07:03:04.733 INFO:teuthology.orchestra.run.smithi118.stdout:4011 punch from 0x684d3b9 to 0x685b28a, (0xded1 bytes) 2024-09-13T07:03:04.733 INFO:teuthology.orchestra.run.smithi118.stdout:4012 read 0xaae1be9 thru 0xaaed765 (0xbb7d bytes) 2024-09-13T07:03:04.734 INFO:teuthology.orchestra.run.smithi118.stdout:4013 read 0xa9a4112 thru 0xa9a4540 (0x42f bytes) 2024-09-13T07:03:04.734 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:04.734 INFO:teuthology.orchestra.run.smithi118.stdout:4017 write 0x9c4d749 thru 0x9c5050b (0x2dc3 bytes) 2024-09-13T07:03:04.734 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:03:04.751 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:04.750+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:04.757 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:04.757 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:04.757 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:04.757 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:04.767 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:04.853 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:04.850+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:04.917 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:04.917 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:04.917 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:04.917 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:04.918 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:04.927 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:04.954 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:04.952+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:05.032 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:03:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.976351640000004, 2024-09-13T07:03:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:03:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:03:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:03:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:03:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:03:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:03:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:03:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:03:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:03:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:03:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:03:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:03:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:03:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:03:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:03:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:03:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.818470406000003, 2024-09-13T07:03:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:03:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:03:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:05.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:03:05.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:05.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:03:05.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:03:05.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:03:05.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:03:05.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:03:05.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:03:05.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:03:05.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:03:05.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:05.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:03:05.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:03:05.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:05.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:03:05.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:03:05.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:05.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:03:05.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:03:05.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:03:05.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:03:05.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:05.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:03:05.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:03:05.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:03:05.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:03:05.055 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.055 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.055 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.056 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:05.056 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:05.056 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.792700694999994, 2024-09-13T07:03:05.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:03:05.056 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.056 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.056 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.057 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.057 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.057 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:03:05.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.057 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:05.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:05.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:03:05.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:05.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:03:05.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:03:05.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:03:05.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:03:05.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:03:05.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:03:05.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:03:05.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:03:05.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:05.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:03:05.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:03:05.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:05.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:03:05.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:03:05.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:05.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:03:05.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:03:05.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:03:05.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:03:05.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:05.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:03:05.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:03:05.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:03:05.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:03:05.064 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.064 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.064 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.064 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:05.064 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:05.064 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.771549528999998, 2024-09-13T07:03:05.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:03:05.064 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.065 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.065 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:03:05.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.065 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:05.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:05.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:03:05.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:05.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:03:05.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:03:05.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:03:05.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:03:05.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:03:05.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:03:05.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:03:05.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:03:05.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:05.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:03:05.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:03:05.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:05.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:03:05.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:03:05.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:05.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:03:05.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:03:05.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:03:05.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:03:05.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:05.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:03:05.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:03:05.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:03:05.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:03:05.071 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.071 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.071 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.072 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:03:05.072 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:05.072 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.654504766000002, 2024-09-13T07:03:05.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:03:05.072 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.072 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.072 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:03:05.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.073 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:05.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:03:05.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:03:05.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:03:05.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:03:05.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:03:05.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:03:05.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:03:05.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:03:05.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:05.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:03:05.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:03:05.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:05.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:03:05.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:03:05.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:05.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:03:05.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:03:05.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:03:05.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:03:05.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:05.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:03:05.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:03:05.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:03:05.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:03:05.084 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.084 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.084 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.084 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94596 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:05.084 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:03:05.084 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.369528287000001, 2024-09-13T07:03:05.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044330268999999999, 2024-09-13T07:03:05.084 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.084 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:05.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.085 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.085 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94596 2024-09-13T07:03:05.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.085 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:03:05.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660996+0000", 2024-09-13T07:03:05.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660995+0000", 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660999+0000", 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2770000000000001e-06 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661007+0000", 2024-09-13T07:03:05.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0029999999999994e-06 2024-09-13T07:03:05.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661012+0000", 2024-09-13T07:03:05.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1279999999999999e-06 2024-09-13T07:03:05.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705253+0000", 2024-09-13T07:03:05.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044241603999999997 2024-09-13T07:03:05.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:05.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705293+0000", 2024-09-13T07:03:05.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9684999999999997e-05 2024-09-13T07:03:05.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705326+0000", 2024-09-13T07:03:05.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2932000000000003e-05 2024-09-13T07:03:05.088 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.088 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.088 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.089 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:05.089 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:05.089 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.369507521000003, 2024-09-13T07:03:05.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:03:05.089 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.089 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:05.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.090 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:03:05.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.090 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:05.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:05.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:03:05.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:03:05.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:03:05.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:03:05.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:03:05.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:03:05.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:03:05.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:03:05.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:03:05.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:03:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:03:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:03:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:03:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94598 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.369336799999999, 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044365860999999999, 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94598 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.095 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:03:05.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:03:05.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661034+0000", 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9998465 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661192+0000", 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015813000000000001 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661199+0000", 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5689999999999999e-06 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705452+0000", 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044246273000000003 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705501+0000", 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8748000000000002e-05 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705553+0000", 2024-09-13T07:03:05.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2241999999999999e-05 2024-09-13T07:03:05.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.099 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.099 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107060 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1003520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:05.099 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:03:05.099 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.117814510999999, 2024-09-13T07:03:05.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044341754999999997, 2024-09-13T07:03:05.099 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107060 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:03:05.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912709+0000", 2024-09-13T07:03:05.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912712+0000", 2024-09-13T07:03:05.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6000000000000001e-06 2024-09-13T07:03:05.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912717+0000", 2024-09-13T07:03:05.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3689999999999998e-06 2024-09-13T07:03:05.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912722+0000", 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7219999999999999e-06 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956876+0000", 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044154469000000002 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956957+0000", 2024-09-13T07:03:05.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1231999999999997e-05 2024-09-13T07:03:05.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957051+0000", 2024-09-13T07:03:05.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3858999999999997e-05 2024-09-13T07:03:05.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.104 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.104 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:05.104 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:05.104 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.117798383, 2024-09-13T07:03:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:03:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:03:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:03:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:03:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:03:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:03:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:03:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:03:05.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.109 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.109 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.109 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:05.109 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:05.109 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.117780349, 2024-09-13T07:03:05.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:03:05.109 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.109 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:05.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.110 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:03:05.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.110 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:05.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:05.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:03:05.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:03:05.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:03:05.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:03:05.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:03:05.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:03:05.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:05.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:03:05.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:03:05.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:03:05.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:03:05.114 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.114 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.114 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.114 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:05.114 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:05.114 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.117764268999998, 2024-09-13T07:03:05.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:03:05.114 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.114 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:05.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:03:05.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:05.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:03:05.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:03:05.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:03:05.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:03:05.118 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.118 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.118 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.118 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:05.118 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:05.119 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.117748593000002, 2024-09-13T07:03:05.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:03:05.119 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.119 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:05.119 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.119 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.119 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.119 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:03:05.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.119 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:05.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:05.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:03:05.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:03:05.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:03:05.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:03:05.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:03:05.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:03:05.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:05.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:03:05.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:03:05.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.117725671000002, 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:05.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:05.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:05.127 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.117709574000003, 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:05.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107067 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:05.131 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:05.132 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.117149556999998, 2024-09-13T07:03:05.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04441055, 2024-09-13T07:03:05.132 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.132 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:05.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.132 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107067 2024-09-13T07:03:05.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.133 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:05.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:05.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912832+0000", 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9994574 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913379+0000", 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00054719999999999997 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913386+0000", 2024-09-13T07:03:05.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4549999999999998e-06 2024-09-13T07:03:05.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913393+0000", 2024-09-13T07:03:05.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6259999999999997e-06 2024-09-13T07:03:05.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957776+0000", 2024-09-13T07:03:05.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044383143999999999 2024-09-13T07:03:05.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957785+0000", 2024-09-13T07:03:05.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9220000000000003e-06 2024-09-13T07:03:05.136 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.136 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.136 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.136 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107068 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:05.137 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:05.137 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.117125125999998, 2024-09-13T07:03:05.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044410502999999997, 2024-09-13T07:03:05.137 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.137 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:05.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.137 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107068 2024-09-13T07:03:05.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.137 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913401+0000", 2024-09-13T07:03:05.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:03:05.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913407+0000", 2024-09-13T07:03:05.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7520000000000002e-06 2024-09-13T07:03:05.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913412+0000", 2024-09-13T07:03:05.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8550000000000001e-06 2024-09-13T07:03:05.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957802+0000", 2024-09-13T07:03:05.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044389606999999998 2024-09-13T07:03:05.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957809+0000", 2024-09-13T07:03:05.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9160000000000001e-06 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.117107961999999, 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.141 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:05.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:03:05.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:03:05.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:03:05.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:03:05.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:03:05.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:03:05.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:05.145 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.117065185999998, 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:05.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:05.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:03:05.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:03:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:03:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:03:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:03:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:03:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:03:05.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.960047415999998, 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:05.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:05.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:05.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:05.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:05.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:05.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:03:05.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:05.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:03:05.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:03:05.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:05.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:03:05.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:03:05.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:03:05.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:03:05.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:05.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:03:05.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:03:05.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:05.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:05.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:05.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:03:05.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:03:05.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.154 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:05.154 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:05.154 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:05.156 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:05.156 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:05.152+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:05.256 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:05.252+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:05.280 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:05.280 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:05.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.280 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137474 3.6 3.ec8c3a6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-13T07:03:05.280 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:05.280 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.53804782299999998, 2024-09-13T07:03:05.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.53808155599999996, 2024-09-13T07:03:05.280 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.280 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "waiting for peered", 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137474 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:05.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743934+0000", 2024-09-13T07:03:05.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:05.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743949+0000", 2024-09-13T07:03:05.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5705999999999998e-05 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743960+0000", 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0084e-05 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743971+0000", 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0942e-05 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744045+0000", 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4626e-05 2024-09-13T07:03:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:05.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744062+0000", 2024-09-13T07:03:05.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7339000000000001e-05 2024-09-13T07:03:05.284 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.284 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.284 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.284 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.284 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:03:05.284 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 1 2024-09-13T07:03:05.284 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:05.291 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:05.358 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:05.354+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:05.403 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:05.404 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:05.404 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:05.404 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:05.404 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:05.414 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:05.458 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:05.454+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:05.528 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:05.529 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:05.529 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:05.529 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:05.529 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.529 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:05.529 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:03:05.529 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 100.022637997, 2024-09-13T07:03:05.529 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:03:05.529 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.529 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.529 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.529 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.529 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.530 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.531 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.532 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.532 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:03:05.532 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:03:05.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.532 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.532 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:03:05.532 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:03:05.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.532 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.532 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.533 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:03:05.533 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:03:05.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.533 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.533 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:05.533 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:03:05.533 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:03:05.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.533 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.534 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:05.534 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:03:05.534 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:03:05.534 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.534 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.534 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.534 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:03:05.534 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:03:05.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.535 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:05.535 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:03:05.535 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:03:05.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.535 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.536 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.536 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:03:05.536 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:03:05.536 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.536 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.536 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.536 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.537 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:05.537 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:03:05.537 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.997810286999993, 2024-09-13T07:03:05.537 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:03:05.537 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.537 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.537 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.538 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.538 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.538 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:03:05.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.538 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.538 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.538 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.538 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:03:05.539 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.539 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.539 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:03:05.539 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.539 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.540 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.540 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:03:05.540 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:05.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.540 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.540 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.540 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:03:05.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:03:05.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.541 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.541 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:03:05.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:03:05.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:03:05.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:03:05.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:03:05.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:03:05.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.543 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:03:05.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:03:05.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:05.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:03:05.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:03:05.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.544 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:05.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:03:05.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:03:05.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:03:05.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:03:05.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:05.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:03:05.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:03:05.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.546 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:03:05.546 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:03:05.546 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.546 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.546 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.546 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:05.546 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:03:05.546 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.459138463000002, 2024-09-13T07:03:05.546 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:03:05.546 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.546 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.547 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.547 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.547 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.547 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:03:05.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.547 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.547 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:03:05.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:03:05.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:03:05.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:03:05.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:03:05.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:03:05.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:03:05.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:03:05.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:03:05.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:03:05.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.550 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:03:05.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:03:05.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.550 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:03:05.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:03:05.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.550 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:05.550 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.551 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:05.552 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:03:05.552 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:03:05.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.552 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.552 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:03:05.552 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:03:05.552 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.552 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.552 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.553 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:05.553 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:03:05.553 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.429513270000001, 2024-09-13T07:03:05.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:03:05.553 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.553 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.553 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.553 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.553 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.555 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:03:05.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:03:05.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:03:05.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:03:05.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:03:05.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:03:05.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:05.557 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:03:05.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:03:05.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:05.557 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:03:05.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:03:05.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:03:05.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:03:05.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:05.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:03:05.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:03:05.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:03:05.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:03:05.558 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.559 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.559 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.559 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.559 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:05.559 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:05.559 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.896799638999994, 2024-09-13T07:03:05.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:03:05.559 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.559 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.559 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.559 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.559 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.560 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:03:05.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.560 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:05.560 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:03:05.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.564 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:03:05.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:03:05.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:05.564 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:03:05.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:03:05.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.564 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.564 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.896717183000007, 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.565 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:03:05.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:05.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:03:05.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:03:05.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:03:05.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:03:05.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:03:05.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:03:05.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:03:05.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:03:05.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.570 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.896685372999997, 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.571 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:03:05.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:03:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:03:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:03:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:03:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:05.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:03:05.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:03:05.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:03:05.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:03:05.576 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.576 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.576 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:03:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.896630215000002, 2024-09-13T07:03:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:03:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:03:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:03:05.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:03:05.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:03:05.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:05.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:03:05.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:03:05.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:03:05.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:03:05.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:03:05.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:03:05.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:03:05.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:03:05.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:03:05.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:03:05.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:05.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:03:05.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:03:05.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:03:05.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:03:05.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.582 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.583 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.583 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:05.583 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:03:05.583 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.896584935999996, 2024-09-13T07:03:05.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:03:05.583 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.583 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.583 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.584 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.584 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.584 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:03:05.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.584 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:03:05.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:03:05.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:03:05.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:05.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:03:05.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:03:05.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:05.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:03:05.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:03:05.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:03:05.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:03:05.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:05.589 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:03:05.590 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.896546401999998, 2024-09-13T07:03:05.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:03:05.590 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.590 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.590 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.590 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.590 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.590 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:03:05.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.590 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:03:05.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:03:05.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:03:05.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:05.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:03:05.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:03:05.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:03:05.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:03:05.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:03:05.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:03:05.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:03:05.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:03:05.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:03:05.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:03:05.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:03:05.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:03:05.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:05.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:03:05.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:03:05.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:03:05.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:03:05.596 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.596 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.596 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.596 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:05.596 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:03:05.596 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.896484783000005, 2024-09-13T07:03:05.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:03:05.596 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.596 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.597 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.597 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.597 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.597 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:03:05.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.597 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:03:05.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:03:05.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:03:05.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:05.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:03:05.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:03:05.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:03:05.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:03:05.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:03:05.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:03:05.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:03:05.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:03:05.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:03:05.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:03:05.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:03:05.602 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.602 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.602 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.602 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:05.602 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:03:05.602 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.882262100000005, 2024-09-13T07:03:05.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:03:05.603 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.603 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.603 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.603 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.603 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.603 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:03:05.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.603 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:03:05.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:03:05.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:03:05.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:03:05.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:03:05.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:03:05.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:05.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:03:05.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:03:05.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:03:05.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:03:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:03:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:3952 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:03:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.870383579999995, 2024-09-13T07:03:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091538060000000004, 2024-09-13T07:03:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 3952 2024-09-13T07:03:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657065+0000", 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657064+0000", 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657067+0000", 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.745e-06 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657073+0000", 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0469999999999999e-06 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657078+0000", 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8740000000000003e-06 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666075+0000", 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089973120000000004 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666144+0000", 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8718000000000002e-05 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666218+0000", 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4555999999999996e-05 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4001 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:03:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.869826519, 2024-09-13T07:03:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090973870000000002, 2024-09-13T07:03:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4001 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657622+0000", 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657621+0000", 2024-09-13T07:03:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657629+0000", 2024-09-13T07:03:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6109999999999996e-06 2024-09-13T07:03:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657635+0000", 2024-09-13T07:03:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0950000000000001e-06 2024-09-13T07:03:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657639+0000", 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5379999999999996e-06 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666573+0000", 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089335060000000008 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666640+0000", 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6857e-05 2024-09-13T07:03:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666719+0000", 2024-09-13T07:03:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9206e-05 2024-09-13T07:03:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:03:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.869804364000004, 2024-09-13T07:03:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:03:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:03:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.869784647000003, 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:03:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:03:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:03:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:03:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:03:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:03:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:03:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:03:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:03:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:03:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:03:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:03:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:03:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.869767316999997, 2024-09-13T07:03:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.869471654999998, 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:03:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:03:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:03:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:03:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:03:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:03:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:03:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:03:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:03:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:03:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.308268835999996, 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:03:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:03:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:03:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.276189932999998, 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:03:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:03:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:03:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:03:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:03:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:05.644 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:05.646 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:05.646 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:05.644+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:05.746 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:03:05.746 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:05.746+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:05.772 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:05.772 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:05.773 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:05.782 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:05.847 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:05.846+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:05.932 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:05.932 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:05.932 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:05.937 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:03:05.941 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:05.948 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:05.944+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:06.047 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:06.048 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:06.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:06.048 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:06.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.048 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:03:06.048 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:06.049 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 82.553978634999993, 2024-09-13T07:03:06.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:03:06.049 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.049 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:06.049 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.049 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.050 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.050 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:03:06.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.050 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:06.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:06.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.053 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:03:06.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:06.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.053 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:03:06.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:03:06.054 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.054 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.054 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.054 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:03:06.054 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:03:06.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.055 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.055 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:03:06.055 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:03:06.055 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.055 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:03:06.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:03:06.056 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.056 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.056 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:06.056 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:03:06.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:03:06.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:03:06.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:03:06.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:06.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:03:06.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:03:06.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:03:06.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:03:06.059 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.059 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.059 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.059 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.059 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:06.059 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.067495070999996, 2024-09-13T07:03:06.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:03:06.060 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.060 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:06.060 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.060 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.060 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.060 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:03:06.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.060 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:06.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:06.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:06.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:03:06.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:03:06.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.062 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.062 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:06.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:06.064 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.067425142000005, 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:03:06.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:06.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:03:06.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:03:06.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:03:06.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:03:06.068 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.068 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.068 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.067409269999999, 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:03:06.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:03:06.072 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.073 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.073 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.073 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.073 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:06.073 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.067338813000006, 2024-09-13T07:03:06.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:03:06.073 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.073 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:06.073 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.073 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.073 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:03:06.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:03:06.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:03:06.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:03:06.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:06.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:03:06.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:03:06.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:03:06.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:03:06.077 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.078 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.078 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.078 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.078 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:06.079 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063986775000004, 2024-09-13T07:03:06.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:03:06.079 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.079 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.079 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.079 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.080 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.080 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:03:06.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.080 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:06.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:06.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:06.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:03:06.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:03:06.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:03:06.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:03:06.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:03:06.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:03:06.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:03:06.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:03:06.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:03:06.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:03:06.086 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.087 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.087 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.087 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.087 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:06.088 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063970474000001, 2024-09-13T07:03:06.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:03:06.088 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.088 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.088 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.088 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.089 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.089 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:03:06.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.089 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:06.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:06.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:03:06.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:03:06.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:03:06.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:03:06.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:06.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:03:06.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:03:06.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:03:06.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:03:06.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:03:06.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:03:06.099 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.099 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.099 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.100 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.100 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:06.100 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063950688999995, 2024-09-13T07:03:06.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:03:06.100 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.101 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.101 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.101 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.101 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.101 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:03:06.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.102 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:06.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:06.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:06.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.105 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.105 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:03:06.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:03:06.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:03:06.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:03:06.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:03:06.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:03:06.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:03:06.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:03:06.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:03:06.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:03:06.109 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.109 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.109 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.109 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.110 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:06.110 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063934415999995, 2024-09-13T07:03:06.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:03:06.110 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.110 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.110 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.111 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.111 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.111 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:03:06.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.111 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:06.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:06.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:03:06.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:03:06.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:03:06.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:03:06.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:03:06.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:03:06.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:03:06.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:03:06.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:03:06.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.117 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.117 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:03:06.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:03:06.117 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.117 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.118 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.118 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.118 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:06.118 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063916590999995, 2024-09-13T07:03:06.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:03:06.118 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.118 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.118 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.118 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:06.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:03:06.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:03:06.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:03:06.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:03:06.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:03:06.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:03:06.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:03:06.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:03:06.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063900497999995, 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.122 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:03:06.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:03:06.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:03:06.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:03:06.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063879311999997, 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.126 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.127 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:03:06.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.127 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:06.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:06.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:03:06.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:03:06.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063863279000003, 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.130 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.131 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:03:06.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.131 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:06.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.131 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.131 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:06.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:03:06.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:03:06.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063814039999997, 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:03:06.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.135 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:03:06.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:03:06.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:03:06.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:03:06.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063795979999995, 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.138 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:06.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:03:06.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:03:06.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:03:06.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:03:06.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:03:06.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:03:06.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:03:06.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:03:06.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:03:06.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:03:06.142 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.142 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.142 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.142 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.142 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:06.142 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063779464999996, 2024-09-13T07:03:06.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:03:06.142 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.142 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.142 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:06.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:03:06.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:03:06.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:03:06.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:03:06.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:03:06.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:03:06.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063761099999994, 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.146 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:06.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:03:06.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:03:06.150 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.150 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.150 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.150 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.150 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:06.150 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063745002000005, 2024-09-13T07:03:06.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:03:06.150 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.150 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:06.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:06.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:03:06.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:03:06.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:03:06.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063729066999997, 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:03:06.154 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:06.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:06.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:03:06.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:03:06.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:03:06.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:03:06.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:03:06.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:03:06.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:06.158 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 68.063712570000007, 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:03:06.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:03:06.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:06.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:03:06.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:03:06.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:06.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:03:06.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:03:06.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:06.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:03:06.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:03:06.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:06.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:06.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:06.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:03:06.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:03:06.162 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.162 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.162 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.162 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:06.162 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:06.162 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:06.164 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:06.165 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:06.166 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:06.166 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:06.166 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:06.166 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:06.166 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:06.160+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:06.177 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:06.201 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:06.225 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:06.227 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:06.228 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:06.228 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:06.228 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:06.228 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:06.228 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:06.228 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:06.228 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:06.281 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:06.280+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:06.285 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:06.285 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:06.295 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:06.366 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:06.364+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:06.398 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:06.398 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:06.408 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:06.467 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:06.466+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:06.518 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:06.518 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:06.528 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:06.568 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:06.566+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:06.631 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:06.632 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:06.642 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:06.669 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:06.666+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:06.730 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7, 2, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:03:06.730 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:03:06.730 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [1, 3, 4, 5, 6, 7, 2, 0] 2024-09-13T07:03:06.730 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-13T07:03:06.755 INFO:teuthology.orchestra.run.smithi118.stdout:4020 writesame 0xc6fad8f thru 0xc703804 data_size 0x25(0x8a76 bytes) 2024-09-13T07:03:06.762 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:06.763 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:06.769 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:06.766+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:06.772 INFO:teuthology.orchestra.run.smithi118.stdout:4021 trunc from 0xc703805 to 0x21be3f7 2024-09-13T07:03:06.777 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:06.820 INFO:teuthology.orchestra.run.smithi118.stdout:4023 write 0x89d3c82 thru 0x89d51a0 (0x151f bytes) 2024-09-13T07:03:06.826 INFO:teuthology.orchestra.run.smithi118.stdout:4026 write 0x6b16c1c thru 0x6b215c5 (0xa9aa bytes) 2024-09-13T07:03:06.833 INFO:teuthology.orchestra.run.smithi118.stdout:4029 write 0x5446770 thru 0x54551e3 (0xea74 bytes) 2024-09-13T07:03:06.840 INFO:teuthology.orchestra.run.smithi118.stdout:4031 write 0x33142bc thru 0x3322563 (0xe2a8 bytes) 2024-09-13T07:03:06.847 INFO:teuthology.orchestra.run.smithi118.stdout:4032 read 0x3f6575c thru 0x3f6908f (0x3934 bytes) 2024-09-13T07:03:06.847 INFO:teuthology.orchestra.run.smithi118.stdout:4033 punch from 0x292b439 to 0x292c6fb, (0x12c2 bytes) 2024-09-13T07:03:06.850 INFO:teuthology.orchestra.run.smithi118.stdout:4034 trunc from 0x89d51a1 to 0xa597d17 2024-09-13T07:03:06.859 INFO:teuthology.orchestra.run.smithi118.stdout:4035 read 0x9416963 thru 0x941a2c8 (0x3966 bytes) 2024-09-13T07:03:06.859 INFO:teuthology.orchestra.run.smithi118.stdout:4037 write 0x12d7cd8 thru 0x12e3faf (0xc2d8 bytes) 2024-09-13T07:03:06.867 INFO:teuthology.orchestra.run.smithi118.stdout:4038 write 0x9a3c0b thru 0x9ac424 (0x881a bytes) 2024-09-13T07:03:06.870 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:06.866+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:06.873 INFO:teuthology.orchestra.run.smithi118.stdout:4040 read 0x468afe thru 0x475f36 (0xd439 bytes) 2024-09-13T07:03:06.874 INFO:teuthology.orchestra.run.smithi118.stdout:4042 trunc from 0xa597d17 to 0xa2082e1 2024-09-13T07:03:06.884 INFO:teuthology.orchestra.run.smithi118.stdout:4043 punch from 0x43a2b98 to 0x43ab0b1, (0x8519 bytes) 2024-09-13T07:03:06.884 INFO:teuthology.orchestra.run.smithi118.stdout:4045 read 0x6cdaad6 thru 0x6ce5160 (0xa68b bytes) 2024-09-13T07:03:06.884 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:06.884 INFO:teuthology.orchestra.run.smithi118.stdout:4047 read 0x839e8b1 thru 0x83abae4 (0xd234 bytes) 2024-09-13T07:03:06.884 INFO:teuthology.orchestra.run.smithi118.stdout:4048 write 0x16a62c7 thru 0x16b424c (0xdf86 bytes) 2024-09-13T07:03:06.890 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:06.890 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:06.891 INFO:teuthology.orchestra.run.smithi118.stdout:4049 punch from 0x40da12f to 0x40e7c63, (0xdb34 bytes) 2024-09-13T07:03:06.891 INFO:teuthology.orchestra.run.smithi118.stdout:4050 writesame 0x351202c thru 0x351a3ea data_size 0x1(0x83bf bytes) 2024-09-13T07:03:06.900 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:06.924 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:06.925 INFO:teuthology.orchestra.run.smithi118.stdout:4052 writesame 0x9deba1f thru 0x9df6723 data_size 0x1(0xad05 bytes) 2024-09-13T07:03:06.955 INFO:teuthology.orchestra.run.smithi118.stdout:4053 writesame 0x936b776 thru 0x937aa5a data_size 0x1b(0xf2e5 bytes) 2024-09-13T07:03:06.971 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:06.968+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:06.972 INFO:teuthology.orchestra.run.smithi118.stdout:4054 writesame 0xaa641bc thru 0xaa6af87 data_size 0x2(0x6dcc bytes) 2024-09-13T07:03:06.972 INFO:teuthology.orchestra.run.smithi118.stdout:4055 write 0x6f08a78 thru 0x6f162ed (0xd876 bytes) 2024-09-13T07:03:06.978 INFO:teuthology.orchestra.run.smithi118.stdout:4056 punch from 0x5ac9525 to 0x5ad90df, (0xfbba bytes) 2024-09-13T07:03:06.978 INFO:teuthology.orchestra.run.smithi118.stdout:4058 write 0x35b0c59 thru 0x35b43c4 (0x376c bytes) 2024-09-13T07:03:06.994 INFO:teuthology.orchestra.run.smithi118.stdout:4059 write 0xaebf25c thru 0xaec1e6b (0x2c10 bytes) 2024-09-13T07:03:07.003 INFO:teuthology.orchestra.run.smithi118.stdout:4060 write 0x9bee837 thru 0x9beeb70 (0x33a bytes) 2024-09-13T07:03:07.008 INFO:teuthology.orchestra.run.smithi118.stdout:4061 write 0x3dc2497 thru 0x3dcff0d (0xda77 bytes) 2024-09-13T07:03:07.013 INFO:teuthology.orchestra.run.smithi118.stdout:4063 write 0x5b311cd thru 0x5b34f08 (0x3d3c bytes) 2024-09-13T07:03:07.019 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:07.019 INFO:teuthology.orchestra.run.smithi118.stdout:4066 read 0x6a08a thru 0x71267 (0x71de bytes) 2024-09-13T07:03:07.020 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:07.020 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:07.020 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:07.020 INFO:teuthology.orchestra.run.smithi118.stdout:4068 read 0x8846141 thru 0x88548c8 (0xe788 bytes) 2024-09-13T07:03:07.020 INFO:teuthology.orchestra.run.smithi118.stdout:4069 read 0x5d66ecd thru 0x5d73129 (0xc25d bytes) 2024-09-13T07:03:07.021 INFO:teuthology.orchestra.run.smithi118.stdout:4071 read 0x5ab087 thru 0x5b4e22 (0x9d9c bytes) 2024-09-13T07:03:07.021 INFO:teuthology.orchestra.run.smithi118.stdout:4073 writesame 0x239274d thru 0x2395d6c data_size 0x10(0x3620 bytes) 2024-09-13T07:03:07.028 INFO:teuthology.orchestra.run.smithi118.stdout:4075 write 0x284071 thru 0x28ab04 (0x6a94 bytes) 2024-09-13T07:03:07.030 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:07.036 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:07.036 INFO:teuthology.orchestra.run.smithi118.stdout:4077 punch from 0x7c0728e to 0x7c0dd80, (0x6af2 bytes) 2024-09-13T07:03:07.036 INFO:teuthology.orchestra.run.smithi118.stdout:4078 read 0x6d2b4e3 thru 0x6d2eab0 (0x35ce bytes) 2024-09-13T07:03:07.051 INFO:tasks.ceph.mon.a.smithi118.stderr:2024-09-13T07:03:07.048+0000 7fd1ec319700 -1 mon.a@0(leader).osd e118 definitely_dead 0 2024-09-13T07:03:07.053 INFO:teuthology.orchestra.run.smithi118.stdout:4080 write 0xd86a5a5 thru 0xd86a657 (0xb3 bytes) 2024-09-13T07:03:07.058 INFO:teuthology.orchestra.run.smithi118.stdout:4082 read 0x19ba6be thru 0x19bdb34 (0x3477 bytes) 2024-09-13T07:03:07.059 INFO:teuthology.orchestra.run.smithi118.stdout:4086 writesame 0xbdce0c2 thru 0xbdd4292 data_size 0x11(0x61d1 bytes) 2024-09-13T07:03:07.066 INFO:teuthology.orchestra.run.smithi118.stdout:4088 writesame 0x5f7567d thru 0x5f76604 data_size 0x8(0xf88 bytes) 2024-09-13T07:03:07.072 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:07.068+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:07.073 INFO:teuthology.orchestra.run.smithi118.stdout:4089 write 0x6dfa460 thru 0x6e03e3f (0x99e0 bytes) 2024-09-13T07:03:07.080 INFO:teuthology.orchestra.run.smithi118.stdout:4090 read 0x5748630 thru 0x574ed15 (0x66e6 bytes) 2024-09-13T07:03:07.081 INFO:teuthology.orchestra.run.smithi118.stdout:4091 flatten 2024-09-13T07:03:07.146 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:07.146 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:07.155 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:07.156 INFO:teuthology.orchestra.run.smithi118.stdout:4092 trunc from 0xd86a658 to 0x45a47ac 2024-09-13T07:03:07.173 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:07.168+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:07.176 INFO:teuthology.orchestra.run.smithi118.stdout:4094 write 0x1ba9ee4 thru 0x1baa63b (0x758 bytes) 2024-09-13T07:03:07.178 INFO:teuthology.orchestra.run.smithi118.stdout:4095 punch from 0x2f6be11 to 0x2f79a0a, (0xdbf9 bytes) 2024-09-13T07:03:07.187 INFO:teuthology.orchestra.run.smithi118.stdout:4096 trunc from 0x45a47ac to 0x75d2802 2024-09-13T07:03:07.213 INFO:teuthology.orchestra.run.smithi118.stdout:4097 trunc from 0x75d2802 to 0xe3cf88c 2024-09-13T07:03:07.225 INFO:teuthology.orchestra.run.smithi118.stdout:4098 punch from 0x9d2a08b to 0x9d34da1, (0xad16 bytes) 2024-09-13T07:03:07.231 INFO:teuthology.orchestra.run.smithi118.stdout:4099 trunc from 0xe3cf88c to 0xa743048 2024-09-13T07:03:07.245 INFO:teuthology.orchestra.run.smithi118.stdout:4100 read 0x24235a1 thru 0x242a569 (0x6fc9 bytes) 2024-09-13T07:03:07.246 INFO:teuthology.orchestra.run.smithi118.stdout:4102 punch from 0x38bf128 to 0x38c6ec3, (0x7d9b bytes) 2024-09-13T07:03:07.246 INFO:teuthology.orchestra.run.smithi118.stdout:4103 write 0x4cda5e4 thru 0x4cdca95 (0x24b2 bytes) 2024-09-13T07:03:07.252 INFO:teuthology.orchestra.run.smithi118.stdout:4104 write 0x4d4390d thru 0x4d48b3a (0x522e bytes) 2024-09-13T07:03:07.257 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:07.257 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:07.259 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:07.259 INFO:teuthology.orchestra.run.smithi118.stdout:4106 read 0x4e7886d thru 0x4e79607 (0xd9b bytes) 2024-09-13T07:03:07.265 INFO:teuthology.orchestra.run.smithi118.stdout:4107 trunc from 0xa743048 to 0x84ca675 2024-09-13T07:03:07.268 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:03:07.273 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:07.270+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:07.277 INFO:teuthology.orchestra.run.smithi118.stdout:4108 writesame 0x614fcb thru 0x62467a data_size 0x8(0xf6b0 bytes) 2024-09-13T07:03:07.291 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:07.291 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:07.291 INFO:teuthology.orchestra.run.smithi118.stdout:4111 read 0x5b13e3c thru 0x5b174ab (0x3670 bytes) 2024-09-13T07:03:07.315 INFO:teuthology.orchestra.run.smithi118.stdout:4112 write 0xc4cd55b thru 0xc4daa22 (0xd4c8 bytes) 2024-09-13T07:03:07.324 INFO:teuthology.orchestra.run.smithi118.stdout:4113 read 0x939bb77 thru 0x939c725 (0xbaf bytes) 2024-09-13T07:03:07.324 INFO:teuthology.orchestra.run.smithi118.stdout:4114 writesame 0x9e75ad6 thru 0x9e79afd data_size 0x4(0x4028 bytes) 2024-09-13T07:03:07.332 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:07.333 INFO:teuthology.orchestra.run.smithi118.stdout:4116 write 0x60bc73a thru 0x60c9459 (0xcd20 bytes) 2024-09-13T07:03:07.345 INFO:teuthology.orchestra.run.smithi118.stdout:4117 trunc from 0xc4daa23 to 0xae936f0 2024-09-13T07:03:07.374 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:07.374+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:07.379 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:07.379 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:07.383 INFO:teuthology.orchestra.run.smithi118.stdout:4118 write 0xd34cfbb thru 0xd34ddd1 (0xe17 bytes) 2024-09-13T07:03:07.389 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:03:07.389 INFO:teuthology.orchestra.run.smithi118.stdout:4120 write 0x590420d thru 0x590bb15 (0x7909 bytes) 2024-09-13T07:03:07.395 INFO:teuthology.orchestra.run.smithi118.stdout:4121 write 0x4b02088 thru 0x4b0412d (0x20a6 bytes) 2024-09-13T07:03:07.402 INFO:teuthology.orchestra.run.smithi118.stdout:4122 read 0x6b8875a thru 0x6b8e85c (0x6103 bytes) 2024-09-13T07:03:07.402 INFO:teuthology.orchestra.run.smithi118.stdout:4123 punch from 0x6867ec2 to 0x686ba1c, (0x3b5a bytes) 2024-09-13T07:03:07.402 INFO:teuthology.orchestra.run.smithi118.stdout:4124 writesame 0x601315d thru 0x6023145 data_size 0x31(0xffe9 bytes) 2024-09-13T07:03:07.434 INFO:teuthology.orchestra.run.smithi118.stdout:4125 trunc from 0xd34ddd2 to 0x16380db 2024-09-13T07:03:07.475 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:07.474+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:07.500 INFO:teuthology.orchestra.run.smithi118.stdout:4126 write 0xb4e42dd thru 0xb4f0fa7 (0xcccb bytes) 2024-09-13T07:03:07.506 INFO:teuthology.orchestra.run.smithi118.stdout:4129 read 0xea5634 thru 0xeaba72 (0x643f bytes) 2024-09-13T07:03:07.506 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:07.506 INFO:teuthology.orchestra.run.smithi118.stdout:4132 read 0xb0dc31d thru 0xb0eaa95 (0xe779 bytes) 2024-09-13T07:03:07.527 INFO:teuthology.orchestra.run.smithi118.stdout:4134 write 0xe4193dc thru 0xe41f8ee (0x6513 bytes) 2024-09-13T07:03:07.534 INFO:teuthology.orchestra.run.smithi118.stdout:4136 read 0x16684c2 thru 0x166bb9a (0x36d9 bytes) 2024-09-13T07:03:07.535 INFO:teuthology.orchestra.run.smithi118.stdout:4137 read 0x74ec189 thru 0x74f007f (0x3ef7 bytes) 2024-09-13T07:03:07.535 INFO:teuthology.orchestra.run.smithi118.stdout:4138 read 0xad386ea thru 0xad3c50b (0x3e22 bytes) 2024-09-13T07:03:07.535 INFO:teuthology.orchestra.run.smithi118.stdout:4139 punch from 0x694f8c7 to 0x6954091, (0x47ca bytes) 2024-09-13T07:03:07.535 INFO:teuthology.orchestra.run.smithi118.stdout:4140 read 0x62e07e5 thru 0x62e24d9 (0x1cf5 bytes) 2024-09-13T07:03:07.535 INFO:teuthology.orchestra.run.smithi118.stdout:4141 write 0x6162432 thru 0x61631c2 (0xd91 bytes) 2024-09-13T07:03:07.539 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:07.540 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:07.540 INFO:teuthology.orchestra.run.smithi118.stdout:4142 read 0xd2669ae thru 0xd26ded8 (0x752b bytes) 2024-09-13T07:03:07.540 INFO:teuthology.orchestra.run.smithi118.stdout:4143 punch from 0x1c9b64e to 0x1ca24ea, (0x6e9c bytes) 2024-09-13T07:03:07.540 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:07.550 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:03:07.553 INFO:teuthology.orchestra.run.smithi118.stdout:4146 write 0xea45b5f thru 0xea55ae0 (0xff82 bytes) 2024-09-13T07:03:07.559 INFO:teuthology.orchestra.run.smithi118.stdout:4148 read 0x8b4c68b thru 0x8b5c0ee (0xfa64 bytes) 2024-09-13T07:03:07.559 INFO:teuthology.orchestra.run.smithi118.stdout:4150 punch from 0x14df3d9 to 0x14e26f3, (0x331a bytes) 2024-09-13T07:03:07.560 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:07.560 INFO:teuthology.orchestra.run.smithi118.stdout:4153 write 0xc340f29 thru 0xc349d09 (0x8de1 bytes) 2024-09-13T07:03:07.566 INFO:teuthology.orchestra.run.smithi118.stdout:4154 write 0x4f2fd54 thru 0x4f30b3a (0xde7 bytes) 2024-09-13T07:03:07.573 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:07.573 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:07.573 INFO:teuthology.orchestra.run.smithi118.stdout:4161 read 0x620728e thru 0x6212594 (0xb307 bytes) 2024-09-13T07:03:07.573 INFO:teuthology.orchestra.run.smithi118.stdout:4162 punch from 0x6fcc057 to 0x6fd0322, (0x42cb bytes) 2024-09-13T07:03:07.579 INFO:teuthology.orchestra.run.smithi118.stdout:4163 trunc from 0xea55ae1 to 0xc5ea12e 2024-09-13T07:03:07.594 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:07.594+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:07.599 INFO:teuthology.orchestra.run.smithi118.stdout:4164 read 0x6291cbc thru 0x6299c54 (0x7f99 bytes) 2024-09-13T07:03:07.600 INFO:teuthology.orchestra.run.smithi118.stdout:4167 write 0x4a568dd thru 0x4a577ad (0xed1 bytes) 2024-09-13T07:03:07.623 INFO:teuthology.orchestra.run.smithi118.stdout:4169 trunc from 0xc5ea12e to 0x36aa028 2024-09-13T07:03:07.642 INFO:teuthology.orchestra.run.smithi118.stdout:4172 read 0x20e9999 thru 0x20f7dc5 (0xe42d bytes) 2024-09-13T07:03:07.649 INFO:teuthology.orchestra.run.smithi118.stdout:4173 trunc from 0x36aa028 to 0x6ced467 2024-09-13T07:03:07.650 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:07.650 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:07.659 INFO:teuthology.orchestra.run.smithi118.stdout:4174 clone 40 order 25 su 262144 sc 11 2024-09-13T07:03:07.661 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:07.678 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:07.676+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:07.757 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:03:07.766 INFO:teuthology.orchestra.run.smithi118.stderr:marked out osd.1. 2024-09-13T07:03:07.766 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:07.766 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:07.777 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:07.779 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:07.778+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:07.880 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:07.878+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:07.883 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:07.883 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:07.893 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:07.981 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:07.980+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:07.985 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-13T07:03:07.998 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:07.998 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:08.008 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:08.082 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:08.080+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:08.114 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:08.115 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:08.124 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:08.183 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:08.180+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:08.230 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:08.230 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:08.239 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:03:08.240 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:08.285 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:08.284+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:08.344 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:08.344 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:08.354 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:08.359 INFO:teuthology.orchestra.run.smithi118.stdout:4175 read 0x535131d thru 0x5359845 (0x8529 bytes) 2024-09-13T07:03:08.384 INFO:teuthology.orchestra.run.smithi118.stdout:4178 write 0xd065e51 thru 0xd06735d (0x150d bytes) 2024-09-13T07:03:08.384 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:08.380+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:08.388 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:08.392 INFO:teuthology.orchestra.run.smithi118.stdout:4179 writesame 0xb95d434 thru 0xb9616b4 data_size 0x19(0x4281 bytes) 2024-09-13T07:03:08.399 INFO:teuthology.orchestra.run.smithi118.stdout:4182 write 0x3b8533d thru 0x3b92bc9 (0xd88d bytes) 2024-09-13T07:03:08.409 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:08.418 INFO:teuthology.orchestra.run.smithi118.stdout:4183 trunc from 0xd06735e to 0x7a19061 2024-09-13T07:03:08.433 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:08.449 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:08.450 INFO:teuthology.orchestra.run.smithi118.stdout:4186 read 0x255b531 thru 0x255f8d1 (0x43a1 bytes) 2024-09-13T07:03:08.450 INFO:teuthology.orchestra.run.smithi118.stdout:4187 punch from 0x4f0076d to 0x4f0e5a9, (0xde3c bytes) 2024-09-13T07:03:08.450 INFO:teuthology.orchestra.run.smithi118.stdout:4190 read 0x48c850b thru 0x48cb347 (0x2e3d bytes) 2024-09-13T07:03:08.455 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:08.460 INFO:teuthology.orchestra.run.smithi118.stdout:4192 write 0x934ed50 thru 0x935a27b (0xb52c bytes) 2024-09-13T07:03:08.471 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:08.471 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:08.471 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:08.471 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:08.475 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:08.476 INFO:teuthology.orchestra.run.smithi118.stdout:4193 write 0xbe7c5d0 thru 0xbe892ce (0xccff bytes) 2024-09-13T07:03:08.479 INFO:teuthology.orchestra.run.smithi118.stdout:4194 punch from 0x2eaf49b to 0x2eb0515, (0x107a bytes) 2024-09-13T07:03:08.479 INFO:teuthology.orchestra.run.smithi118.stdout:4195 flatten 2024-09-13T07:03:08.482 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:08.485 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:08.482+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:08.491 INFO:teuthology.orchestra.run.smithi118.stdout:4196 writesame 0xe1ffac thru 0xe20032 data_size 0x5(0x87 bytes) 2024-09-13T07:03:08.494 INFO:teuthology.orchestra.run.smithi118.stdout:4197 write 0x79c4e4 thru 0x7a44f8 (0x8015 bytes) 2024-09-13T07:03:08.497 INFO:teuthology.orchestra.run.smithi118.stdout:4198 write 0x61b9204 thru 0x61bb8bf (0x26bc bytes) 2024-09-13T07:03:08.499 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:08.499 INFO:teuthology.orchestra.run.smithi118.stdout:4199 write 0x35902ff thru 0x3599633 (0x9335 bytes) 2024-09-13T07:03:08.502 INFO:teuthology.orchestra.run.smithi118.stdout:4200 write 0x73f0f88 thru 0x73f1a49 (0xac2 bytes) 2024-09-13T07:03:08.505 INFO:teuthology.orchestra.run.smithi118.stdout:4201 punch from 0x13ca288 to 0x13cf9ce, (0x5746 bytes) 2024-09-13T07:03:08.505 INFO:teuthology.orchestra.run.smithi118.stdout:4202 read 0x69daeb2 thru 0x69db934 (0xa83 bytes) 2024-09-13T07:03:08.506 INFO:teuthology.orchestra.run.smithi118.stdout:4203 write 0x6ca852a thru 0x6cb04f3 (0x7fca bytes) 2024-09-13T07:03:08.522 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:08.522 INFO:teuthology.orchestra.run.smithi118.stdout:4204 write 0xe2cf5c7 thru 0xe2d644d (0x6e87 bytes) 2024-09-13T07:03:08.525 INFO:teuthology.orchestra.run.smithi118.stdout:4205 read 0xb23e43 thru 0xb3226b (0xe429 bytes) 2024-09-13T07:03:08.526 INFO:teuthology.orchestra.run.smithi118.stdout:4206 writesame 0xeaedae thru 0xeb0d31 data_size 0x2(0x1f84 bytes) 2024-09-13T07:03:08.530 INFO:teuthology.orchestra.run.smithi118.stdout:4207 read 0xc61f1e8 thru 0xc62ebcd (0xf9e6 bytes) 2024-09-13T07:03:08.531 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:08.531 INFO:teuthology.orchestra.run.smithi118.stdout:4209 read 0xb304074 thru 0xb307db8 (0x3d45 bytes) 2024-09-13T07:03:08.531 INFO:teuthology.orchestra.run.smithi118.stdout:4211 read 0x3758d7b thru 0x3766002 (0xd288 bytes) 2024-09-13T07:03:08.532 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:08.533 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:08.533 INFO:teuthology.orchestra.run.smithi118.stdout:4214 punch from 0xa78fb4 to 0xa83de2, (0xae2e bytes) 2024-09-13T07:03:08.533 INFO:teuthology.orchestra.run.smithi118.stdout:4215 write 0x2b0c2e4 thru 0x2b17c81 (0xb99e bytes) 2024-09-13T07:03:08.536 INFO:teuthology.orchestra.run.smithi118.stdout:4216 writesame 0x287a0ba thru 0x2887272 data_size 0x1(0xd1b9 bytes) 2024-09-13T07:03:08.545 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:08.547 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:08.547 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:08.547 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:08.547 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:08.547 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:08.547 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:08.547 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:08.547 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:08.583 INFO:teuthology.orchestra.run.smithi118.stdout:4217 writesame 0x33cd0a7 thru 0x33d58f1 data_size 0x25(0x884b bytes) 2024-09-13T07:03:08.586 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:08.586+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:08.587 INFO:teuthology.orchestra.run.smithi118.stdout:4218 read 0x6b10c5 thru 0x6b7a5b (0x6997 bytes) 2024-09-13T07:03:08.587 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:08.587 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:08.587 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:08.587 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:08.587 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:08.588 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:08.588 INFO:teuthology.orchestra.run.smithi118.stdout:4221 punch from 0x3c375f0 to 0x3c3df4e, (0x695e bytes) 2024-09-13T07:03:08.588 INFO:teuthology.orchestra.run.smithi118.stdout:4222 punch from 0x96bf42f to 0x96cb5b9, (0xc18a bytes) 2024-09-13T07:03:08.588 INFO:teuthology.orchestra.run.smithi118.stdout:4224 read 0xdd43194 thru 0xdd45bf0 (0x2a5d bytes) 2024-09-13T07:03:08.588 INFO:teuthology.orchestra.run.smithi118.stdout:4228 write 0xca22ee0 thru 0xca32e8f (0xffb0 bytes) 2024-09-13T07:03:08.591 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:08.591 INFO:teuthology.orchestra.run.smithi118.stdout:4230 read 0x25b65bd thru 0x25beec3 (0x8907 bytes) 2024-09-13T07:03:08.592 INFO:teuthology.orchestra.run.smithi118.stdout:4231 write 0x4141a61 thru 0x414de0c (0xc3ac bytes) 2024-09-13T07:03:08.596 INFO:teuthology.orchestra.run.smithi118.stdout:4232 write 0x881e3a7 thru 0x882c4b1 (0xe10b bytes) 2024-09-13T07:03:08.598 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:08.599 INFO:teuthology.orchestra.run.smithi118.stdout:4235 writesame 0x5d8a2c8 thru 0x5d92692 data_size 0x1(0x83cb bytes) 2024-09-13T07:03:08.642 INFO:teuthology.orchestra.run.smithi118.stdout:4237 writesame 0xea4af65 thru 0xea4b074 data_size 0x8(0x110 bytes) 2024-09-13T07:03:08.645 INFO:teuthology.orchestra.run.smithi118.stdout:4238 writesame 0xe20cb22 thru 0xe20fd86 data_size 0x1(0x3265 bytes) 2024-09-13T07:03:08.658 INFO:teuthology.orchestra.run.smithi118.stdout:4239 write 0x59a879a thru 0x59aa198 (0x19ff bytes) 2024-09-13T07:03:08.661 INFO:teuthology.orchestra.run.smithi118.stdout:4240 read 0x4dbf228 thru 0x4dc2ecf (0x3ca8 bytes) 2024-09-13T07:03:08.662 INFO:teuthology.orchestra.run.smithi118.stdout:4244 writesame 0x9d815e9 thru 0x9d8c273 data_size 0x1(0xac8b bytes) 2024-09-13T07:03:08.687 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:08.684+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:08.696 INFO:teuthology.orchestra.run.smithi118.stdout:4245 read 0x9e633c4 thru 0x9e67224 (0x3e61 bytes) 2024-09-13T07:03:08.697 INFO:teuthology.orchestra.run.smithi118.stdout:4246 trunc from 0xea4b075 to 0xe49c7c4 2024-09-13T07:03:08.709 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:08.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:03:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.20425311699999, 2024-09-13T07:03:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:03:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:03:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:03:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:03:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:03:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:03:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:03:08.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:03:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:03:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:03:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:03:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:03:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.713 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:03:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:03:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:03:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:03:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:03:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:03:08.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.179425407, 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:03:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:03:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:03:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:03:08.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:03:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:03:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:03:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:03:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:08.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:03:08.721 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 102.64075358300001, 2024-09-13T07:03:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:03:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:03:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:03:08.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:03:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:03:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:03:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 102.61112839, 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.727 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:03:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:03:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:03:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:03:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:03:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:03:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:03:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:03:08.732 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 102.078414759, 2024-09-13T07:03:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:03:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.733 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:03:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:03:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:03:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:03:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:03:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:03:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:03:08.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:03:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:03:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:03:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:03:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 102.078332303, 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:03:08.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:03:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:03:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:03:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:03:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:08.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:03:08.744 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 102.078300493, 2024-09-13T07:03:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:03:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:03:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:03:08.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:03:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:03:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:03:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:03:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:03:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:03:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:03:08.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:03:08.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 102.07824533500001, 2024-09-13T07:03:08.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:03:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:03:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:03:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:03:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:03:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:03:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:03:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:03:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:03:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:03:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:03:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:03:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:03:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:03:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:03:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:03:08.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:03:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:03:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:03:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:03:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:03:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:03:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:03:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:03:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:03:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.756 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:03:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 102.078200056, 2024-09-13T07:03:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:03:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:03:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:03:08.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:03:08.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:03:08.762 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 102.078161522, 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:03:08.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:03:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:03:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:03:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:03:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:03:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:03:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:03:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:03:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 102.07809990299999, 2024-09-13T07:03:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:03:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.768 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:03:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:03:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:03:08.773 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 102.06387721999999, 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:03:08.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:08.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:03:08.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:03:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:03:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:03:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:03:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:03:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:03:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:03:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:03:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:03:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:03:08.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.779 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:03:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.051419483999993, 2024-09-13T07:03:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:03:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:03:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:03:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:03:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:03:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:08.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:03:08.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:03:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:03:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:03:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:03:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:03:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.051399767000007, 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:08.784 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:08.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:03:08.788 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.051382437000001, 2024-09-13T07:03:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:03:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:03:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:03:08.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:03:08.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:03:08.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.051086775000002, 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:03:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:03:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:03:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:03:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:03:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:03:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:03:08.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:03:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:03:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:03:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 88.489883956, 2024-09-13T07:03:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:03:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:03:08.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:03:08.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:03:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:03:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:03:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:03:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:03:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.802 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:03:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:03:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 88.457805053000001, 2024-09-13T07:03:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:03:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:08.803 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:03:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:03:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:03:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:03:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:03:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:03:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:03:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:03:08.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:03:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:03:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137474 3.6 3:65c31370:::rbd_data.1194af11889b.000000000000013b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 186185~11715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-13T07:03:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.965127844, 2024-09-13T07:03:08.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0054357140000001, 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137474 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743934+0000", 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743949+0000", 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5705999999999998e-05 2024-09-13T07:03:08.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743960+0000", 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0084e-05 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743971+0000", 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0942e-05 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744045+0000", 2024-09-13T07:03:08.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4626e-05 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744062+0000", 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7339000000000001e-05 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746366+0000", 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.002303457 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746520+0000", 2024-09-13T07:03:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015385900000000001 2024-09-13T07:03:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746734+0000", 2024-09-13T07:03:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000214253 2024-09-13T07:03:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749128+0000", 2024-09-13T07:03:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002394129 2024-09-13T07:03:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749267+0000", 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013878299999999999 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749328+0000", 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0918000000000002e-05 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749371+0000", 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3188000000000001e-05 2024-09-13T07:03:08.813 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138463 3.17 3:efe7ec01:::rbd_object_map.11949f62878b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.429734+0000", 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.27932923700000001, 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092770100000000005, 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:08.814 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138463 2024-09-13T07:03:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429734+0000", 2024-09-13T07:03:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429734+0000", 2024-09-13T07:03:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429734+0000", 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429736+0000", 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.621e-06 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429740+0000", 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0350000000000003e-06 2024-09-13T07:03:08.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429744+0000", 2024-09-13T07:03:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3909999999999998e-06 2024-09-13T07:03:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.432092+0000", 2024-09-13T07:03:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023486330000000001 2024-09-13T07:03:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.432189+0000", 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6279000000000004e-05 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.432610+0000", 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042100200000000002 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.435051+0000", 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024418500000000002 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.438887+0000", 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0038353900000000002 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.438959+0000", 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1880999999999998e-05 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.439011+0000", 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2268999999999997e-05 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.819 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:08.820 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:08.820 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:08.821 INFO:teuthology.orchestra.run.smithi118.stdout:4247 writesame 0xae65ddb thru 0xae6d2a9 data_size 0x1(0x74cf bytes) 2024-09-13T07:03:08.822 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:08.823 INFO:teuthology.orchestra.run.smithi118.stdout:4248 punch from 0x3971437 to 0x3971b73, (0x73c bytes) 2024-09-13T07:03:08.823 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:08.823 INFO:teuthology.orchestra.run.smithi118.stdout:4251 write 0xdbf8d8b thru 0xdc0122e (0x84a4 bytes) 2024-09-13T07:03:08.823 INFO:teuthology.orchestra.run.smithi118.stdout:4252 writesame 0x93a15de thru 0x93afbfc data_size 0x3(0xe61f bytes) 2024-09-13T07:03:08.824 INFO:teuthology.orchestra.run.smithi118.stdout:4254 punch from 0x512cfdd to 0x513acfe, (0xdd21 bytes) 2024-09-13T07:03:08.824 INFO:teuthology.orchestra.run.smithi118.stdout:4256 punch from 0xd35b083 to 0xd361442, (0x63bf bytes) 2024-09-13T07:03:08.824 INFO:teuthology.orchestra.run.smithi118.stdout:4257 read 0x1dde7f8 thru 0x1ddfaac (0x12b5 bytes) 2024-09-13T07:03:08.824 INFO:teuthology.orchestra.run.smithi118.stdout:4258 write 0xd532afe thru 0xd53889c (0x5d9f bytes) 2024-09-13T07:03:08.824 INFO:teuthology.orchestra.run.smithi118.stdout:4259 punch from 0xaf50b9 to 0xaf79a6, (0x28ed bytes) 2024-09-13T07:03:08.824 INFO:teuthology.orchestra.run.smithi118.stdout:4260 write 0xc124aa8 thru 0xc124d33 (0x28c bytes) 2024-09-13T07:03:08.824 INFO:teuthology.orchestra.run.smithi118.stdout:4261 write 0xa30f27c thru 0xa31106a (0x1def bytes) 2024-09-13T07:03:08.824 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:08.824 INFO:teuthology.orchestra.run.smithi118.stdout:4264 write 0x198d4e7 thru 0x19983a8 (0xaec2 bytes) 2024-09-13T07:03:08.824 INFO:teuthology.orchestra.run.smithi118.stdout:4265 write 0xadf5f9b thru 0xae03e84 (0xdeea bytes) 2024-09-13T07:03:08.824 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:03:08.825 INFO:teuthology.orchestra.run.smithi118.stdout:4266 trunc from 0xe49c7c4 to 0xc73369f 2024-09-13T07:03:08.826 INFO:teuthology.orchestra.run.smithi118.stdout:4268 read 0x2937f0f thru 0x293f607 (0x76f9 bytes) 2024-09-13T07:03:08.826 INFO:teuthology.orchestra.run.smithi118.stdout:4269 writesame 0x5599f96 thru 0x559dc03 data_size 0x41(0x3c6e bytes) 2024-09-13T07:03:08.827 INFO:teuthology.orchestra.run.smithi118.stdout:4270 write 0x974f4d3 thru 0x97560af (0x6bdd bytes) 2024-09-13T07:03:08.827 INFO:teuthology.orchestra.run.smithi118.stdout:4271 writesame 0x42116eb thru 0x421daa3 data_size 0xb(0xc3b9 bytes) 2024-09-13T07:03:08.827 INFO:teuthology.orchestra.run.smithi118.stdout:4273 write 0x51bc9c thru 0x5207b9 (0x4b1e bytes) 2024-09-13T07:03:08.827 INFO:teuthology.orchestra.run.smithi118.stdout:4274 write 0x2029522 thru 0x2032f2b (0x9a0a bytes) 2024-09-13T07:03:08.827 INFO:teuthology.orchestra.run.smithi118.stdout:4276 punch from 0xbbb9c32 to 0xbbc0c51, (0x701f bytes) 2024-09-13T07:03:08.827 INFO:teuthology.orchestra.run.smithi118.stdout:4277 writesame 0x1f5fd05 thru 0x1f61855 data_size 0x25(0x1b51 bytes) 2024-09-13T07:03:08.827 INFO:teuthology.orchestra.run.smithi118.stdout:4279 writesame 0x26389a5 thru 0x2638f02 data_size 0x3(0x55e bytes) 2024-09-13T07:03:08.827 INFO:teuthology.orchestra.run.smithi118.stdout:4280 read 0x6ed18c6 thru 0x6ee0d84 (0xf4bf bytes) 2024-09-13T07:03:08.827 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:08.820+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:08.836 INFO:teuthology.orchestra.run.smithi118.stdout:4281 writesame 0xe241779 thru 0xe24ec2e data_size 0x13(0xd4b6 bytes) 2024-09-13T07:03:08.846 INFO:teuthology.orchestra.run.smithi118.stdout:4283 trunc from 0xe24ec2f to 0xb7a1cd1 2024-09-13T07:03:08.861 INFO:teuthology.orchestra.run.smithi118.stdout:4284 read 0x50d3e42 thru 0x50d5008 (0x11c7 bytes) 2024-09-13T07:03:08.875 INFO:teuthology.orchestra.run.smithi118.stdout:4285 write 0xd33be4e thru 0xd340c57 (0x4e0a bytes) 2024-09-13T07:03:08.878 INFO:teuthology.orchestra.run.smithi118.stdout:4286 punch from 0x10fbd89 to 0x110486b, (0x8ae2 bytes) 2024-09-13T07:03:08.879 INFO:teuthology.orchestra.run.smithi118.stdout:4288 trunc from 0xd340c58 to 0xd5daf12 2024-09-13T07:03:08.912 INFO:teuthology.orchestra.run.smithi118.stdout:4290 trunc from 0xd5daf12 to 0x4f55e9 2024-09-13T07:03:08.922 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:08.920+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:08.938 INFO:teuthology.orchestra.run.smithi118.stdout:4291 write 0x7ef9bb thru 0x7fe5f2 (0xec38 bytes) 2024-09-13T07:03:08.956 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:08.956 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:08.956 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:08.956 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:08.969 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:08.971 INFO:teuthology.orchestra.run.smithi118.stdout:4292 writesame 0xe37589b thru 0xe37dd86 data_size 0x4(0x84ec bytes) 2024-09-13T07:03:08.974 INFO:teuthology.orchestra.run.smithi118.stdout:4293 write 0x4d6522 thru 0x4e080c (0xa2eb bytes) 2024-09-13T07:03:08.977 INFO:teuthology.orchestra.run.smithi118.stdout:4294 write 0x698113f thru 0x698e3b3 (0xd275 bytes) 2024-09-13T07:03:08.980 INFO:teuthology.orchestra.run.smithi118.stdout:4295 write 0x1df32dd thru 0x1df5f96 (0x2cba bytes) 2024-09-13T07:03:08.983 INFO:teuthology.orchestra.run.smithi118.stdout:4297 write 0x5b744b8 thru 0x5b80f59 (0xcaa2 bytes) 2024-09-13T07:03:08.986 INFO:teuthology.orchestra.run.smithi118.stdout:4299 read 0x1db0f2c thru 0x1db8eeb (0x7fc0 bytes) 2024-09-13T07:03:08.987 INFO:teuthology.orchestra.run.smithi118.stdout:4300 punch from 0x9984793 to 0x998781a, (0x3087 bytes) 2024-09-13T07:03:08.987 INFO:teuthology.orchestra.run.smithi118.stdout:4301 read 0xce0c218 thru 0xce0fc00 (0x39e9 bytes) 2024-09-13T07:03:08.987 INFO:teuthology.orchestra.run.smithi118.stdout:4302 read 0x27df1a3 thru 0x27e3efa (0x4d58 bytes) 2024-09-13T07:03:08.988 INFO:teuthology.orchestra.run.smithi118.stdout:4303 writesame 0x7c9d9f3 thru 0x7ca6256 data_size 0x2b(0x8864 bytes) 2024-09-13T07:03:08.992 INFO:teuthology.orchestra.run.smithi118.stdout:4304 writesame 0x9122093 thru 0x9131229 data_size 0x1(0xf197 bytes) 2024-09-13T07:03:09.024 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:09.022+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:09.038 INFO:teuthology.orchestra.run.smithi118.stdout:4305 trunc from 0xe37dd87 to 0x816411a 2024-09-13T07:03:09.053 INFO:teuthology.orchestra.run.smithi118.stdout:4308 write 0x27d91d thru 0x2886ae (0xad92 bytes) 2024-09-13T07:03:09.069 INFO:teuthology.orchestra.run.smithi118.stdout:4309 write 0xa28c156 thru 0xa29bf5e (0xfe09 bytes) 2024-09-13T07:03:09.072 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.072 INFO:teuthology.orchestra.run.smithi118.stdout:4311 punch from 0x9ed7ad9 to 0x9ee394d, (0xbe74 bytes) 2024-09-13T07:03:09.072 INFO:teuthology.orchestra.run.smithi118.stdout:4312 writesame 0x38734fb thru 0x387cc31 data_size 0x1(0x9737 bytes) 2024-09-13T07:03:09.085 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:09.085 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:09.085 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:09.085 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:09.085 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:09.095 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:09.103 INFO:teuthology.orchestra.run.smithi118.stdout:4313 write 0x2838a3c thru 0x2846d0c (0xe2d1 bytes) 2024-09-13T07:03:09.107 INFO:teuthology.orchestra.run.smithi118.stdout:4315 read 0x6f5d8a9 thru 0x6f6bf73 (0xe6cb bytes) 2024-09-13T07:03:09.124 INFO:teuthology.orchestra.run.smithi118.stdout:4316 write 0xd2ee5ee thru 0xd2f46bd (0x60d0 bytes) 2024-09-13T07:03:09.124 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:09.120+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:09.127 INFO:teuthology.orchestra.run.smithi118.stdout:4317 read 0x27bbfd6 thru 0x27c5919 (0x9944 bytes) 2024-09-13T07:03:09.142 INFO:teuthology.orchestra.run.smithi118.stdout:4318 write 0xec2b7f4 thru 0xec35d4a (0xa557 bytes) 2024-09-13T07:03:09.145 INFO:teuthology.orchestra.run.smithi118.stdout:4320 writesame 0x45db048 thru 0x45e0e7e data_size 0x1(0x5e37 bytes) 2024-09-13T07:03:09.164 INFO:teuthology.orchestra.run.smithi118.stdout:4321 write 0x9e89306 thru 0x9e95136 (0xbe31 bytes) 2024-09-13T07:03:09.185 INFO:teuthology.orchestra.run.smithi118.stdout:4322 trunc from 0xec35d4b to 0x4e571cb 2024-09-13T07:03:09.220 INFO:teuthology.orchestra.run.smithi118.stdout:4323 write 0xae4d699 thru 0xae513ea (0x3d52 bytes) 2024-09-13T07:03:09.223 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:09.223 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:09.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:09.223 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:09.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.224 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:03:09.224 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:09.224 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.729096467999995, 2024-09-13T07:03:09.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:03:09.224 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.224 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.224 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.224 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.224 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.224 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:03:09.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.224 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:09.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:09.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:03:09.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.228 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.242612903999998, 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.229 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:09.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:09.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.242542975000006, 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.233 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:03:09.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:09.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:03:09.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:03:09.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:03:09.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.242527103, 2024-09-13T07:03:09.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:03:09.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:03:09.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:03:09.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:03:09.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:03:09.241 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.241 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.241 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.242456645999994, 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:03:09.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:03:09.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:03:09.246 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.246 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.246 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.246 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.246 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:09.246 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.239104608000005, 2024-09-13T07:03:09.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:03:09.246 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.246 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.246 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.247 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.247 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.247 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:03:09.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.247 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:09.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.239088307000003, 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.250 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:09.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:03:09.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:03:09.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:03:09.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:03:09.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.239068521999997, 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:03:09.254 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:09.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:09.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:03:09.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:03:09.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:03:09.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.239052248999997, 2024-09-13T07:03:09.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:03:09.259 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.259 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.259 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.259 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.259 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.259 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:03:09.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.259 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:09.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:03:09.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:03:09.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.239034423999996, 2024-09-13T07:03:09.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:09.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:09.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:03:09.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:03:09.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:03:09.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:03:09.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:03:09.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:03:09.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.267 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.267 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:09.267 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.239018330999997, 2024-09-13T07:03:09.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:03:09.267 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.267 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.267 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:03:09.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.267 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:03:09.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:03:09.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:03:09.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:03:09.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:03:09.270 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.270 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.270 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.238997144999999, 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:03:09.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.238981112000005, 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:03:09.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:03:09.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:03:09.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:03:09.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:03:09.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.238931872999999, 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:09.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:09.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.282 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.238913812999996, 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:03:09.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:09.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:03:09.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:03:09.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:03:09.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:03:09.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:03:09.286 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.238897297999998, 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.287 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:03:09.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:03:09.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:03:09.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:03:09.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:03:09.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.238878932999995, 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.291 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.292 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:03:09.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.292 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:09.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:09.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:03:09.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:03:09.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.238862835000006, 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:03:09.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:09.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:03:09.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:03:09.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:03:09.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:03:09.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:03:09.298 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.238846899999999, 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.299 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:03:09.302 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.238830402999994, 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:03:09.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:09.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:03:09.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:03:09.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:03:09.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:03:09.307 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.307 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.307 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.307 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.307 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.307 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:09.308 INFO:teuthology.orchestra.run.smithi118.stdout:4324 trunc from 0xae513eb to 0x6511b0e 2024-09-13T07:03:09.309 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:09.310 INFO:teuthology.orchestra.run.smithi118.stdout:4325 punch from 0x39bac15 to 0x39c15fa, (0x69e5 bytes) 2024-09-13T07:03:09.310 INFO:teuthology.orchestra.run.smithi118.stdout:4326 punch from 0x5870318 to 0x587805c, (0x7d44 bytes) 2024-09-13T07:03:09.310 INFO:teuthology.orchestra.run.smithi118.stdout:4327 write 0xca7dbb2 thru 0xca88542 (0xa991 bytes) 2024-09-13T07:03:09.310 INFO:teuthology.orchestra.run.smithi118.stdout:4328 trunc from 0xca88543 to 0x2be6937 2024-09-13T07:03:09.310 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.310 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:09.306+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:09.320 INFO:teuthology.orchestra.run.smithi118.stdout:4330 trunc from 0x2be6937 to 0xd2e6c07 2024-09-13T07:03:09.332 INFO:teuthology.orchestra.run.smithi118.stdout:4331 writesame 0xca87f48 thru 0xca8972f data_size 0x9(0x17e8 bytes) 2024-09-13T07:03:09.337 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.337 INFO:teuthology.orchestra.run.smithi118.stdout:4335 write 0x1be61f6 thru 0x1bea206 (0x4011 bytes) 2024-09-13T07:03:09.340 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.340 INFO:teuthology.orchestra.run.smithi118.stdout:4337 read 0x176ab6e thru 0x17775ea (0xca7d bytes) 2024-09-13T07:03:09.341 INFO:teuthology.orchestra.run.smithi118.stdout:4340 writesame 0x81f3bd1 thru 0x81f4b96 data_size 0x3(0xfc6 bytes) 2024-09-13T07:03:09.344 INFO:teuthology.orchestra.run.smithi118.stdout:4342 write 0x22f4af9 thru 0x22f7639 (0x2b41 bytes) 2024-09-13T07:03:09.347 INFO:teuthology.orchestra.run.smithi118.stdout:4343 read 0x5bf52c2 thru 0x5bf78ab (0x25ea bytes) 2024-09-13T07:03:09.348 INFO:teuthology.orchestra.run.smithi118.stdout:4345 punch from 0x3d59a2d to 0x3d5eb74, (0x5147 bytes) 2024-09-13T07:03:09.348 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.362 INFO:teuthology.orchestra.run.smithi118.stdout:4348 trunc from 0xd2e6c07 to 0x6f49aad 2024-09-13T07:03:09.376 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.377 INFO:teuthology.orchestra.run.smithi118.stdout:4351 read 0x6f6fb5 thru 0x6fdf82 (0x6fce bytes) 2024-09-13T07:03:09.377 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.395 INFO:teuthology.orchestra.run.smithi118.stdout:4354 write 0xba373bd thru 0xba42c74 (0xb8b8 bytes) 2024-09-13T07:03:09.400 INFO:teuthology.orchestra.run.smithi118.stdout:4355 write 0x96a1d26 thru 0x96ae5cb (0xc8a6 bytes) 2024-09-13T07:03:09.410 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:09.406+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:09.420 INFO:teuthology.orchestra.run.smithi118.stdout:4356 write 0xede5f3c thru 0xede76d5 (0x179a bytes) 2024-09-13T07:03:09.423 INFO:teuthology.orchestra.run.smithi118.stdout:4357 punch from 0xde22130 to 0xde30b9c, (0xea6c bytes) 2024-09-13T07:03:09.423 INFO:teuthology.orchestra.run.smithi118.stdout:4358 read 0x7011dfc thru 0x701bddf (0x9fe4 bytes) 2024-09-13T07:03:09.424 INFO:teuthology.orchestra.run.smithi118.stdout:4359 writesame 0x39e3b7c thru 0x39f0ddd data_size 0x7(0xd262 bytes) 2024-09-13T07:03:09.431 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:09.431 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:09.431 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:09.431 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:09.436 INFO:teuthology.orchestra.run.smithi118.stdout:4360 read 0x66af6d8 thru 0x66b9634 (0x9f5d bytes) 2024-09-13T07:03:09.437 INFO:teuthology.orchestra.run.smithi118.stdout:4362 punch from 0xe6bc106 to 0xe6bd30e, (0x1208 bytes) 2024-09-13T07:03:09.441 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:09.443 INFO:teuthology.orchestra.run.smithi118.stdout:4363 trunc from 0xede76d6 to 0xbe0c5ba 2024-09-13T07:03:09.470 INFO:teuthology.orchestra.run.smithi118.stdout:4368 writesame 0xd315586 thru 0xd31dc3e data_size 0xd(0x86b9 bytes) 2024-09-13T07:03:09.473 INFO:teuthology.orchestra.run.smithi118.stdout:4369 punch from 0xa19816a to 0xa19f613, (0x74a9 bytes) 2024-09-13T07:03:09.485 INFO:teuthology.orchestra.run.smithi118.stdout:4370 trunc from 0xd31dc3f to 0x60f86ca 2024-09-13T07:03:09.503 INFO:teuthology.orchestra.run.smithi118.stdout:4374 punch from 0x3b709a7 to 0x3b78e19, (0x8472 bytes) 2024-09-13T07:03:09.503 INFO:teuthology.orchestra.run.smithi118.stdout:4377 read 0x1b30a47 thru 0x1b31d2d (0x12e7 bytes) 2024-09-13T07:03:09.503 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.503 INFO:teuthology.orchestra.run.smithi118.stdout:4379 writesame 0x2d11d3b thru 0x2d158d0 data_size 0x1d(0x3b96 bytes) 2024-09-13T07:03:09.506 INFO:teuthology.orchestra.run.smithi118.stdout:4380 read 0x58bf2fc thru 0x58c33e8 (0x40ed bytes) 2024-09-13T07:03:09.506 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.510 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:09.508+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:09.520 INFO:teuthology.orchestra.run.smithi118.stdout:4385 write 0x7c96a10 thru 0x7ca628d (0xf87e bytes) 2024-09-13T07:03:09.523 INFO:teuthology.orchestra.run.smithi118.stdout:4386 write 0x6b6ae4e thru 0x6b7a2de (0xf491 bytes) 2024-09-13T07:03:09.526 INFO:teuthology.orchestra.run.smithi118.stdout:4387 read 0x62b7445 thru 0x62c3c04 (0xc7c0 bytes) 2024-09-13T07:03:09.527 INFO:teuthology.orchestra.run.smithi118.stdout:4388 read 0x1563c8c thru 0x156907e (0x53f3 bytes) 2024-09-13T07:03:09.551 INFO:teuthology.orchestra.run.smithi118.stdout:4389 writesame 0xe9e33d5 thru 0xe9e99bf data_size 0x27(0x65eb bytes) 2024-09-13T07:03:09.554 INFO:teuthology.orchestra.run.smithi118.stdout:4390 read 0x686e1e9 thru 0x6873c28 (0x5a40 bytes) 2024-09-13T07:03:09.563 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:09.563 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:09.563 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:09.563 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:09.563 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:09.577 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:09.579 INFO:teuthology.orchestra.run.smithi118.stdout:4393 trunc from 0xe9e99c0 to 0xca0c84 2024-09-13T07:03:09.603 INFO:teuthology.orchestra.run.smithi118.stdout:4394 trunc from 0xca0c84 to 0x69526fd 2024-09-13T07:03:09.611 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:09.610+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:09.612 INFO:teuthology.orchestra.run.smithi118.stdout:4396 read 0x490d187 thru 0x491bca3 (0xeb1d bytes) 2024-09-13T07:03:09.630 INFO:teuthology.orchestra.run.smithi118.stdout:4397 write 0xa826bbb thru 0xa83563f (0xea85 bytes) 2024-09-13T07:03:09.634 INFO:teuthology.orchestra.run.smithi118.stdout:4398 read 0x9c11f9d thru 0x9c1b792 (0x97f6 bytes) 2024-09-13T07:03:09.635 INFO:teuthology.orchestra.run.smithi118.stdout:4400 write 0x4655b87 thru 0x465642b (0x8a5 bytes) 2024-09-13T07:03:09.637 INFO:teuthology.orchestra.run.smithi118.stdout:4401 writesame 0x2045473 thru 0x204f5f7 data_size 0x4d(0xa185 bytes) 2024-09-13T07:03:09.641 INFO:teuthology.orchestra.run.smithi118.stdout:4402 read 0x54f5da0 thru 0x550354f (0xd7b0 bytes) 2024-09-13T07:03:09.658 INFO:teuthology.orchestra.run.smithi118.stdout:4403 write 0xd0c51d4 thru 0xd0d44ca (0xf2f7 bytes) 2024-09-13T07:03:09.661 INFO:teuthology.orchestra.run.smithi118.stdout:4404 punch from 0xcd0bd53 to 0xcd0c9a7, (0xc54 bytes) 2024-09-13T07:03:09.671 INFO:teuthology.orchestra.run.smithi118.stdout:4405 trunc from 0xd0d44cb to 0x65daa12 2024-09-13T07:03:09.689 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.632332011, 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:03:09.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:03:09.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:03:09.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:03:09.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:03:09.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:03:09.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:03:09.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:03:09.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:03:09.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:03:09.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:09.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133467 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874255+0000", 2024-09-13T07:03:09.694 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.814668053999998, 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085814259999999996, 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133467 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874255+0000", 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874255+0000", 2024-09-13T07:03:09.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874254+0000", 2024-09-13T07:03:09.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:09.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874260+0000", 2024-09-13T07:03:09.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4430000000000002e-06 2024-09-13T07:03:09.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874284+0000", 2024-09-13T07:03:09.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4145e-05 2024-09-13T07:03:09.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874295+0000", 2024-09-13T07:03:09.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0857e-05 2024-09-13T07:03:09.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882136+0000", 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078415900000000007 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882218+0000", 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1866999999999994e-05 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882837+0000", 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061833000000000005 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.698 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133476 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.814572148, 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0096071349999999993, 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133476 2024-09-13T07:03:09.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.700 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:03:09.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:03:09.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874350+0000", 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874356+0000", 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5999999999999997e-06 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874369+0000", 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3125000000000001e-05 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874380+0000", 2024-09-13T07:03:09.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1396999999999999e-05 2024-09-13T07:03:09.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882884+0000", 2024-09-13T07:03:09.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.008503666 2024-09-13T07:03:09.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883050+0000", 2024-09-13T07:03:09.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000165926 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883958+0000", 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00090819200000000005 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133477 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1310720~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.814532928999999, 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010531547, 2024-09-13T07:03:09.703 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.704 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.704 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.704 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.704 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.704 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133477 2024-09-13T07:03:09.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.704 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:09.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:09.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:09.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:09.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874394+0000", 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7829999999999998e-06 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874407+0000", 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3050999999999999e-05 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874418+0000", 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0974e-05 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883998+0000", 2024-09-13T07:03:09.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095798310000000005 2024-09-13T07:03:09.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884148+0000", 2024-09-13T07:03:09.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014957499999999999 2024-09-13T07:03:09.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884922+0000", 2024-09-13T07:03:09.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000774127 2024-09-13T07:03:09.707 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.708 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.708 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.708 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133484 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:09.708 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:03:09.708 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.814449775, 2024-09-13T07:03:09.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011150092, 2024-09-13T07:03:09.708 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.708 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.708 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.708 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.709 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.709 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133484 2024-09-13T07:03:09.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.709 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:03:09.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:03:09.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874472+0000", 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874479+0000", 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5700000000000004e-06 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874497+0000", 2024-09-13T07:03:09.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7385999999999999e-05 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874510+0000", 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3628999999999999e-05 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884968+0000", 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010458053 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885119+0000", 2024-09-13T07:03:09.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000150932 2024-09-13T07:03:09.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.712 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.712 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885624+0000", 2024-09-13T07:03:09.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050432500000000002 2024-09-13T07:03:09.712 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.712 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.712 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.712 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.712 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.712 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133485 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:09.712 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:03:09.713 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.814398083, 2024-09-13T07:03:09.713 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011776926, 2024-09-13T07:03:09.713 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.713 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.713 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.713 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133485 2024-09-13T07:03:09.713 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874524+0000", 2024-09-13T07:03:09.714 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874530+0000", 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8599999999999998e-06 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874545+0000", 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5186e-05 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.715 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874558+0000", 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3040000000000001e-05 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885700+0000", 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011141514999999999 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885859+0000", 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015918599999999999 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.716 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886302+0000", 2024-09-13T07:03:09.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044316899999999998 2024-09-13T07:03:09.717 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.717 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.717 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.717 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.717 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.717 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133494 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:09.717 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:03:09.717 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.814262286000002, 2024-09-13T07:03:09.717 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012188055999999999, 2024-09-13T07:03:09.717 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.717 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.717 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133494 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874660+0000", 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874666+0000", 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7300000000000002e-06 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874701+0000", 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5095999999999997e-05 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874714+0000", 2024-09-13T07:03:09.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2833000000000001e-05 2024-09-13T07:03:09.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886333+0000", 2024-09-13T07:03:09.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011619078 2024-09-13T07:03:09.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886405+0000", 2024-09-13T07:03:09.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2525000000000001e-05 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886849+0000", 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000443697 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133498 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.814197524000001, 2024-09-13T07:03:09.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012651943000000001, 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133498 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874725+0000", 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874731+0000", 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0159999999999997e-06 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874788+0000", 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7021000000000002e-05 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874797+0000", 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3430000000000002e-06 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886880+0000", 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012083287 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886959+0000", 2024-09-13T07:03:09.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8582000000000001e-05 2024-09-13T07:03:09.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887378+0000", 2024-09-13T07:03:09.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041888400000000001 2024-09-13T07:03:09.725 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.725 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.725 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.725 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.814115653999998, 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:03:09.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:03:09.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.814066266000001, 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:09.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.733 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.814043559000002, 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:03:09.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:03:09.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:03:09.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:03:09.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:03:09.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:03:09.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:03:09.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:03:09.738 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.738 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.738 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.738 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:09.738 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:09.738 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.813999366000001, 2024-09-13T07:03:09.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:03:09.738 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.738 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.738 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.738 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.739 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.739 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:03:09.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.739 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:09.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:09.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:03:09.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:03:09.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:03:09.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:03:09.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:03:09.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:03:09.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:03:09.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:03:09.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:03:09.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:03:09.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:03:09.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:03:09.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:03:09.742 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.742 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.742 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133516 3.0 3:07bbab57:::rbd_data.11946d7ee8e1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.803164185, 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010861226999999999, 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133516 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.743 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:03:09.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:03:09.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885757+0000", 2024-09-13T07:03:09.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:09.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886801+0000", 2024-09-13T07:03:09.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010439550000000001 2024-09-13T07:03:09.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886827+0000", 2024-09-13T07:03:09.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5803e-05 2024-09-13T07:03:09.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886839+0000", 2024-09-13T07:03:09.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2136e-05 2024-09-13T07:03:09.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886894+0000", 2024-09-13T07:03:09.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4434000000000001e-05 2024-09-13T07:03:09.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887043+0000", 2024-09-13T07:03:09.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014903200000000001 2024-09-13T07:03:09.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:09.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887376+0000", 2024-09-13T07:03:09.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00033327399999999999 2024-09-13T07:03:09.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:09.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.894299+0000", 2024-09-13T07:03:09.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069232620000000003 2024-09-13T07:03:09.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:09.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896556+0000", 2024-09-13T07:03:09.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022571309999999999 2024-09-13T07:03:09.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:09.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896599+0000", 2024-09-13T07:03:09.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2620999999999997e-05 2024-09-13T07:03:09.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896620+0000", 2024-09-13T07:03:09.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1367999999999999e-05 2024-09-13T07:03:09.748 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.783020276999999, 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:09.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:09.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:09.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:09.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:03:09.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:03:09.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:03:09.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:03:09.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:03:09.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:03:09.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:03:09.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:03:09.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:03:09.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:03:09.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137457 3.18 3:1a596dcc:::rbd_data.1194af11889b.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T07:03:09.753 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.037398467, 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033734403000000003, 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137457 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:09.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651523+0000", 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651531+0000", 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.328e-06 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651545+0000", 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4270000000000001e-05 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651555+0000", 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5210000000000008e-06 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651626+0000", 2024-09-13T07:03:09.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1915999999999994e-05 2024-09-13T07:03:09.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651761+0000", 2024-09-13T07:03:09.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000134838 2024-09-13T07:03:09.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:09.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.658577+0000", 2024-09-13T07:03:09.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068151820000000004 2024-09-13T07:03:09.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:09.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685181+0000", 2024-09-13T07:03:09.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026604200000000001 2024-09-13T07:03:09.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:09.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685193+0000", 2024-09-13T07:03:09.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2128e-05 2024-09-13T07:03:09.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:09.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685226+0000", 2024-09-13T07:03:09.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3504999999999999e-05 2024-09-13T07:03:09.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685259+0000", 2024-09-13T07:03:09.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2866e-05 2024-09-13T07:03:09.759 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.759 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.759 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.759 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138565 3.0 3:002d0425:::rbd_data.11949f62878b.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:09.759 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:09.759 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 1.0240279750000001, 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033394489999999999, 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138565 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:09.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664893+0000", 2024-09-13T07:03:09.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:09.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664906+0000", 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2996000000000001e-05 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664936+0000", 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9609999999999999e-05 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664949+0000", 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383e-05 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665046+0000", 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6459000000000006e-05 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665259+0000", 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000213141 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.672522+0000", 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072632110000000003 2024-09-13T07:03:09.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:09.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.697982+0000", 2024-09-13T07:03:09.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025459699999999998 2024-09-13T07:03:09.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:09.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698018+0000", 2024-09-13T07:03:09.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6375999999999999e-05 2024-09-13T07:03:09.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:09.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698166+0000", 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014818700000000001 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698290+0000", 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012333999999999999 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138587 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:09.765 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.97556230799999999, 2024-09-13T07:03:09.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019224502000000001, 2024-09-13T07:03:09.766 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.766 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:09.766 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.766 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.766 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.766 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138587 2024-09-13T07:03:09.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.766 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:09.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713360+0000", 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713368+0000", 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713383+0000", 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5176e-05 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713390+0000", 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8669999999999996e-06 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713449+0000", 2024-09-13T07:03:09.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8938999999999999e-05 2024-09-13T07:03:09.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713549+0000", 2024-09-13T07:03:09.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010009 2024-09-13T07:03:09.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:09.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.716087+0000", 2024-09-13T07:03:09.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025383820000000001 2024-09-13T07:03:09.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:09.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732351+0000", 2024-09-13T07:03:09.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016263564000000001 2024-09-13T07:03:09.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:09.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732450+0000", 2024-09-13T07:03:09.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9529000000000002e-05 2024-09-13T07:03:09.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:09.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732532+0000", 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1526000000000005e-05 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732585+0000", 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3819000000000001e-05 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138687 3.18 3:1821bbb4:::rbd_data.11949f62878b.0000000000000009:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.915325+0000", 2024-09-13T07:03:09.771 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.77359797200000002, 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0081066490000000005, 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138687 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915325+0000", 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915325+0000", 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915324+0000", 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915331+0000", 2024-09-13T07:03:09.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:03:09.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915346+0000", 2024-09-13T07:03:09.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5781999999999999e-05 2024-09-13T07:03:09.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915359+0000", 2024-09-13T07:03:09.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2763e-05 2024-09-13T07:03:09.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.918914+0000", 2024-09-13T07:03:09.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035547980000000001 2024-09-13T07:03:09.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.919238+0000", 2024-09-13T07:03:09.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000324223 2024-09-13T07:03:09.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:09.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.919952+0000", 2024-09-13T07:03:09.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00071423099999999996 2024-09-13T07:03:09.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:09.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.921692+0000", 2024-09-13T07:03:09.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0017394820000000001 2024-09-13T07:03:09.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:09.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.923319+0000", 2024-09-13T07:03:09.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0016274869999999999 2024-09-13T07:03:09.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:09.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.923383+0000", 2024-09-13T07:03:09.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3317000000000004e-05 2024-09-13T07:03:09.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.923432+0000", 2024-09-13T07:03:09.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9322e-05 2024-09-13T07:03:09.778 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138688 3.18 3:181a7d86:::rbd_data.11949f62878b.000000000000000a:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.915397+0000", 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.77352617700000004, 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0082266089999999993, 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.779 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138688 2024-09-13T07:03:09.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.780 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915397+0000", 2024-09-13T07:03:09.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915397+0000", 2024-09-13T07:03:09.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915396+0000", 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915402+0000", 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0970000000000002e-06 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915418+0000", 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5766000000000001e-05 2024-09-13T07:03:09.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915431+0000", 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2677000000000001e-05 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.921758+0000", 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0063273670000000004 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.921945+0000", 2024-09-13T07:03:09.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000187479 2024-09-13T07:03:09.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:09.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.922174+0000", 2024-09-13T07:03:09.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000228855 2024-09-13T07:03:09.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:09.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.923490+0000", 2024-09-13T07:03:09.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001315891 2024-09-13T07:03:09.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:09.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.923504+0000", 2024-09-13T07:03:09.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3750999999999999e-05 2024-09-13T07:03:09.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:09.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.923569+0000", 2024-09-13T07:03:09.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5480000000000003e-05 2024-09-13T07:03:09.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.923624+0000", 2024-09-13T07:03:09.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4265999999999998e-05 2024-09-13T07:03:09.784 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138786 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.54134705299999997, 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018847452000000001, 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.785 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138786 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147575+0000", 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147583+0000", 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147601+0000", 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7961e-05 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147610+0000", 2024-09-13T07:03:09.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-13T07:03:09.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147661+0000", 2024-09-13T07:03:09.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0447e-05 2024-09-13T07:03:09.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147754+0000", 2024-09-13T07:03:09.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2974999999999997e-05 2024-09-13T07:03:09.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:09.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.150580+0000", 2024-09-13T07:03:09.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002826566 2024-09-13T07:03:09.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:09.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166135+0000", 2024-09-13T07:03:09.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015554772 2024-09-13T07:03:09.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:09.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166225+0000", 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0258000000000002e-05 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166334+0000", 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010816500000000001 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166424+0000", 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0116000000000003e-05 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.790 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.791 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.791 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:09.792 INFO:teuthology.orchestra.run.smithi118.stdout:4406 trunc from 0x65daa12 to 0xc9cacf2 2024-09-13T07:03:09.793 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:09.794 INFO:teuthology.orchestra.run.smithi118.stdout:4407 read 0x22e7875 thru 0x22eb685 (0x3e11 bytes) 2024-09-13T07:03:09.794 INFO:teuthology.orchestra.run.smithi118.stdout:4408 write 0x615715b thru 0x61664c4 (0xf36a bytes) 2024-09-13T07:03:09.794 INFO:teuthology.orchestra.run.smithi118.stdout:4409 trunc from 0xc9cacf2 to 0x3204b2f 2024-09-13T07:03:09.794 INFO:teuthology.orchestra.run.smithi118.stdout:4412 write 0xadf5dce thru 0xadfa757 (0x498a bytes) 2024-09-13T07:03:09.794 INFO:teuthology.orchestra.run.smithi118.stdout:4413 read 0x5bbabed thru 0x5bbdfcd (0x33e1 bytes) 2024-09-13T07:03:09.794 INFO:teuthology.orchestra.run.smithi118.stdout:4414 write 0x2a1659f thru 0x2a1c7da (0x623c bytes) 2024-09-13T07:03:09.794 INFO:teuthology.orchestra.run.smithi118.stdout:4416 writesame 0x7cf7648 thru 0x7cfec4e data_size 0x1(0x7607 bytes) 2024-09-13T07:03:09.794 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.794 INFO:teuthology.orchestra.run.smithi118.stdout:4418 read 0x68f186 thru 0x699706 (0xa581 bytes) 2024-09-13T07:03:09.794 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:09.788+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:09.811 INFO:teuthology.orchestra.run.smithi118.stdout:4420 write 0xdd5246c thru 0xdd54072 (0x1c07 bytes) 2024-09-13T07:03:09.814 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.814 INFO:teuthology.orchestra.run.smithi118.stdout:4422 punch from 0xb61c2e6 to 0xb6298e4, (0xd5fe bytes) 2024-09-13T07:03:09.814 INFO:teuthology.orchestra.run.smithi118.stdout:4423 punch from 0x48cd167 to 0x48db36a, (0xe203 bytes) 2024-09-13T07:03:09.815 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.815 INFO:teuthology.orchestra.run.smithi118.stdout:4425 writesame 0x51c125c thru 0x51c4ffd data_size 0x2(0x3da2 bytes) 2024-09-13T07:03:09.826 INFO:teuthology.orchestra.run.smithi118.stdout:4427 writesame 0x6065bd0 thru 0x606afd9 data_size 0x1f(0x540a bytes) 2024-09-13T07:03:09.829 INFO:teuthology.orchestra.run.smithi118.stdout:4428 read 0x35de93e thru 0x35e0dc4 (0x2487 bytes) 2024-09-13T07:03:09.829 INFO:teuthology.orchestra.run.smithi118.stdout:4429 write 0x6cdde7f thru 0x6cdf29d (0x141f bytes) 2024-09-13T07:03:09.832 INFO:teuthology.orchestra.run.smithi118.stdout:4430 punch from 0x77f81fc to 0x77f8c95, (0xa99 bytes) 2024-09-13T07:03:09.832 INFO:teuthology.orchestra.run.smithi118.stdout:4431 read 0x8633262 thru 0x86334e1 (0x280 bytes) 2024-09-13T07:03:09.833 INFO:teuthology.orchestra.run.smithi118.stdout:4432 read 0x7c531b4 thru 0x7c5f869 (0xc6b6 bytes) 2024-09-13T07:03:09.834 INFO:teuthology.orchestra.run.smithi118.stdout:4433 read 0xbf7686 thru 0xbf9832 (0x21ad bytes) 2024-09-13T07:03:09.836 INFO:teuthology.orchestra.run.smithi118.stdout:4435 trunc from 0xdd54073 to 0xe7cf0bb 2024-09-13T07:03:09.848 INFO:teuthology.orchestra.run.smithi118.stdout:4436 write 0xbddb10a thru 0xbde0a5c (0x5953 bytes) 2024-09-13T07:03:09.852 INFO:teuthology.orchestra.run.smithi118.stdout:4437 read 0xa2eb6ab thru 0xa2ebb6a (0x4c0 bytes) 2024-09-13T07:03:09.852 INFO:teuthology.orchestra.run.smithi118.stdout:4438 writesame 0x561f970 thru 0x5625eeb data_size 0x1e(0x657c bytes) 2024-09-13T07:03:09.856 INFO:teuthology.orchestra.run.smithi118.stdout:4439 punch from 0xbf87822 to 0xbf95b33, (0xe311 bytes) 2024-09-13T07:03:09.856 INFO:teuthology.orchestra.run.smithi118.stdout:4440 write 0x3ebcdf6 thru 0x3ec4a0f (0x7c1a bytes) 2024-09-13T07:03:09.859 INFO:teuthology.orchestra.run.smithi118.stdout:4441 read 0xc376603 thru 0xc3864c7 (0xfec5 bytes) 2024-09-13T07:03:09.862 INFO:teuthology.orchestra.run.smithi118.stdout:4443 trunc from 0xe7cf0bb to 0xd758571 2024-09-13T07:03:09.877 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.877 INFO:teuthology.orchestra.run.smithi118.stdout:4445 read 0x3d55d39 thru 0x3d55f2d (0x1f5 bytes) 2024-09-13T07:03:09.877 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:09.880 INFO:teuthology.orchestra.run.smithi118.stdout:4448 trunc from 0xd758571 to 0xea564e2 2024-09-13T07:03:09.893 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:09.890+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:09.903 INFO:teuthology.orchestra.run.smithi118.stdout:4449 trunc from 0xea564e2 to 0x746b1a6 2024-09-13T07:03:09.918 INFO:teuthology.orchestra.run.smithi118.stdout:4450 punch from 0x5c86f52 to 0x5c92468, (0xb516 bytes) 2024-09-13T07:03:09.922 INFO:teuthology.orchestra.run.smithi118.stdout:4451 trunc from 0x746b1a6 to 0x480d53b 2024-09-13T07:03:09.935 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:09.935 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:09.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.935 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:139228 3.19 3:98f250a1:::rbd_header.11949f62878b:head [call rbd.parent_get in=13b out=30b,call rbd.parent_overlap_get in=29b out=1b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:09.935 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.936379+0000", 2024-09-13T07:03:09.935 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 0.00050841999999999999, 2024-09-13T07:03:09.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00053614999999999999, 2024-09-13T07:03:09.935 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:09.935 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:09.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:09.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:09.936 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:09.936 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139228 2024-09-13T07:03:09.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.936 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:09.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:09.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.936379+0000", 2024-09-13T07:03:09.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:09.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.936379+0000", 2024-09-13T07:03:09.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.936378+0000", 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.936385+0000", 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4099999999999996e-06 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.936396+0000", 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1249e-05 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.936406+0000", 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0105999999999999e-05 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.936457+0000", 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0977000000000003e-05 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:09.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.936559+0000", 2024-09-13T07:03:09.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010169 2024-09-13T07:03:09.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:09.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:09.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:09.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.936920+0000", 2024-09-13T07:03:09.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00036098899999999998 2024-09-13T07:03:09.939 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.939 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:09.939 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.939 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:09.939 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:03:09.939 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 1 2024-09-13T07:03:09.939 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:09.947 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:09.957 INFO:teuthology.orchestra.run.smithi118.stdout:4452 writesame 0xb16a0ec thru 0xb178371 data_size 0xf(0xe286 bytes) 2024-09-13T07:03:09.960 INFO:teuthology.orchestra.run.smithi118.stdout:4453 trunc from 0xb178372 to 0xb5e1970 2024-09-13T07:03:09.971 INFO:teuthology.orchestra.run.smithi118.stdout:4455 write 0x4ba9137 thru 0x4bab785 (0x264f bytes) 2024-09-13T07:03:09.974 INFO:teuthology.orchestra.run.smithi118.stdout:4456 write 0xab57fb3 thru 0xab6175b (0x97a9 bytes) 2024-09-13T07:03:09.981 INFO:teuthology.orchestra.run.smithi118.stdout:4458 trunc from 0xb5e1970 to 0x94e11c6 2024-09-13T07:03:09.994 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:09.990+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:09.996 INFO:teuthology.orchestra.run.smithi118.stdout:4459 punch from 0x7e94d2 to 0x7f79e1, (0xe50f bytes) 2024-09-13T07:03:09.996 INFO:teuthology.orchestra.run.smithi118.stdout:4460 read 0x29df516 thru 0x29e66ad (0x7198 bytes) 2024-09-13T07:03:10.011 INFO:teuthology.orchestra.run.smithi118.stdout:4461 write 0xc451560 thru 0xc45d0e1 (0xbb82 bytes) 2024-09-13T07:03:10.013 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.013 INFO:teuthology.orchestra.run.smithi118.stdout:4463 punch from 0x521d3d0 to 0x52236ad, (0x62dd bytes) 2024-09-13T07:03:10.013 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.013 INFO:teuthology.orchestra.run.smithi118.stdout:4465 write 0x4577829 thru 0x4582312 (0xaaea bytes) 2024-09-13T07:03:10.016 INFO:teuthology.orchestra.run.smithi118.stdout:4467 read 0x859c2a9 thru 0x85abe70 (0xfbc8 bytes) 2024-09-13T07:03:10.017 INFO:teuthology.orchestra.run.smithi118.stdout:4468 write 0x32d8ee9 thru 0x32e5388 (0xc4a0 bytes) 2024-09-13T07:03:10.020 INFO:teuthology.orchestra.run.smithi118.stdout:4469 write 0x3ec4d4f thru 0x3ed2e7c (0xe12e bytes) 2024-09-13T07:03:10.023 INFO:teuthology.orchestra.run.smithi118.stdout:4470 writesame 0x3c97053 thru 0x3ca045b data_size 0x1(0x9409 bytes) 2024-09-13T07:03:10.050 INFO:teuthology.orchestra.run.smithi118.stdout:4472 punch from 0xb3d8f85 to 0xb3de64d, (0x56c8 bytes) 2024-09-13T07:03:10.051 INFO:teuthology.orchestra.run.smithi118.stdout:4473 write 0x149b9b2 thru 0x149d5ea (0x1c39 bytes) 2024-09-13T07:03:10.054 INFO:teuthology.orchestra.run.smithi118.stdout:4474 write 0x89617c thru 0x8a437c (0xe201 bytes) 2024-09-13T07:03:10.057 INFO:teuthology.orchestra.run.smithi118.stdout:4476 punch from 0x5aca2e2 to 0x5ad9d49, (0xfa67 bytes) 2024-09-13T07:03:10.057 INFO:teuthology.orchestra.run.smithi118.stdout:4477 writesame 0x9887ae8 thru 0x988bae4 data_size 0x1(0x3ffd bytes) 2024-09-13T07:03:10.068 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:10.068 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:10.068 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:10.068 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:10.068 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:10.073 INFO:teuthology.orchestra.run.smithi118.stdout:4478 punch from 0x36eae52 to 0x36f7142, (0xc2f0 bytes) 2024-09-13T07:03:10.074 INFO:teuthology.orchestra.run.smithi118.stdout:4479 write 0xbd0a767 thru 0xbd16665 (0xbeff bytes) 2024-09-13T07:03:10.078 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:10.079 INFO:teuthology.orchestra.run.smithi118.stdout:4483 read 0x1d9cefd thru 0x1da83e0 (0xb4e4 bytes) 2024-09-13T07:03:10.079 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.079 INFO:teuthology.orchestra.run.smithi118.stdout:4486 write 0x891a0fd thru 0x8922c74 (0x8b78 bytes) 2024-09-13T07:03:10.082 INFO:teuthology.orchestra.run.smithi118.stdout:4488 write 0x21443a1 thru 0x214b6a3 (0x7303 bytes) 2024-09-13T07:03:10.094 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:10.092+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:10.097 INFO:teuthology.orchestra.run.smithi118.stdout:4489 trunc from 0xc45d0e2 to 0x615b4ba 2024-09-13T07:03:10.111 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.112 INFO:teuthology.orchestra.run.smithi118.stdout:4491 punch from 0x3cd2137 to 0x3cd74a0, (0x5369 bytes) 2024-09-13T07:03:10.112 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.125 INFO:teuthology.orchestra.run.smithi118.stdout:4493 trunc from 0x615b4ba to 0xeb07d34 2024-09-13T07:03:10.134 INFO:teuthology.orchestra.run.smithi118.stdout:4494 write 0x9fe7fae thru 0x9fec207 (0x425a bytes) 2024-09-13T07:03:10.137 INFO:teuthology.orchestra.run.smithi118.stdout:4495 read 0x7876826 thru 0x7885be3 (0xf3be bytes) 2024-09-13T07:03:10.138 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.138 INFO:teuthology.orchestra.run.smithi118.stdout:4497 writesame 0xa845f3a thru 0xa84de63 data_size 0x29(0x7f2a bytes) 2024-09-13T07:03:10.148 INFO:teuthology.orchestra.run.smithi118.stdout:4498 trunc from 0xeb07d34 to 0xb1c80fc 2024-09-13T07:03:10.162 INFO:teuthology.orchestra.run.smithi118.stdout:4499 write 0x5cec249 thru 0x5cf9782 (0xd53a bytes) 2024-09-13T07:03:10.165 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.165 INFO:teuthology.orchestra.run.smithi118.stdout:4504 writesame 0x1d213d3 thru 0x1d28722 data_size 0x2d(0x7350 bytes) 2024-09-13T07:03:10.169 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.169 INFO:teuthology.orchestra.run.smithi118.stdout:4506 write 0x30bda15 thru 0x30bfda4 (0x2390 bytes) 2024-09-13T07:03:10.172 INFO:teuthology.orchestra.run.smithi118.stdout:4507 read 0x2cabe65 thru 0x2cb9c8d (0xde29 bytes) 2024-09-13T07:03:10.173 INFO:teuthology.orchestra.run.smithi118.stdout:4508 punch from 0xa3e5714 to 0xa3e87b3, (0x309f bytes) 2024-09-13T07:03:10.173 INFO:teuthology.orchestra.run.smithi118.stdout:4509 write 0xb09d604 thru 0xb0ad4e4 (0xfee1 bytes) 2024-09-13T07:03:10.175 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.175 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.191 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:10.191 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:10.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:10.192 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:10.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.192 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.192 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:10.192 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.069735964000003, 2024-09-13T07:03:10.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:03:10.192 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.192 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.192 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.192 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.192 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.192 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:03:10.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.193 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:10.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:10.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:03:10.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:03:10.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:03:10.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:03:10.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:03:10.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:03:10.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:03:10.196 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.196 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.196 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.069715487000003, 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:03:10.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:03:10.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:03:10.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:03:10.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:03:10.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87036 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 6963200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.063031420999998, 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042468831999999998, 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.201 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87036 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127964+0000", 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7799999999999999e-06 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127967+0000", 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9730000000000002e-06 2024-09-13T07:03:10.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127971+0000", 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4349999999999999e-06 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170202+0000", 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042230940000000002 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170293+0000", 2024-09-13T07:03:10.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0858000000000003e-05 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170431+0000", 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013821699999999999 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.063018884000002, 2024-09-13T07:03:10.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:03:10.206 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.206 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.206 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.206 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.206 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.206 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:03:10.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.206 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:10.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:10.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:10.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:10.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:03:10.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:03:10.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:03:10.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:03:10.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:03:10.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:03:10.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:03:10.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:03:10.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:03:10.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:03:10.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.063006815000001, 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.210 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:10.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:10.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:03:10.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:03:10.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:03:10.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:03:10.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:03:10.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:03:10.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:03:10.214 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.214 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.214 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.215 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.215 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:10.215 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.062994371999999, 2024-09-13T07:03:10.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:03:10.215 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.215 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.215 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.215 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.215 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.215 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:03:10.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:10.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:10.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:03:10.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:03:10.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:03:10.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:03:10.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:03:10.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.062976773999999, 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.219 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.220 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.220 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.220 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.220 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:03:10.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.220 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:10.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:03:10.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:03:10.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:03:10.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:03:10.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:03:10.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.062966070999998, 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:10.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:10.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:10.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:03:10.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:03:10.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:03:10.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:03:10.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:03:10.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:03:10.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:03:10.228 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.228 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.228 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.228 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.228 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:10.228 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.062922137999998, 2024-09-13T07:03:10.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:03:10.229 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.229 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.229 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.229 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.229 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.229 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:03:10.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.229 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:10.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:03:10.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:03:10.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.233 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.233 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:10.233 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.062902352000002, 2024-09-13T07:03:10.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:03:10.233 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.233 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.233 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.233 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.233 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.233 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:03:10.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.233 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:03:10.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:03:10.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:03:10.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:03:10.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:03:10.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:03:10.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.062886601999999, 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.237 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:03:10.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:03:10.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:03:10.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:03:10.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:03:10.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.241 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:10.242 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.062875349999999, 2024-09-13T07:03:10.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:03:10.242 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.242 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.242 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.242 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.242 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.242 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:03:10.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.242 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:03:10.245 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.062864750999999, 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.246 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.249 INFO:teuthology.orchestra.run.smithi118.stdout:4512 write 0xea41eab thru 0xea421a3 (0x2f9 bytes) 2024-09-13T07:03:10.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:03:10.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.062855257999999, 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.251 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.252 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.252 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:03:10.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.252 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:10.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:10.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:03:10.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:03:10.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:03:10.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:03:10.255 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.255 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.255 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.255 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.255 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.062845713999998, 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:03:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.062813110999997, 2024-09-13T07:03:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:03:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:03:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:03:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.062800088000003, 2024-09-13T07:03:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:03:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:03:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:03:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:03:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.059449102999999, 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:03:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:03:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:03:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:03:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:03:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:03:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:03:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:03:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.059428826000001, 2024-09-13T07:03:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:03:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:03:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:03:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:03:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:03:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.059408625000003, 2024-09-13T07:03:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:03:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:03:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:03:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:03:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:03:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:03:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:03:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:03:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:03:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:03:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:10.282 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:10.282 INFO:teuthology.orchestra.run.smithi118.stdout:4514 writesame 0xc3cdfb2 thru 0xc3d4c74 data_size 0x1(0x6cc3 bytes) 2024-09-13T07:03:10.283 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:10.284 INFO:teuthology.orchestra.run.smithi118.stdout:4515 trunc from 0xea421a4 to 0x6bc7e9c 2024-09-13T07:03:10.284 INFO:teuthology.orchestra.run.smithi118.stdout:4516 write 0xc5aaa53 thru 0xc5ba60b (0xfbb9 bytes) 2024-09-13T07:03:10.284 INFO:teuthology.orchestra.run.smithi118.stdout:4517 read 0x10d2d3b thru 0x10da035 (0x72fb bytes) 2024-09-13T07:03:10.284 INFO:teuthology.orchestra.run.smithi118.stdout:4518 read 0x43e89fa thru 0x43f898e (0xff95 bytes) 2024-09-13T07:03:10.285 INFO:teuthology.orchestra.run.smithi118.stdout:4519 write 0x8f42607 thru 0x8f4677d (0x4177 bytes) 2024-09-13T07:03:10.285 INFO:teuthology.orchestra.run.smithi118.stdout:4520 write 0x1ea9919 thru 0x1eacfad (0x3695 bytes) 2024-09-13T07:03:10.285 INFO:teuthology.orchestra.run.smithi118.stdout:4521 write 0x6c4ad51 thru 0x6c553ef (0xa69f bytes) 2024-09-13T07:03:10.285 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.286 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:10.282+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:10.293 INFO:teuthology.orchestra.run.smithi118.stdout:4523 write 0xd9098d5 thru 0xd915300 (0xba2c bytes) 2024-09-13T07:03:10.297 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.297 INFO:teuthology.orchestra.run.smithi118.stdout:4527 writesame 0x5f329a7 thru 0x5f3cb20 data_size 0xb(0xa17a bytes) 2024-09-13T07:03:10.302 INFO:teuthology.orchestra.run.smithi118.stdout:4528 punch from 0xcea2b59 to 0xceb2ac5, (0xff6c bytes) 2024-09-13T07:03:10.316 INFO:teuthology.orchestra.run.smithi118.stdout:4529 trunc from 0xd915301 to 0x6985670 2024-09-13T07:03:10.339 INFO:teuthology.orchestra.run.smithi118.stdout:4530 trunc from 0x6985670 to 0xcadd8ea 2024-09-13T07:03:10.349 INFO:teuthology.orchestra.run.smithi118.stdout:4532 write 0xbd29ea8 thru 0xbd2b49e (0x15f7 bytes) 2024-09-13T07:03:10.353 INFO:teuthology.orchestra.run.smithi118.stdout:4535 write 0x4e73cb0 thru 0x4e7c613 (0x8964 bytes) 2024-09-13T07:03:10.356 INFO:teuthology.orchestra.run.smithi118.stdout:4537 writesame 0x95380f thru 0x95cf1a data_size 0xe(0x970c bytes) 2024-09-13T07:03:10.361 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.361 INFO:teuthology.orchestra.run.smithi118.stdout:4540 punch from 0x83e5bfe to 0x83eff39, (0xa33b bytes) 2024-09-13T07:03:10.361 INFO:teuthology.orchestra.run.smithi118.stdout:4541 read 0x5b61275 thru 0x5b6fb56 (0xe8e2 bytes) 2024-09-13T07:03:10.362 INFO:teuthology.orchestra.run.smithi118.stdout:4542 write 0x49e28a8 thru 0x49eb16d (0x88c6 bytes) 2024-09-13T07:03:10.365 INFO:teuthology.orchestra.run.smithi118.stdout:4543 write 0x37a782b thru 0x37b5206 (0xd9dc bytes) 2024-09-13T07:03:10.368 INFO:teuthology.orchestra.run.smithi118.stdout:4546 write 0x3e4c257 thru 0x3e5baa2 (0xf84c bytes) 2024-09-13T07:03:10.371 INFO:teuthology.orchestra.run.smithi118.stdout:4547 punch from 0x2a467dc to 0x2a48cdb, (0x24ff bytes) 2024-09-13T07:03:10.372 INFO:teuthology.orchestra.run.smithi118.stdout:4548 read 0xb614c44 thru 0xb620ac1 (0xbe7e bytes) 2024-09-13T07:03:10.372 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.372 INFO:teuthology.orchestra.run.smithi118.stdout:4551 read 0x6db7a4a thru 0x6dbf04c (0x7603 bytes) 2024-09-13T07:03:10.373 INFO:teuthology.orchestra.run.smithi118.stdout:4552 read 0x50930bc thru 0x5093484 (0x3c9 bytes) 2024-09-13T07:03:10.387 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:10.386+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:10.387 INFO:teuthology.orchestra.run.smithi118.stdout:4553 writesame 0xd803370 thru 0xd8113b7 data_size 0x4(0xe048 bytes) 2024-09-13T07:03:10.392 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.392 INFO:teuthology.orchestra.run.smithi118.stdout:4555 writesame 0xaaf96ea thru 0xab068fc data_size 0x1(0xd213 bytes) 2024-09-13T07:03:10.403 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:10.403 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:10.403 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:10.403 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:10.413 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:10.417 INFO:teuthology.orchestra.run.smithi118.stdout:4557 read 0x2608fef thru 0x260f4fd (0x650f bytes) 2024-09-13T07:03:10.418 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.418 INFO:teuthology.orchestra.run.smithi118.stdout:4560 write 0x5759dbf thru 0x575fd32 (0x5f74 bytes) 2024-09-13T07:03:10.420 INFO:teuthology.orchestra.run.smithi118.stdout:4561 write 0xa636e05 thru 0xa63e7e2 (0x79de bytes) 2024-09-13T07:03:10.423 INFO:teuthology.orchestra.run.smithi118.stdout:4562 read 0x9046088 thru 0x9047fd2 (0x1f4b bytes) 2024-09-13T07:03:10.424 INFO:teuthology.orchestra.run.smithi118.stdout:4564 read 0xe408eb thru 0xe4f8d2 (0xefe8 bytes) 2024-09-13T07:03:10.425 INFO:teuthology.orchestra.run.smithi118.stdout:4565 read 0xcbdf6 thru 0xda3ad (0xe5b8 bytes) 2024-09-13T07:03:10.426 INFO:teuthology.orchestra.run.smithi118.stdout:4566 writesame 0x82d9105 thru 0x82e1bfc data_size 0x1(0x8af8 bytes) 2024-09-13T07:03:10.455 INFO:teuthology.orchestra.run.smithi118.stdout:4567 writesame 0x59db83d thru 0x59e7816 data_size 0xd(0xbfda bytes) 2024-09-13T07:03:10.460 INFO:teuthology.orchestra.run.smithi118.stdout:4568 read 0x7abddce thru 0x7ac3276 (0x54a9 bytes) 2024-09-13T07:03:10.479 INFO:teuthology.orchestra.run.smithi118.stdout:4569 trunc from 0xd8113b8 to 0x59f7b34 2024-09-13T07:03:10.488 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:10.486+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:10.495 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.495 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.496 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.510 INFO:teuthology.orchestra.run.smithi118.stdout:4573 write 0x85daaa4 thru 0x85dd127 (0x2684 bytes) 2024-09-13T07:03:10.513 INFO:teuthology.orchestra.run.smithi118.stdout:4576 write 0x2112023 thru 0x2113cf9 (0x1cd7 bytes) 2024-09-13T07:03:10.521 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:10.521 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:10.521 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:10.521 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:10.521 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:10.527 INFO:teuthology.orchestra.run.smithi118.stdout:4577 trunc from 0x85dd128 to 0x3adef43 2024-09-13T07:03:10.532 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:10.543 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.543 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.543 INFO:teuthology.orchestra.run.smithi118.stdout:4581 read 0x294e926 thru 0x2950e49 (0x2524 bytes) 2024-09-13T07:03:10.544 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.544 INFO:teuthology.orchestra.run.smithi118.stdout:4583 read 0x60cd63 thru 0x60f463 (0x2701 bytes) 2024-09-13T07:03:10.544 INFO:teuthology.orchestra.run.smithi118.stdout:4584 writesame 0x2262d2b thru 0x2267c17 data_size 0x9(0x4eed bytes) 2024-09-13T07:03:10.547 INFO:teuthology.orchestra.run.smithi118.stdout:4585 read 0xc4ad6f thru 0xc50281 (0x5513 bytes) 2024-09-13T07:03:10.548 INFO:teuthology.orchestra.run.smithi118.stdout:4586 read 0x32e72ba thru 0x32ef399 (0x80e0 bytes) 2024-09-13T07:03:10.549 INFO:teuthology.orchestra.run.smithi118.stdout:4587 write 0x1408268 thru 0x1416f34 (0xeccd bytes) 2024-09-13T07:03:10.551 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.551 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.551 INFO:teuthology.orchestra.run.smithi118.stdout:4595 read 0x18887f4 thru 0x1898532 (0xfd3f bytes) 2024-09-13T07:03:10.552 INFO:teuthology.orchestra.run.smithi118.stdout:4596 read 0x30ff41a thru 0x310bf8c (0xcb73 bytes) 2024-09-13T07:03:10.552 INFO:teuthology.orchestra.run.smithi118.stdout:4597 writesame 0x2e3318e thru 0x2e3dfa2 data_size 0x5(0xae15 bytes) 2024-09-13T07:03:10.559 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:03:10.562 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.562 INFO:teuthology.orchestra.run.smithi118.stdout:4601 write 0x1dbca8b thru 0x1dbe900 (0x1e76 bytes) 2024-09-13T07:03:10.564 INFO:teuthology.orchestra.run.smithi118.stdout:4602 write 0x1923564 thru 0x193017b (0xcc18 bytes) 2024-09-13T07:03:10.568 INFO:teuthology.orchestra.run.smithi118.stdout:4603 punch from 0x271ae7b to 0x272a0b0, (0xf235 bytes) 2024-09-13T07:03:10.568 INFO:teuthology.orchestra.run.smithi118.stdout:4604 punch from 0x17cc84a to 0x17da585, (0xdd3b bytes) 2024-09-13T07:03:10.568 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.588 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:10.584+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:10.594 INFO:teuthology.orchestra.run.smithi118.stdout:4606 write 0xa7ab531 thru 0xa7acf2c (0x19fc bytes) 2024-09-13T07:03:10.597 INFO:teuthology.orchestra.run.smithi118.stdout:4607 writesame 0x3a7cd19 thru 0x3a8aa6c data_size 0xa(0xdd54 bytes) 2024-09-13T07:03:10.610 INFO:teuthology.orchestra.run.smithi118.stdout:4610 trunc from 0xa7acf2d to 0xd870a78 2024-09-13T07:03:10.644 INFO:teuthology.orchestra.run.smithi118.stdout:4611 trunc from 0xd870a78 to 0x3000b3 2024-09-13T07:03:10.658 INFO:teuthology.orchestra.run.smithi118.stdout:4612 punch from 0x51e2a to 0x60206, (0xe3dc bytes) 2024-09-13T07:03:10.658 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.659 INFO:teuthology.orchestra.run.smithi118.stdout:4614 punch from 0x2f4187 to 0x2fc09e, (0x7f17 bytes) 2024-09-13T07:03:10.659 INFO:teuthology.orchestra.run.smithi118.stdout:4618 read 0x1d3dbe thru 0x1d8b6f (0x4db2 bytes) 2024-09-13T07:03:10.659 INFO:teuthology.orchestra.run.smithi118.stdout:4619 read 0xc6e5a thru 0xcb363 (0x450a bytes) 2024-09-13T07:03:10.673 INFO:teuthology.orchestra.run.smithi118.stdout:4620 write 0x2e545b9 thru 0x2e566cf (0x2117 bytes) 2024-09-13T07:03:10.674 INFO:teuthology.orchestra.run.smithi118.stdout:4621 punch from 0x59a4bb to 0x59b218, (0xd5d bytes) 2024-09-13T07:03:10.674 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.674 INFO:teuthology.orchestra.run.smithi118.stdout:4623 read 0x1e41dd6 thru 0x1e4203b (0x266 bytes) 2024-09-13T07:03:10.683 INFO:teuthology.orchestra.run.smithi118.stdout:4624 trunc from 0x2e566d0 to 0x7b292bb 2024-09-13T07:03:10.692 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:10.692 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:10.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:10.692 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:10.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.692 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.692 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:10.692 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.086161234999999, 2024-09-13T07:03:10.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:03:10.693 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.693 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.693 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:03:10.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.693 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:10.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:03:10.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:03:10.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:03:10.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:03:10.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:03:10.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:03:10.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:03:10.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.086130134000001, 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.697 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.698 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:03:10.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.698 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:10.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.701 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.086107819999999, 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.086084789000001, 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:03:10.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:03:10.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:03:10.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:03:10.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:03:10.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.086060238999998, 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:03:10.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:03:10.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.715 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.08602453, 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:10.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:10.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:03:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.720 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.085999851, 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:03:10.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:03:10.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:03:10.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:03:10.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:03:10.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:03:10.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:03:10.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:10.725 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.725 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.725 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.725 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.725 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:10.725 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.085966577000001, 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:03:10.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:03:10.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:03:10.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:03:10.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:03:10.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:03:10.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:03:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:03:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.085939789000001, 2024-09-13T07:03:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:03:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:03:10.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:03:10.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:03:10.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:03:10.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:03:10.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:03:10.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.085915235000002, 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.735 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:03:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:03:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:03:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:03:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:03:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:03:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:03:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:03:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:03:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:03:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.085885466000001, 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:03:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:03:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:03:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:03:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.085825623000002, 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:03:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:03:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:03:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:03:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:03:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:03:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:03:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:03:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:03:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.085758772999998, 2024-09-13T07:03:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:03:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:03:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:03:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:03:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:03:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:03:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:03:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.085726270999999, 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:03:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:03:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:03:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:03:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:03:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.085691650000001, 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:03:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:03:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.085641419000002, 2024-09-13T07:03:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:03:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:03:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:03:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:03:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:03:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:03:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:03:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:03:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:03:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:03:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.085593233000001, 2024-09-13T07:03:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:03:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:03:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:03:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:03:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:03:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:03:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:03:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:03:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:03:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:03:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:03:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.085566595, 2024-09-13T07:03:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:03:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:03:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:03:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:03:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:03:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:03:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:03:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:03:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.08551976, 2024-09-13T07:03:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:03:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:03:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.082165343, 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:03:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:03:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:10.787 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:10.787 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:10.686+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:10.788 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:03:10.789 INFO:teuthology.orchestra.run.smithi118.stdout:4627 write 0xde532f1 thru 0xde5a844 (0x7554 bytes) 2024-09-13T07:03:10.789 INFO:teuthology.orchestra.run.smithi118.stdout:4628 write 0xd0a3a01 thru 0xd0af6b5 (0xbcb5 bytes) 2024-09-13T07:03:10.789 INFO:teuthology.orchestra.run.smithi118.stdout:4629 write 0x1436397 thru 0x143f30e (0x8f78 bytes) 2024-09-13T07:03:10.789 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:10.790 INFO:teuthology.orchestra.run.smithi118.stdout:4630 read 0xb9b57ca thru 0xb9c087b (0xb0b2 bytes) 2024-09-13T07:03:10.790 INFO:teuthology.orchestra.run.smithi118.stdout:4631 write 0xc7b939f thru 0xc7ba024 (0xc86 bytes) 2024-09-13T07:03:10.790 INFO:teuthology.orchestra.run.smithi118.stdout:4632 writesame 0x7362462 thru 0x7372389 data_size 0x2e(0xff28 bytes) 2024-09-13T07:03:10.791 INFO:teuthology.orchestra.run.smithi118.stdout:4633 punch from 0x5917d8b to 0x591fc2d, (0x7ea2 bytes) 2024-09-13T07:03:10.791 INFO:teuthology.orchestra.run.smithi118.stdout:4634 punch from 0x301b622 to 0x301bf4d, (0x92b bytes) 2024-09-13T07:03:10.791 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.791 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.791 INFO:teuthology.orchestra.run.smithi118.stdout:4638 punch from 0xa31db79 to 0xa3202bc, (0x2743 bytes) 2024-09-13T07:03:10.791 INFO:teuthology.orchestra.run.smithi118.stdout:4639 read 0x3ba316 thru 0x3c57da (0xb4c5 bytes) 2024-09-13T07:03:10.791 INFO:teuthology.orchestra.run.smithi118.stdout:4640 writesame 0x7270ecb thru 0x7279346 data_size 0x4(0x847c bytes) 2024-09-13T07:03:10.791 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:10.792 INFO:teuthology.orchestra.run.smithi118.stdout:4641 write 0x4555392 thru 0x4563ded (0xea5c bytes) 2024-09-13T07:03:10.792 INFO:teuthology.orchestra.run.smithi118.stdout:4642 read 0xc7fe51d thru 0xc80a9e3 (0xc4c7 bytes) 2024-09-13T07:03:10.792 INFO:teuthology.orchestra.run.smithi118.stdout:4643 writesame 0xcb360f7 thru 0xcb45e11 data_size 0x1(0xfd1b bytes) 2024-09-13T07:03:10.792 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:10.793 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.793 INFO:teuthology.orchestra.run.smithi118.stdout:4645 write 0x2e5095 thru 0x2f0744 (0xb6b0 bytes) 2024-09-13T07:03:10.793 INFO:teuthology.orchestra.run.smithi118.stdout:4646 writesame 0xc264abc thru 0xc266073 data_size 0x14(0x15b8 bytes) 2024-09-13T07:03:10.793 INFO:teuthology.orchestra.run.smithi118.stdout:4647 read 0x5488c3 thru 0x551a32 (0x9170 bytes) 2024-09-13T07:03:10.793 INFO:teuthology.orchestra.run.smithi118.stdout:4648 writesame 0x36bdabf thru 0x36bf9c1 data_size 0x1(0x1f03 bytes) 2024-09-13T07:03:10.793 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:10.794 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:10.786+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:10.811 INFO:teuthology.orchestra.run.smithi118.stdout:4649 trunc from 0xde5a845 to 0x43c91ae 2024-09-13T07:03:10.816 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:10.826 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:03:10.829 INFO:teuthology.orchestra.run.smithi118.stdout:4650 punch from 0x19f2389 to 0x19f6020, (0x3c97 bytes) 2024-09-13T07:03:10.829 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.829 INFO:teuthology.orchestra.run.smithi118.stdout:4652 read 0x3bb85c6 thru 0x3bc0c71 (0x86ac bytes) 2024-09-13T07:03:10.838 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:10.842 INFO:teuthology.orchestra.run.smithi118.stdout:4653 trunc from 0x43c91ae to 0xa43b670 2024-09-13T07:03:10.862 INFO:teuthology.orchestra.run.smithi118.stdout:4654 write 0xb7d06f6 thru 0xb7d79e2 (0x72ed bytes) 2024-09-13T07:03:10.874 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:10.881 INFO:teuthology.orchestra.run.smithi118.stdout:4656 write 0xe15aada thru 0xe164462 (0x9989 bytes) 2024-09-13T07:03:10.884 INFO:teuthology.orchestra.run.smithi118.stdout:4657 write 0x9551c81 thru 0x955c7f7 (0xab77 bytes) 2024-09-13T07:03:10.887 INFO:teuthology.orchestra.run.smithi118.stdout:4659 write 0xd95550e thru 0xd957479 (0x1f6c bytes) 2024-09-13T07:03:10.890 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.890 INFO:teuthology.orchestra.run.smithi118.stdout:4661 write 0x122e268 thru 0x123d88e (0xf627 bytes) 2024-09-13T07:03:10.890 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:10.890+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:10.892 INFO:teuthology.orchestra.run.smithi118.stdout:4662 writesame 0x9166ba0 thru 0x916ca19 data_size 0x3a(0x5e7a bytes) 2024-09-13T07:03:10.895 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.901 INFO:teuthology.orchestra.run.smithi118.stdout:4664 trunc from 0xe164463 to 0xa93cfe3 2024-09-13T07:03:10.910 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:10.911 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:10.911 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:10.912 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:10.912 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:10.912 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:10.912 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:10.912 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:10.912 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:10.912 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:10.912 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:10.921 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:03:10.924 INFO:teuthology.orchestra.run.smithi118.stdout:4665 write 0xb62800a thru 0xb62c921 (0x4918 bytes) 2024-09-13T07:03:10.927 INFO:teuthology.orchestra.run.smithi118.stdout:4666 writesame 0x907cf36 thru 0x908102e data_size 0x1(0x40f9 bytes) 2024-09-13T07:03:10.954 INFO:teuthology.orchestra.run.smithi118.stdout:4667 trunc from 0xb62c922 to 0x29f3a30 2024-09-13T07:03:10.967 INFO:teuthology.orchestra.run.smithi118.stdout:4669 read 0x296ff0c thru 0x297cc9f (0xcd94 bytes) 2024-09-13T07:03:10.968 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.990 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:10.987+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:10.992 INFO:teuthology.orchestra.run.smithi118.stdout:4672 writesame 0xb297b18 thru 0xb298c65 data_size 0x1(0x114e bytes) 2024-09-13T07:03:10.994 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:10.994 INFO:teuthology.orchestra.run.smithi118.stdout:4675 punch from 0x579e2c1 to 0x57a293f, (0x467e bytes) 2024-09-13T07:03:10.994 INFO:teuthology.orchestra.run.smithi118.stdout:4676 write 0x528c810 thru 0x528f69b (0x2e8c bytes) 2024-09-13T07:03:10.997 INFO:teuthology.orchestra.run.smithi118.stdout:4677 punch from 0x20d011f to 0x20dc7bc, (0xc69d bytes) 2024-09-13T07:03:10.997 INFO:teuthology.orchestra.run.smithi118.stdout:4678 punch from 0x816fed5 to 0x817359d, (0x36c8 bytes) 2024-09-13T07:03:11.011 INFO:teuthology.orchestra.run.smithi118.stdout:4679 trunc from 0xb298c66 to 0x2eed420 2024-09-13T07:03:11.025 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:11.025 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:11.026 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:11.027 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:11.028 INFO:teuthology.orchestra.run.smithi118.stdout:4684 trunc from 0x2eed420 to 0x13dc00a 2024-09-13T07:03:11.038 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:03:11.066 INFO:teuthology.orchestra.run.smithi118.stdout:4686 write 0xbfee4f7 thru 0xbffbd20 (0xd82a bytes) 2024-09-13T07:03:11.069 INFO:teuthology.orchestra.run.smithi118.stdout:4687 read 0xb836437 thru 0xb83a83a (0x4404 bytes) 2024-09-13T07:03:11.070 INFO:teuthology.orchestra.run.smithi118.stdout:4688 write 0x7702308 thru 0x7708259 (0x5f52 bytes) 2024-09-13T07:03:11.072 INFO:teuthology.orchestra.run.smithi118.stdout:4693 write 0x83c82ec thru 0x83c8933 (0x648 bytes) 2024-09-13T07:03:11.075 INFO:teuthology.orchestra.run.smithi118.stdout:4694 writesame 0x33214ba thru 0x332cd9f data_size 0x62(0xb8e6 bytes) 2024-09-13T07:03:11.078 INFO:teuthology.orchestra.run.smithi118.stdout:4695 punch from 0x6dbd43c to 0x6dbf7e2, (0x23a6 bytes) 2024-09-13T07:03:11.078 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:11.078 INFO:teuthology.orchestra.run.smithi118.stdout:4697 write 0x10710c8 thru 0x1071fa3 (0xedc bytes) 2024-09-13T07:03:11.079 INFO:teuthology.orchestra.run.smithi118.stdout:4698 writesame 0x7c5e2ff thru 0x7c637e2 data_size 0x6(0x54e4 bytes) 2024-09-13T07:03:11.085 INFO:teuthology.orchestra.run.smithi118.stdout:4701 punch from 0x87fb840 to 0x88077ef, (0xbfaf bytes) 2024-09-13T07:03:11.085 INFO:teuthology.orchestra.run.smithi118.stdout:4702 writesame 0x9518d90 thru 0x9523320 data_size 0x31(0xa591 bytes) 2024-09-13T07:03:11.089 INFO:teuthology.orchestra.run.smithi118.stdout:4703 trunc from 0xbffbd21 to 0xb341d22 2024-09-13T07:03:11.092 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:11.087+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:11.103 INFO:teuthology.orchestra.run.smithi118.stdout:4705 punch from 0x8f8a235 to 0x8f90768, (0x6533 bytes) 2024-09-13T07:03:11.103 INFO:teuthology.orchestra.run.smithi118.stdout:4706 write 0x4b0f876 thru 0x4b16aeb (0x7276 bytes) 2024-09-13T07:03:11.105 INFO:teuthology.orchestra.run.smithi118.stdout:4707 read 0x7c7781f thru 0x7c81bbc (0xa39e bytes) 2024-09-13T07:03:11.109 INFO:teuthology.orchestra.run.smithi118.stdout:4708 trunc from 0xb341d22 to 0xd902fe6 2024-09-13T07:03:11.120 INFO:teuthology.orchestra.run.smithi118.stdout:4709 write 0x2e4192c thru 0x2e4ac02 (0x92d7 bytes) 2024-09-13T07:03:11.123 INFO:teuthology.orchestra.run.smithi118.stdout:4710 write 0x8a84756 thru 0x8a8f7f6 (0xb0a1 bytes) 2024-09-13T07:03:11.136 INFO:teuthology.orchestra.run.smithi118.stdout:4711 write 0xded36aa thru 0xded38b3 (0x20a bytes) 2024-09-13T07:03:11.138 INFO:teuthology.orchestra.run.smithi118.stdout:4713 punch from 0xdea7f33 to 0xdeaf988, (0x7a55 bytes) 2024-09-13T07:03:11.139 INFO:teuthology.orchestra.run.smithi118.stdout:4714 writesame 0x8025207 thru 0x8032791 data_size 0x1(0xd58b bytes) 2024-09-13T07:03:11.141 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:11.141 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:11.151 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:11.172 INFO:teuthology.orchestra.run.smithi118.stdout:4716 writesame 0x1e65694 thru 0x1e73c3b data_size 0x4(0xe5a8 bytes) 2024-09-13T07:03:11.185 INFO:teuthology.orchestra.run.smithi118.stdout:4717 write 0x8e0c950 thru 0x8e0e6d0 (0x1d81 bytes) 2024-09-13T07:03:11.193 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:11.190+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:11.202 INFO:teuthology.orchestra.run.smithi118.stdout:4719 write 0xebbf4d3 thru 0xebc7cc0 (0x87ee bytes) 2024-09-13T07:03:11.208 INFO:teuthology.orchestra.run.smithi118.stdout:4720 read 0x8dc48fe thru 0x8dd2614 (0xdd17 bytes) 2024-09-13T07:03:11.209 INFO:teuthology.orchestra.run.smithi118.stdout:4721 read 0xb0e1536 thru 0xb0ec31f (0xadea bytes) 2024-09-13T07:03:11.209 INFO:teuthology.orchestra.run.smithi118.stdout:4722 read 0x41a14fc thru 0x41a46e5 (0x31ea bytes) 2024-09-13T07:03:11.209 INFO:teuthology.orchestra.run.smithi118.stdout:4723 read 0x6e3532f thru 0x6e43493 (0xe165 bytes) 2024-09-13T07:03:11.210 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:11.211 INFO:teuthology.orchestra.run.smithi118.stdout:4727 write 0xe39ebd4 thru 0xe3aea86 (0xfeb3 bytes) 2024-09-13T07:03:11.213 INFO:teuthology.orchestra.run.smithi118.stdout:4728 read 0x1dfd691 thru 0x1e0bc80 (0xe5f0 bytes) 2024-09-13T07:03:11.215 INFO:teuthology.orchestra.run.smithi118.stdout:4730 punch from 0x4ad685f to 0x4ae5a5e, (0xf1ff bytes) 2024-09-13T07:03:11.242 INFO:teuthology.orchestra.run.smithi118.stdout:4731 trunc from 0xebc7cc1 to 0x1a0d 2024-09-13T07:03:11.259 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:11.260 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:11.271 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:11.276 INFO:teuthology.orchestra.run.smithi118.stdout:4733 read 0x16fc thru 0x1a0c (0x311 bytes) 2024-09-13T07:03:11.276 INFO:teuthology.orchestra.run.smithi118.stdout:4735 read 0x183c thru 0x1a0c (0x1d1 bytes) 2024-09-13T07:03:11.277 INFO:teuthology.orchestra.run.smithi118.stdout:4736 read 0x728 thru 0x1a0c (0x12e5 bytes) 2024-09-13T07:03:11.293 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:11.291+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:11.306 INFO:teuthology.orchestra.run.smithi118.stdout:4737 writesame 0x9f1ff89 thru 0x9f28904 data_size 0x2a(0x897c bytes) 2024-09-13T07:03:11.307 INFO:teuthology.orchestra.run.smithi118.stdout:4740 read 0x6c8635d thru 0x6c8a9f6 (0x469a bytes) 2024-09-13T07:03:11.307 INFO:teuthology.orchestra.run.smithi118.stdout:4741 read 0x6126050 thru 0x6127d5c (0x1d0d bytes) 2024-09-13T07:03:11.325 INFO:teuthology.orchestra.run.smithi118.stdout:4742 write 0xdbc3f49 thru 0xdbcbe71 (0x7f29 bytes) 2024-09-13T07:03:11.330 INFO:teuthology.orchestra.run.smithi118.stdout:4743 write 0x3ffed9a thru 0x4005731 (0x6998 bytes) 2024-09-13T07:03:11.340 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:11.340 INFO:teuthology.orchestra.run.smithi118.stdout:4745 read 0x1bcba5 thru 0x1c0f00 (0x435c bytes) 2024-09-13T07:03:11.340 INFO:teuthology.orchestra.run.smithi118.stdout:4746 punch from 0xc6cdc85 to 0xc6d1a71, (0x3dec bytes) 2024-09-13T07:03:11.340 INFO:teuthology.orchestra.run.smithi118.stdout:4747 punch from 0x78a7187 to 0x78ac130, (0x4fa9 bytes) 2024-09-13T07:03:11.340 INFO:teuthology.orchestra.run.smithi118.stdout:4748 read 0xcbe1ecd thru 0xcbeb388 (0x94bc bytes) 2024-09-13T07:03:11.340 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:11.349 INFO:teuthology.orchestra.run.smithi118.stdout:4752 trunc from 0xdbcbe72 to 0x9dd2c5b 2024-09-13T07:03:11.363 INFO:teuthology.orchestra.run.smithi118.stdout:4754 read 0x993193c thru 0x993da05 (0xc0ca bytes) 2024-09-13T07:03:11.363 INFO:teuthology.orchestra.run.smithi118.stdout:4756 writesame 0x2e13239 thru 0x2e13d42 data_size 0x9(0xb0a bytes) 2024-09-13T07:03:11.368 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:11.369 INFO:teuthology.orchestra.run.smithi118.stdout:4758 punch from 0x59b7540 to 0x59c0235, (0x8cf5 bytes) 2024-09-13T07:03:11.369 INFO:teuthology.orchestra.run.smithi118.stdout:4759 write 0xe3a9d thru 0xf28eb (0xee4f bytes) 2024-09-13T07:03:11.372 INFO:teuthology.orchestra.run.smithi118.stdout:4760 punch from 0x78f0e75 to 0x78fe5de, (0xd769 bytes) 2024-09-13T07:03:11.372 INFO:teuthology.orchestra.run.smithi118.stdout:4761 read 0x5ec2e06 thru 0x5ed0ad0 (0xdccb bytes) 2024-09-13T07:03:11.372 INFO:teuthology.orchestra.run.smithi118.stdout:4762 write 0x388a6a7 thru 0x3896e37 (0xc791 bytes) 2024-09-13T07:03:11.373 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:11.374 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:11.378 INFO:teuthology.orchestra.run.smithi118.stdout:4763 punch from 0x91a42a9 to 0x91a4b05, (0x85c bytes) 2024-09-13T07:03:11.378 INFO:teuthology.orchestra.run.smithi118.stdout:4765 write 0x8466ed3 thru 0x846f79d (0x88cb bytes) 2024-09-13T07:03:11.384 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:11.384 INFO:teuthology.orchestra.run.smithi118.stdout:4767 read 0x3ed3b16 thru 0x3edcf15 (0x9400 bytes) 2024-09-13T07:03:11.394 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:11.391+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:11.405 INFO:teuthology.orchestra.run.smithi118.stdout:4768 write 0xd8600f8 thru 0xd86afd3 (0xaedc bytes) 2024-09-13T07:03:11.427 INFO:teuthology.orchestra.run.smithi118.stdout:4770 writesame 0xe7b9e35 thru 0xe7bf0ab data_size 0x1f(0x5277 bytes) 2024-09-13T07:03:11.430 INFO:teuthology.orchestra.run.smithi118.stdout:4771 write 0x54ca3dd thru 0x54d7985 (0xd5a9 bytes) 2024-09-13T07:03:11.437 INFO:teuthology.orchestra.run.smithi118.stdout:4772 write 0x104c9b thru 0x11411c (0xf482 bytes) 2024-09-13T07:03:11.443 INFO:teuthology.orchestra.run.smithi118.stdout:4773 punch from 0x1d8dedb to 0x1d91deb, (0x3f10 bytes) 2024-09-13T07:03:11.450 INFO:teuthology.orchestra.run.smithi118.stdout:4774 trunc from 0xe7bf0ac to 0xbec1986 2024-09-13T07:03:11.479 INFO:teuthology.orchestra.run.smithi118.stdout:4775 trunc from 0xbec1986 to 0x4c841c2 2024-09-13T07:03:11.492 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:11.492 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:11.495 INFO:teuthology.orchestra.run.smithi118.stdout:4776 punch from 0x14207ac to 0x1425914, (0x5168 bytes) 2024-09-13T07:03:11.495 INFO:teuthology.orchestra.run.smithi118.stdout:4778 read 0x3c8db54 thru 0x3c9b4cb (0xd978 bytes) 2024-09-13T07:03:11.495 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:11.494+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:11.495 INFO:teuthology.orchestra.run.smithi118.stdout:4779 read 0x170f659 thru 0x17140de (0x4a86 bytes) 2024-09-13T07:03:11.496 INFO:teuthology.orchestra.run.smithi118.stdout:4780 read 0x213b1da thru 0x213e587 (0x33ae bytes) 2024-09-13T07:03:11.502 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:11.524 INFO:teuthology.orchestra.run.smithi118.stdout:4781 write 0xd8be0cd thru 0xd8c4943 (0x6877 bytes) 2024-09-13T07:03:11.527 INFO:teuthology.orchestra.run.smithi118.stdout:4783 punch from 0x8b07121 to 0x8b13a87, (0xc966 bytes) 2024-09-13T07:03:11.527 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:11.527 INFO:teuthology.orchestra.run.smithi118.stdout:4785 writesame 0xabb70e5 thru 0xabc2881 data_size 0x11(0xb79d bytes) 2024-09-13T07:03:11.533 INFO:teuthology.orchestra.run.smithi118.stdout:4786 write 0x41adb47 thru 0x41b0ddf (0x3299 bytes) 2024-09-13T07:03:11.535 INFO:teuthology.orchestra.run.smithi118.stdout:4789 write 0x7f62439 thru 0x7f6522a (0x2df2 bytes) 2024-09-13T07:03:11.538 INFO:teuthology.orchestra.run.smithi118.stdout:4790 read 0x608b9f3 thru 0x60989b1 (0xcfbf bytes) 2024-09-13T07:03:11.539 INFO:teuthology.orchestra.run.smithi118.stdout:4791 write 0x194844d thru 0x1958317 (0xfecb bytes) 2024-09-13T07:03:11.550 INFO:teuthology.orchestra.run.smithi118.stdout:4792 trunc from 0xd8c4944 to 0x9943e82 2024-09-13T07:03:11.567 INFO:teuthology.orchestra.run.smithi118.stdout:4794 read 0x8548484 thru 0x854d02f (0x4bac bytes) 2024-09-13T07:03:11.582 INFO:teuthology.orchestra.run.smithi118.stdout:4795 write 0xb9d7bbd thru 0xb9e3ac8 (0xbf0c bytes) 2024-09-13T07:03:11.586 INFO:teuthology.orchestra.run.smithi118.stdout:4796 writesame 0x33877a4 thru 0x3396679 data_size 0x13(0xeed6 bytes) 2024-09-13T07:03:11.592 INFO:teuthology.orchestra.run.smithi118.stdout:4797 punch from 0x1c95a6f to 0x1c99818, (0x3da9 bytes) 2024-09-13T07:03:11.592 INFO:teuthology.orchestra.run.smithi118.stdout:4798 read 0x1cf18c0 thru 0x1cfd615 (0xbd56 bytes) 2024-09-13T07:03:11.593 INFO:teuthology.orchestra.run.smithi118.stdout:4800 writesame 0x5ee1f2b thru 0x5eedf94 data_size 0xb(0xc06a bytes) 2024-09-13T07:03:11.596 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:11.594+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:11.598 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:11.604 INFO:teuthology.orchestra.run.smithi118.stdout:4802 trunc from 0xb9e3ac9 to 0xddc69d8 2024-09-13T07:03:11.615 INFO:teuthology.orchestra.run.smithi118.stdout:4803 punch from 0xbe6bf12 to 0xbe72e43, (0x6f31 bytes) 2024-09-13T07:03:11.615 INFO:teuthology.orchestra.run.smithi118.stdout:4804 clone 41 order 19 su 4096 sc 13 2024-09-13T07:03:11.618 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:11.618 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:11.627 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:11.697 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:11.695+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:11.731 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:11.731 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:11.742 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:11.790 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:03:11.798 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:11.795+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:11.866 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:11.866 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:11.877 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:11.899 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:11.895+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:11.992 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:11.992 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:12.000 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:11.995+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:12.003 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:12.094 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-13T07:03:12.100 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:12.095+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:12.157 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:12.158 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:12.168 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:12.201 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:12.198+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:12.273 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:12.273 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:12.283 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:12.294 INFO:teuthology.orchestra.run.smithi118.stdout:4805 write 0xa6bd5cd thru 0xa6c4e37 (0x786b bytes) 2024-09-13T07:03:12.302 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:12.299+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:12.380 INFO:teuthology.orchestra.run.smithi118.stdout:4807 writesame 0x12d6d08 thru 0x12e0ee6 data_size 0x13(0xa1df bytes) 2024-09-13T07:03:12.402 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:12.402+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:12.428 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:12.429 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:12.442 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:12.496 INFO:teuthology.orchestra.run.smithi118.stdout:4808 read 0xae4d9bc thru 0xae4e223 (0x868 bytes) 2024-09-13T07:03:12.496 INFO:teuthology.orchestra.run.smithi118.stdout:4809 read 0x4c1bce2 thru 0x4c29dac (0xe0cb bytes) 2024-09-13T07:03:12.503 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:12.499+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:12.523 INFO:teuthology.orchestra.run.smithi118.stdout:4810 trunc from 0xddc69d8 to 0x2eaae6 2024-09-13T07:03:12.572 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:12.572 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:12.585 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:12.629 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:12.627+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:12.682 INFO:teuthology.orchestra.run.smithi118.stdout:4811 writesame 0x52388cf thru 0x5240007 data_size 0x1(0x7739 bytes) 2024-09-13T07:03:12.682 INFO:teuthology.orchestra.run.smithi118.stdout:4814 read 0x5ddea0 thru 0x5e5c8c (0x7ded bytes) 2024-09-13T07:03:12.683 INFO:teuthology.orchestra.run.smithi118.stdout:4816 write 0x50d5867 thru 0x50e1a2e (0xc1c8 bytes) 2024-09-13T07:03:12.705 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:12.703+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:12.709 INFO:teuthology.orchestra.run.smithi118.stdout:4817 read 0x30d9a1e thru 0x30e428c (0xa86f bytes) 2024-09-13T07:03:12.709 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:12.709 INFO:teuthology.orchestra.run.smithi118.stdout:4820 write 0xd76a2d thru 0xd8508e (0xe662 bytes) 2024-09-13T07:03:12.730 INFO:teuthology.orchestra.run.smithi118.stdout:4821 read 0x40ad7e3 thru 0x40b2df8 (0x5616 bytes) 2024-09-13T07:03:12.730 INFO:teuthology.orchestra.run.smithi118.stdout:4822 read 0xa56023 thru 0xa64b45 (0xeb23 bytes) 2024-09-13T07:03:12.730 INFO:teuthology.orchestra.run.smithi118.stdout:4825 write 0xe2e2c1 thru 0xe3b739 (0xd479 bytes) 2024-09-13T07:03:12.736 INFO:teuthology.orchestra.run.smithi118.stdout:4826 read 0xa15c08 thru 0xa21404 (0xb7fd bytes) 2024-09-13T07:03:12.736 INFO:teuthology.orchestra.run.smithi118.stdout:4827 writesame 0x17ed944 thru 0x17f391d data_size 0x1(0x5fda bytes) 2024-09-13T07:03:12.743 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:12.743 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:12.753 INFO:teuthology.orchestra.run.smithi118.stdout:4828 writesame 0x396ec58 thru 0x39757cd data_size 0x46(0x6b76 bytes) 2024-09-13T07:03:12.754 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:12.765 INFO:teuthology.orchestra.run.smithi118.stdout:4831 writesame 0x330850f thru 0x3310841 data_size 0x1(0x8333 bytes) 2024-09-13T07:03:12.781 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 7, 2, 0] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:03:12.781 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:03:12.781 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T07:03:12.782 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T07:03:12.807 INFO:teuthology.orchestra.run.smithi118.stdout:4832 writesame 0xeacd075 thru 0xead456e data_size 0x45(0x74fa bytes) 2024-09-13T07:03:12.808 INFO:teuthology.orchestra.run.smithi118.stdout:4833 write 0x3d43057 thru 0x3d504ad (0xd457 bytes) 2024-09-13T07:03:12.808 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:12.803+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:12.835 INFO:teuthology.orchestra.run.smithi118.stdout:4834 writesame 0x30bb44d thru 0x30c2017 data_size 0x1(0x6bcb bytes) 2024-09-13T07:03:12.853 INFO:teuthology.orchestra.run.smithi118.stdout:4835 write 0xb0c52be thru 0xb0c764b (0x238e bytes) 2024-09-13T07:03:12.859 INFO:teuthology.orchestra.run.smithi118.stdout:4836 writesame 0xd45a8b5 thru 0xd462668 data_size 0xa(0x7db4 bytes) 2024-09-13T07:03:12.869 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:12.869 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:12.876 INFO:teuthology.orchestra.run.smithi118.stdout:4837 read 0x34b71e7 thru 0x34b801c (0xe36 bytes) 2024-09-13T07:03:12.877 INFO:teuthology.orchestra.run.smithi118.stdout:4838 read 0x9fc673d thru 0x9fce072 (0x7936 bytes) 2024-09-13T07:03:12.877 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:12.877 INFO:teuthology.orchestra.run.smithi118.stdout:4841 write 0xc9fa005 thru 0xca05849 (0xb845 bytes) 2024-09-13T07:03:12.881 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:12.898 INFO:teuthology.orchestra.run.smithi118.stdout:4843 read 0xc910ddb thru 0xc914b4d (0x3d73 bytes) 2024-09-13T07:03:12.898 INFO:teuthology.orchestra.run.smithi118.stdout:4844 read 0x4eaa82b thru 0x4eb4102 (0x98d8 bytes) 2024-09-13T07:03:12.900 INFO:teuthology.orchestra.run.smithi118.stdout:4845 writesame 0x47df21a thru 0x47ea832 data_size 0x29(0xb619 bytes) 2024-09-13T07:03:12.907 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:12.906+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:12.919 INFO:teuthology.orchestra.run.smithi118.stdout:4846 punch from 0x1fe0683 to 0x1fe3916, (0x3293 bytes) 2024-09-13T07:03:12.920 INFO:teuthology.orchestra.run.smithi118.stdout:4847 write 0x9a283c1 thru 0x9a318e4 (0x9524 bytes) 2024-09-13T07:03:12.924 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:03:12.937 INFO:teuthology.orchestra.run.smithi118.stdout:4848 write 0x70f3297 thru 0x70ffd25 (0xca8f bytes) 2024-09-13T07:03:12.959 INFO:teuthology.orchestra.run.smithi118.stdout:4849 punch from 0x7cd347d to 0x7ce0d74, (0xd8f7 bytes) 2024-09-13T07:03:12.959 INFO:teuthology.orchestra.run.smithi118.stdout:4850 punch from 0xbd58200 to 0xbd5a3c4, (0x21c4 bytes) 2024-09-13T07:03:12.959 INFO:teuthology.orchestra.run.smithi118.stdout:4851 write 0x6cff625 thru 0x6d008e7 (0x12c3 bytes) 2024-09-13T07:03:12.965 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:12.965 INFO:teuthology.orchestra.run.smithi118.stdout:4854 write 0x32d49c3 thru 0x32e2c28 (0xe266 bytes) 2024-09-13T07:03:12.970 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:12.970 INFO:teuthology.orchestra.run.smithi118.stdout:4856 write 0xe6f98a6 thru 0xe6fb765 (0x1ec0 bytes) 2024-09-13T07:03:12.977 INFO:teuthology.orchestra.run.smithi118.stdout:4857 read 0xd99833c thru 0xd9a7995 (0xf65a bytes) 2024-09-13T07:03:12.977 INFO:teuthology.orchestra.run.smithi118.stdout:4858 read 0x2adea39 thru 0x2ae435c (0x5924 bytes) 2024-09-13T07:03:12.977 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:12.977 INFO:teuthology.orchestra.run.smithi118.stdout:4861 read 0xbdc6f15 thru 0xbdcb74d (0x4839 bytes) 2024-09-13T07:03:12.984 INFO:teuthology.orchestra.run.smithi118.stdout:4862 trunc from 0xead456f to 0xa0b9d41 2024-09-13T07:03:13.006 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:13.007 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:13.003+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:13.017 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:13.017 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:13.018 INFO:teuthology.orchestra.run.smithi118.stdout:4864 write 0xbf88a7c thru 0xbf93b65 (0xb0ea bytes) 2024-09-13T07:03:13.029 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:13.039 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:13.048 INFO:teuthology.orchestra.run.smithi118.stdout:4866 write 0xcb2a276 thru 0xcb30424 (0x61af bytes) 2024-09-13T07:03:13.061 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:13.061 INFO:teuthology.orchestra.run.smithi118.stdout:4869 write 0x77688a9 thru 0x77734ce (0xac26 bytes) 2024-09-13T07:03:13.073 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:13.082 INFO:teuthology.orchestra.run.smithi118.stdout:4870 punch from 0x2dde85d to 0x2de57f8, (0x6f9b bytes) 2024-09-13T07:03:13.082 INFO:teuthology.orchestra.run.smithi118.stdout:4872 write 0x3864779 thru 0x3868202 (0x3a8a bytes) 2024-09-13T07:03:13.090 INFO:teuthology.orchestra.run.smithi118.stdout:4873 writesame 0xb257e43 thru 0xb26086d data_size 0x1f(0x8a2b bytes) 2024-09-13T07:03:13.099 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:13.108 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:13.103+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:13.109 INFO:teuthology.orchestra.run.smithi118.stdout:4875 read 0x78baf75 thru 0x78be122 (0x31ae bytes) 2024-09-13T07:03:13.110 INFO:teuthology.orchestra.run.smithi118.stdout:4876 punch from 0x28e190a to 0x28e2e86, (0x157c bytes) 2024-09-13T07:03:13.110 INFO:teuthology.orchestra.run.smithi118.stdout:4877 read 0x1d752d5 thru 0x1d84961 (0xf68d bytes) 2024-09-13T07:03:13.110 INFO:teuthology.orchestra.run.smithi118.stdout:4879 write 0x1f35ad4 thru 0x1f3a26d (0x479a bytes) 2024-09-13T07:03:13.120 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:13.120 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:13.120 INFO:teuthology.orchestra.run.smithi118.stdout:4882 punch from 0x37326da to 0x373a0fd, (0x7a23 bytes) 2024-09-13T07:03:13.121 INFO:teuthology.orchestra.run.smithi118.stdout:4883 punch from 0xa3d5bbf to 0xa3dd581, (0x79c2 bytes) 2024-09-13T07:03:13.124 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:13.131 INFO:teuthology.orchestra.run.smithi118.stdout:4885 write 0xea2bf48 thru 0xea3167a (0x5733 bytes) 2024-09-13T07:03:13.133 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:03:13.133 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:13.133 INFO:teuthology.orchestra.run.smithi118.stdout: "epoch": 121, 2024-09-13T07:03:13.133 INFO:teuthology.orchestra.run.smithi118.stdout: "fsid": "25bb30d1-9541-4e40-8659-18d83bb1bd8a", 2024-09-13T07:03:13.133 INFO:teuthology.orchestra.run.smithi118.stdout: "created": "2024-09-13T07:00:49.323883+0000", 2024-09-13T07:03:13.133 INFO:teuthology.orchestra.run.smithi118.stdout: "modified": "2024-09-13T07:03:11.775838+0000", 2024-09-13T07:03:13.133 INFO:teuthology.orchestra.run.smithi118.stdout: "last_up_change": "2024-09-13T07:01:00.146832+0000", 2024-09-13T07:03:13.133 INFO:teuthology.orchestra.run.smithi118.stdout: "last_in_change": "2024-09-13T07:03:07.751711+0000", 2024-09-13T07:03:13.133 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T07:03:13.133 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_num": 5804032, 2024-09-13T07:03:13.133 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_set": [ 2024-09-13T07:03:13.134 INFO:teuthology.orchestra.run.smithi118.stdout: "nodeep-scrub", 2024-09-13T07:03:13.134 INFO:teuthology.orchestra.run.smithi118.stdout: "pglog_hardlimit", 2024-09-13T07:03:13.134 INFO:teuthology.orchestra.run.smithi118.stdout: "purged_snapdirs", 2024-09-13T07:03:13.134 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_deletes", 2024-09-13T07:03:13.134 INFO:teuthology.orchestra.run.smithi118.stdout: "sortbitwise" 2024-09-13T07:03:13.134 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:03:13.134 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_version": 4, 2024-09-13T07:03:13.134 INFO:teuthology.orchestra.run.smithi118.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T07:03:13.134 INFO:teuthology.orchestra.run.smithi118.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T07:03:13.134 INFO:teuthology.orchestra.run.smithi118.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T07:03:13.134 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_snapshot": "", 2024-09-13T07:03:13.134 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_max": 3, 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: "max_osd": 8, 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: "require_min_compat_client": "luminous", 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: "min_compat_client": "luminous", 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: "require_osd_release": "reef", 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: "allow_crimson": false, 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: "pools": [ 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 1, 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": ".mgr", 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:00.675365+0000", 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 1, 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool", 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:03:13.135 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:03:13.136 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:03:13.136 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:03:13.136 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:03:13.136 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:03:13.136 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:03:13.136 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:03:13.136 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:03:13.136 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:03:13.136 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 1, 2024-09-13T07:03:13.136 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 1, 2024-09-13T07:03:13.136 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 1, 2024-09-13T07:03:13.136 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 1, 2024-09-13T07:03:13.136 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 1, 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "16", 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:03:13.137 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 0, 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 0, 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:03:13.138 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:03:13.139 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:03:13.139 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:03:13.139 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:03:13.139 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:03:13.139 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:03:13.139 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:03:13.139 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:03:13.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.139 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:03:13.139 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:03:13.139 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:03:13.139 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:03:13.139 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: "options": { 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_max": 32, 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_min": 1 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": {} 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.140 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 8, 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 8, 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.25, 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 2, 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 2, 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 0.92424774169921875, 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 0.92000001668930054, 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 0.92000001668930054 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 2, 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": "rbd", 2024-09-13T07:03:13.141 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:07.404263+0000", 2024-09-13T07:03:13.142 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 8193, 2024-09-13T07:03:13.142 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:03:13.142 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:03:13.142 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:03:13.142 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:03:13.142 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:03:13.142 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:03:13.142 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:03:13.142 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:03:13.142 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:03:13.142 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:03:13.142 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:03:13.142 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 8, 2024-09-13T07:03:13.143 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 8, 2024-09-13T07:03:13.143 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 8, 2024-09-13T07:03:13.143 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 8, 2024-09-13T07:03:13.143 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 8, 2024-09-13T07:03:13.143 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:03:13.143 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:03:13.143 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:03:13.143 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:03:13.143 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:03:13.143 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:03:13.143 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:03:13.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.143 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "20", 2024-09-13T07:03:13.144 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:03:13.144 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:03:13.144 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:03:13.144 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:03:13.144 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:03:13.144 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 3, 2024-09-13T07:03:13.144 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 20, 2024-09-13T07:03:13.144 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:03:13.144 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:03:13.144 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:03:13.144 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:03:13.145 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:03:13.146 INFO:teuthology.orchestra.run.smithi118.stdout: "options": {}, 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd": {} 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 2.9900000095367432, 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 2.9900000095367432, 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 2.630000114440918, 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 2.630000114440918, 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 0.92424774169921875, 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 0.92000001668930054, 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 0.92000001668930054 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.147 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 3, 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": "pool_client.0", 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:17.321011+0000", 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 8193, 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:03:13.148 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 32, 2024-09-13T07:03:13.149 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 32, 2024-09-13T07:03:13.149 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 32, 2024-09-13T07:03:13.149 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 32, 2024-09-13T07:03:13.149 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 32, 2024-09-13T07:03:13.149 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:03:13.149 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:03:13.149 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:03:13.149 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:03:13.149 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:03:13.149 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "121", 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 44, 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 121, 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:03:13.150 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:03:13.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "options": {}, 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd": {} 2024-09-13T07:03:13.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 1.7400000095367432, 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 1.7400000095367432, 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 0.92424774169921875, 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 0.92000001668930054, 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 0.92000001668930054 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: "osds": [ 2024-09-13T07:03:13.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 0, 2024-09-13T07:03:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "21736eee-00cd-4eef-9725-eca7776957b4", 2024-09-13T07:03:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:03:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:03:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.826507568359375, 2024-09-13T07:03:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:03:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:03:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:03:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:03:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 117, 2024-09-13T07:03:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:03:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:03:13.154 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6817", 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6819", 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.155 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6821", 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6823", 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:03:13.156 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6829", 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6831", 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:03:13.157 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6825", 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6827", 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6819/3731386737", 2024-09-13T07:03:13.158 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6823/3731386737", 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6831/3731386737", 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6827/3731386737", 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 1, 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "05818db2-841d-406d-be17-8fbdab4fff69", 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 0, 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0, 2024-09-13T07:03:13.159 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:03:13.160 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:03:13.160 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:03:13.160 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:03:13.160 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 84, 2024-09-13T07:03:13.160 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:03:13.160 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:03:13.160 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:03:13.160 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.160 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.160 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6816", 2024-09-13T07:03:13.160 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:03:13.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6818", 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6820", 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:03:13.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.162 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.162 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6822", 2024-09-13T07:03:13.162 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:03:13.162 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.162 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.162 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:03:13.162 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.162 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.162 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6828", 2024-09-13T07:03:13.163 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:03:13.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.163 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.163 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6830", 2024-09-13T07:03:13.163 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:03:13.163 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.163 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.163 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:03:13.163 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.164 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.164 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6824", 2024-09-13T07:03:13.164 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:03:13.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.164 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.164 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6826", 2024-09-13T07:03:13.164 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:03:13.164 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.164 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.164 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6818/876310814", 2024-09-13T07:03:13.165 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6822/876310814", 2024-09-13T07:03:13.165 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6830/876310814", 2024-09-13T07:03:13.165 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6826/876310814", 2024-09-13T07:03:13.165 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:03:13.165 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:03:13.165 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:03:13.165 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.165 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 2, 2024-09-13T07:03:13.165 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "2387761c-42d2-481b-b5b5-b16d3a943b22", 2024-09-13T07:03:13.165 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:03:13.165 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:03:13.166 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:03:13.166 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:03:13.166 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:03:13.166 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:03:13.166 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:03:13.166 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 102, 2024-09-13T07:03:13.166 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:03:13.166 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:03:13.166 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:03:13.166 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.166 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.166 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6808", 2024-09-13T07:03:13.167 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:03:13.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.167 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.167 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6809", 2024-09-13T07:03:13.167 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:03:13.167 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.167 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.167 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:03:13.167 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6810", 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6811", 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.168 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.169 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6814", 2024-09-13T07:03:13.169 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:03:13.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.169 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.169 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6815", 2024-09-13T07:03:13.169 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:03:13.169 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.169 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6812", 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6813", 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6809/690188762", 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6811/690188762", 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6815/690188762", 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6813/690188762", 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 3, 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "56500e90-f76b-47c8-a649-893400359c53", 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:03:13.171 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.8596343994140625, 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 0.39398193359375, 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 116, 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6800", 2024-09-13T07:03:13.172 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:03:13.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.173 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.173 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6801", 2024-09-13T07:03:13.173 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:03:13.173 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.173 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.173 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:03:13.173 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.173 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.173 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6802", 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6803", 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.174 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6806", 2024-09-13T07:03:13.175 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:03:13.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.175 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.175 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6807", 2024-09-13T07:03:13.175 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:03:13.175 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.175 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.175 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:03:13.175 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.175 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6804", 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6805", 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6801/2198175481", 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6803/2198175481", 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6807/2198175481", 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6805/2198175481", 2024-09-13T07:03:13.176 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:03:13.177 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:03:13.177 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:03:13.177 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.177 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 4, 2024-09-13T07:03:13.177 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "f594a612-aa07-4565-a09e-08368cf397b6", 2024-09-13T07:03:13.177 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:03:13.177 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:03:13.177 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.9017486572265625, 2024-09-13T07:03:13.177 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:03:13.177 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 117, 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6810", 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.178 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.179 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6813", 2024-09-13T07:03:13.179 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:03:13.179 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.179 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.179 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:03:13.179 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.179 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.179 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6817", 2024-09-13T07:03:13.179 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:03:13.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6819", 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6827", 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.180 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6830", 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6822", 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.181 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6825", 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6813/1835601651", 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6819/1835601651", 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6830/1835601651", 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6825/1835601651", 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 5, 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "1013b81f-e6b4-430b-938d-88759d43b881", 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.95001220703125, 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 115, 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:03:13.183 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.184 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.184 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6800", 2024-09-13T07:03:13.184 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:03:13.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.184 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.184 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6801", 2024-09-13T07:03:13.184 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:03:13.184 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.184 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.184 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:03:13.185 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.185 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.185 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6802", 2024-09-13T07:03:13.185 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:03:13.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.185 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.185 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6803", 2024-09-13T07:03:13.185 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:03:13.185 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.185 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6806", 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6807", 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.186 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.187 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:03:13.187 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.187 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.187 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6804", 2024-09-13T07:03:13.187 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:03:13.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.187 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.187 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6805", 2024-09-13T07:03:13.187 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:03:13.188 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.188 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.188 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6801/2853696331", 2024-09-13T07:03:13.188 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6803/2853696331", 2024-09-13T07:03:13.188 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6807/2853696331", 2024-09-13T07:03:13.188 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6805/2853696331", 2024-09-13T07:03:13.188 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:03:13.188 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:03:13.188 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:03:13.188 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.189 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 6, 2024-09-13T07:03:13.189 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "367f5f5b-5919-4534-b88c-229f327e4950", 2024-09-13T07:03:13.189 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:03:13.189 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:03:13.189 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:03:13.189 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:03:13.189 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:03:13.189 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:03:13.189 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:03:13.189 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 117, 2024-09-13T07:03:13.189 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:03:13.189 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:03:13.189 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6808", 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6812", 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6815", 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6818", 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.191 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6828", 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6831", 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.192 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6821", 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6824", 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6812/131309660", 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6818/131309660", 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6831/131309660", 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6824/131309660", 2024-09-13T07:03:13.193 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:03:13.194 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:03:13.194 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:03:13.194 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.194 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 7, 2024-09-13T07:03:13.194 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "90d9e486-e1ae-4305-80a6-6f483e534643", 2024-09-13T07:03:13.194 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:03:13.194 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:03:13.194 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:03:13.194 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:03:13.194 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:03:13.194 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 116, 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6809", 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.195 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6811", 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6814", 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6816", 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:03:13.196 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6826", 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6829", 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.197 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6820", 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6823", 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.198 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6811/652173699", 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6816/652173699", 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6829/652173699", 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6823/652173699", 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_xinfo": [ 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 0, 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:03:13.199 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.059233+0000", 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 1, 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 12087, 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.083130+0000", 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.201 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 2, 2024-09-13T07:03:13.201 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:03:13.201 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:03:13.201 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:03:13.201 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:03:13.201 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:03:13.201 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.848896+0000", 2024-09-13T07:03:13.201 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:03:13.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.201 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 3, 2024-09-13T07:03:13.201 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:03:13.202 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:03:13.202 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:03:13.202 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:03:13.202 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:03:13.202 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.847456+0000", 2024-09-13T07:03:13.202 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:03:13.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.202 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 4, 2024-09-13T07:03:13.202 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:03:13.202 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:03:13.202 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:03:13.202 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:03:13.203 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:03:13.203 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.914135+0000", 2024-09-13T07:03:13.203 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:03:13.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.203 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 5, 2024-09-13T07:03:13.203 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:03:13.203 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:03:13.203 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:03:13.203 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:03:13.203 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:03:13.203 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.834284+0000", 2024-09-13T07:03:13.203 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:03:13.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.204 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 6, 2024-09-13T07:03:13.204 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:03:13.204 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:03:13.204 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:03:13.204 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:03:13.204 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:03:13.204 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.003715+0000", 2024-09-13T07:03:13.204 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:03:13.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.204 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 7, 2024-09-13T07:03:13.205 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:03:13.205 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:03:13.205 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:03:13.205 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:03:13.205 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:03:13.205 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.033514+0000", 2024-09-13T07:03:13.205 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:03:13.205 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.205 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:03:13.205 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap": [ 2024-09-13T07:03:13.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.205 INFO:teuthology.orchestra.run.smithi118.stdout: "pgid": "2.5", 2024-09-13T07:03:13.205 INFO:teuthology.orchestra.run.smithi118.stdout: "osds": [ 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: 6, 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: 4 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: "pgid": "3.d", 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: "osds": [ 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: 6, 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: 2 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap_items": [], 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap_primaries": [], 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_temp": [], 2024-09-13T07:03:13.206 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_temp": [], 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "blocklist": {}, 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "range_blocklist": {}, 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profiles": { 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "default": { 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "crush-failure-domain": "osd", 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "k": "2", 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "m": "1", 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "plugin": "jerasure", 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "technique": "reed_sol_van" 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps_queue": [], 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "new_removed_snaps": [], 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "new_purged_snaps": [], 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_node_flags": {}, 2024-09-13T07:03:13.207 INFO:teuthology.orchestra.run.smithi118.stdout: "device_class_flags": {}, 2024-09-13T07:03:13.208 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode": { 2024-09-13T07:03:13.208 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode_enabled": false, 2024-09-13T07:03:13.208 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_bucket_count": 0, 2024-09-13T07:03:13.208 INFO:teuthology.orchestra.run.smithi118.stdout: "degraded_stretch_mode": 0, 2024-09-13T07:03:13.208 INFO:teuthology.orchestra.run.smithi118.stdout: "recovering_stretch_mode": 0, 2024-09-13T07:03:13.208 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode_bucket": 0 2024-09-13T07:03:13.208 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.208 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:13.209 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:13.209 INFO:teuthology.orchestra.run.smithi118.stdout:4886 write 0x846f549 thru 0x8478538 (0x8ff0 bytes) 2024-09-13T07:03:13.210 INFO:tasks.thrashosds.thrasher:j is {'epoch': 121, 'fsid': '25bb30d1-9541-4e40-8659-18d83bb1bd8a', 'created': '2024-09-13T07:00:49.323883+0000', 'modified': '2024-09-13T07:03:11.775838+0000', 'last_up_change': '2024-09-13T07:01:00.146832+0000', 'last_in_change': '2024-09-13T07:03:07.751711+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T07:01:00.675365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9242477416992188, 'average_primary_affinity': 0.9200000166893005, 'average_primary_affinity_weighted': 0.9200000166893005}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T07:01:07.404263+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.9242477416992188, 'average_primary_affinity': 0.9200000166893005, 'average_primary_affinity_weighted': 0.9200000166893005}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T07:01:17.321011+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 44, 'snap_epoch': 121, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.9242477416992188, 'average_primary_affinity': 0.9200000166893005, 'average_primary_affinity_weighted': 0.9200000166893005}}], 'osds': [{'osd': 0, 'uuid': '21736eee-00cd-4eef-9725-eca7776957b4', 'up': 1, 'in': 1, 'weight': 0.826507568359375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6817', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6819', 'nonce': 3731386737}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6821', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6823', 'nonce': 3731386737}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6829', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6831', 'nonce': 3731386737}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6825', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6827', 'nonce': 3731386737}]}, 'public_addr': '172.21.15.118:6819/3731386737', 'cluster_addr': '172.21.15.118:6823/3731386737', 'heartbeat_back_addr': '172.21.15.118:6831/3731386737', 'heartbeat_front_addr': '172.21.15.118:6827/3731386737', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '05818db2-841d-406d-be17-8fbdab4fff69', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6816', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6818', 'nonce': 876310814}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6820', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6822', 'nonce': 876310814}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6828', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6830', 'nonce': 876310814}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6824', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6826', 'nonce': 876310814}]}, 'public_addr': '172.21.15.118:6818/876310814', 'cluster_addr': '172.21.15.118:6822/876310814', 'heartbeat_back_addr': '172.21.15.118:6830/876310814', 'heartbeat_front_addr': '172.21.15.118:6826/876310814', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2387761c-42d2-481b-b5b5-b16d3a943b22', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 102, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6808', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6809', 'nonce': 690188762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6810', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6811', 'nonce': 690188762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6814', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6815', 'nonce': 690188762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6812', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6813', 'nonce': 690188762}]}, 'public_addr': '172.21.15.118:6809/690188762', 'cluster_addr': '172.21.15.118:6811/690188762', 'heartbeat_back_addr': '172.21.15.118:6815/690188762', 'heartbeat_front_addr': '172.21.15.118:6813/690188762', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '56500e90-f76b-47c8-a649-893400359c53', 'up': 1, 'in': 1, 'weight': 0.8596343994140625, 'primary_affinity': 0.39398193359375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6800', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6801', 'nonce': 2198175481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6802', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6803', 'nonce': 2198175481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6806', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6807', 'nonce': 2198175481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6804', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6805', 'nonce': 2198175481}]}, 'public_addr': '172.21.15.118:6801/2198175481', 'cluster_addr': '172.21.15.118:6803/2198175481', 'heartbeat_back_addr': '172.21.15.118:6807/2198175481', 'heartbeat_front_addr': '172.21.15.118:6805/2198175481', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f594a612-aa07-4565-a09e-08368cf397b6', 'up': 1, 'in': 1, 'weight': 0.9017486572265625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 1835601651}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6817', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 1835601651}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6827', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6830', 'nonce': 1835601651}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 1835601651}]}, 'public_addr': '172.21.15.164:6813/1835601651', 'cluster_addr': '172.21.15.164:6819/1835601651', 'heartbeat_back_addr': '172.21.15.164:6830/1835601651', 'heartbeat_front_addr': '172.21.15.164:6825/1835601651', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1013b81f-e6b4-430b-938d-88759d43b881', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 115, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 2853696331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 2853696331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 2853696331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 2853696331}]}, 'public_addr': '172.21.15.164:6801/2853696331', 'cluster_addr': '172.21.15.164:6803/2853696331', 'heartbeat_back_addr': '172.21.15.164:6807/2853696331', 'heartbeat_front_addr': '172.21.15.164:6805/2853696331', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '367f5f5b-5919-4534-b88c-229f327e4950', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6812', 'nonce': 131309660}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6815', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6818', 'nonce': 131309660}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 131309660}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6821', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6824', 'nonce': 131309660}]}, 'public_addr': '172.21.15.164:6812/131309660', 'cluster_addr': '172.21.15.164:6818/131309660', 'heartbeat_back_addr': '172.21.15.164:6831/131309660', 'heartbeat_front_addr': '172.21.15.164:6824/131309660', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '90d9e486-e1ae-4305-80a6-6f483e534643', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6809', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 652173699}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6816', 'nonce': 652173699}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 652173699}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 652173699}]}, 'public_addr': '172.21.15.164:6811/652173699', 'cluster_addr': '172.21.15.164:6816/652173699', 'heartbeat_back_addr': '172.21.15.164:6829/652173699', 'heartbeat_front_addr': '172.21.15.164:6823/652173699', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.059233+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 12087, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.083130+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.848896+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.847456+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.914135+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.834284+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.003715+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.033514+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.5', 'osds': [6, 4]}, {'pgid': '3.d', 'osds': [6, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T07:03:13.211 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.d 2024-09-13T07:03:13.211 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.d 2024-09-13T07:03:13.211 INFO:teuthology.orchestra.run.smithi118.stdout:4889 read 0x92f6596 thru 0x9301e5c (0xb8c7 bytes) 2024-09-13T07:03:13.212 INFO:teuthology.orchestra.run.smithi118.stdout:4890 read 0x65da9f7 thru 0x65e2343 (0x794d bytes) 2024-09-13T07:03:13.212 INFO:teuthology.orchestra.run.smithi118.stdout:4891 read 0x45e438f thru 0x45ec4d1 (0x8143 bytes) 2024-09-13T07:03:13.212 INFO:teuthology.orchestra.run.smithi118.stdout:4892 write 0x485d1b3 thru 0x48614d3 (0x4321 bytes) 2024-09-13T07:03:13.213 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:13.213 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:13.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:13.213 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:13.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.213 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:03:13.213 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:13.213 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 107.11034830200001, 2024-09-13T07:03:13.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:03:13.213 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.213 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:13.213 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:13.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:13.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:03:13.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:13.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:03:13.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:03:13.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:03:13.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:03:13.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:13.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:03:13.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:03:13.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:03:13.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:03:13.217 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.217 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.217 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133467 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874255+0000", 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.292684345000001, 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0085814259999999996, 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133467 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874255+0000", 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874255+0000", 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874254+0000", 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874260+0000", 2024-09-13T07:03:13.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4430000000000002e-06 2024-09-13T07:03:13.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874284+0000", 2024-09-13T07:03:13.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4145e-05 2024-09-13T07:03:13.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874295+0000", 2024-09-13T07:03:13.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0857e-05 2024-09-13T07:03:13.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882136+0000", 2024-09-13T07:03:13.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078415900000000007 2024-09-13T07:03:13.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882218+0000", 2024-09-13T07:03:13.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1866999999999994e-05 2024-09-13T07:03:13.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882837+0000", 2024-09-13T07:03:13.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00061833000000000005 2024-09-13T07:03:13.222 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.222 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.222 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.222 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133476 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:13.222 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:03:13.222 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.292588438999999, 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0096071349999999993, 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133476 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:03:13.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874350+0000", 2024-09-13T07:03:13.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:13.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874356+0000", 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5999999999999997e-06 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874369+0000", 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3125000000000001e-05 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874380+0000", 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1396999999999999e-05 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882884+0000", 2024-09-13T07:03:13.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.008503666 2024-09-13T07:03:13.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883050+0000", 2024-09-13T07:03:13.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000165926 2024-09-13T07:03:13.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883958+0000", 2024-09-13T07:03:13.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00090819200000000005 2024-09-13T07:03:13.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.226 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133477 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1310720~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.292549220000001, 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010531547, 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133477 2024-09-13T07:03:13.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874394+0000", 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7829999999999998e-06 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874407+0000", 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3050999999999999e-05 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874418+0000", 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0974e-05 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883998+0000", 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095798310000000005 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884148+0000", 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014957499999999999 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884922+0000", 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000774127 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133484 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.292466065999999, 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011150092, 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.231 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133484 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874472+0000", 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874479+0000", 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5700000000000004e-06 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874497+0000", 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7385999999999999e-05 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874510+0000", 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3628999999999999e-05 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884968+0000", 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010458053 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885119+0000", 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000150932 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885624+0000", 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050432500000000002 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.235 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133485 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:13.236 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:03:13.236 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.292414374, 2024-09-13T07:03:13.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011776926, 2024-09-13T07:03:13.236 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.236 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.236 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.236 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.236 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.236 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133485 2024-09-13T07:03:13.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.236 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874524+0000", 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874530+0000", 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8599999999999998e-06 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874545+0000", 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5186e-05 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874558+0000", 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3040000000000001e-05 2024-09-13T07:03:13.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885700+0000", 2024-09-13T07:03:13.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011141514999999999 2024-09-13T07:03:13.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885859+0000", 2024-09-13T07:03:13.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015918599999999999 2024-09-13T07:03:13.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886302+0000", 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044316899999999998 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133494 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.292278577000001, 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012188055999999999, 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.240 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.241 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.241 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133494 2024-09-13T07:03:13.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.241 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:03:13.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:03:13.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874660+0000", 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874666+0000", 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7300000000000002e-06 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874701+0000", 2024-09-13T07:03:13.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5095999999999997e-05 2024-09-13T07:03:13.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874714+0000", 2024-09-13T07:03:13.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2833000000000001e-05 2024-09-13T07:03:13.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886333+0000", 2024-09-13T07:03:13.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011619078 2024-09-13T07:03:13.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886405+0000", 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2525000000000001e-05 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886849+0000", 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000443697 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.244 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133498 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:13.245 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:03:13.245 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.292213815, 2024-09-13T07:03:13.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012651943000000001, 2024-09-13T07:03:13.245 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.245 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.245 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.245 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.245 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.245 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133498 2024-09-13T07:03:13.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.245 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:03:13.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:03:13.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874725+0000", 2024-09-13T07:03:13.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874731+0000", 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0159999999999997e-06 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874788+0000", 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7021000000000002e-05 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874797+0000", 2024-09-13T07:03:13.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3430000000000002e-06 2024-09-13T07:03:13.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886880+0000", 2024-09-13T07:03:13.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012083287 2024-09-13T07:03:13.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886959+0000", 2024-09-13T07:03:13.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8582000000000001e-05 2024-09-13T07:03:13.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887378+0000", 2024-09-13T07:03:13.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041888400000000001 2024-09-13T07:03:13.249 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.249 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.249 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.249 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:13.249 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:13.249 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.292131945000001, 2024-09-13T07:03:13.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:03:13.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.253 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:13.254 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:13.254 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.292082557000001, 2024-09-13T07:03:13.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:03:13.254 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.254 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.254 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.254 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.254 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.254 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:03:13.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.254 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:03:13.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:03:13.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:03:13.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:03:13.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:03:13.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.292059850000001, 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.258 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:03:13.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:03:13.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:03:13.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:03:13.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:03:13.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:03:13.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.292015657, 2024-09-13T07:03:13.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:03:13.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:03:13.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:03:13.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:03:13.266 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.266 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.266 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.267 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133516 3.0 3:07bbab57:::rbd_data.11946d7ee8e1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:13.267 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:03:13.267 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.281180475999999, 2024-09-13T07:03:13.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010861226999999999, 2024-09-13T07:03:13.267 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.267 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:13.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.267 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133516 2024-09-13T07:03:13.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.267 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885757+0000", 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:13.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886801+0000", 2024-09-13T07:03:13.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010439550000000001 2024-09-13T07:03:13.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886827+0000", 2024-09-13T07:03:13.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5803e-05 2024-09-13T07:03:13.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886839+0000", 2024-09-13T07:03:13.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2136e-05 2024-09-13T07:03:13.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886894+0000", 2024-09-13T07:03:13.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4434000000000001e-05 2024-09-13T07:03:13.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887043+0000", 2024-09-13T07:03:13.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014903200000000001 2024-09-13T07:03:13.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:13.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887376+0000", 2024-09-13T07:03:13.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00033327399999999999 2024-09-13T07:03:13.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:13.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.894299+0000", 2024-09-13T07:03:13.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069232620000000003 2024-09-13T07:03:13.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:13.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896556+0000", 2024-09-13T07:03:13.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022571309999999999 2024-09-13T07:03:13.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:13.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896599+0000", 2024-09-13T07:03:13.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2620999999999997e-05 2024-09-13T07:03:13.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896620+0000", 2024-09-13T07:03:13.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1367999999999999e-05 2024-09-13T07:03:13.272 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.273 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.273 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.273 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:03:13.273 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:13.273 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.261036568000002, 2024-09-13T07:03:13.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:03:13.273 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.273 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:13.273 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:13.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:13.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:13.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:03:13.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:03:13.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:03:13.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:03:13.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:03:13.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:03:13.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:13.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:03:13.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:03:13.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:03:13.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:03:13.277 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.277 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.277 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137457 3.18 3:1a596dcc:::rbd_data.1194af11889b.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.5154147580000004, 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033734403000000003, 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137457 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:13.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:13.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651523+0000", 2024-09-13T07:03:13.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:13.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651531+0000", 2024-09-13T07:03:13.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.328e-06 2024-09-13T07:03:13.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651545+0000", 2024-09-13T07:03:13.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4270000000000001e-05 2024-09-13T07:03:13.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651555+0000", 2024-09-13T07:03:13.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5210000000000008e-06 2024-09-13T07:03:13.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651626+0000", 2024-09-13T07:03:13.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1915999999999994e-05 2024-09-13T07:03:13.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651761+0000", 2024-09-13T07:03:13.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000134838 2024-09-13T07:03:13.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.658577+0000", 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068151820000000004 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685181+0000", 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026604200000000001 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685193+0000", 2024-09-13T07:03:13.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2128e-05 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685226+0000", 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3504999999999999e-05 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685259+0000", 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2866e-05 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138565 3.0 3:002d0425:::rbd_data.11949f62878b.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.5020442660000004, 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033394489999999999, 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138565 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664893+0000", 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:13.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664906+0000", 2024-09-13T07:03:13.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2996000000000001e-05 2024-09-13T07:03:13.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664936+0000", 2024-09-13T07:03:13.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9609999999999999e-05 2024-09-13T07:03:13.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664949+0000", 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383e-05 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665046+0000", 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6459000000000006e-05 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665259+0000", 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000213141 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.672522+0000", 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072632110000000003 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.697982+0000", 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025459699999999998 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698018+0000", 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6375999999999999e-05 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698166+0000", 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014818700000000001 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698290+0000", 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012333999999999999 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.289 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138587 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.4535785990000001, 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019224502000000001, 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138587 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:13.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713360+0000", 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713368+0000", 2024-09-13T07:03:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:03:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713383+0000", 2024-09-13T07:03:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5176e-05 2024-09-13T07:03:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713390+0000", 2024-09-13T07:03:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8669999999999996e-06 2024-09-13T07:03:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713449+0000", 2024-09-13T07:03:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8938999999999999e-05 2024-09-13T07:03:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713549+0000", 2024-09-13T07:03:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010009 2024-09-13T07:03:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.716087+0000", 2024-09-13T07:03:13.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025383820000000001 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732351+0000", 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016263564000000001 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732450+0000", 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9529000000000002e-05 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:13.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732532+0000", 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1526000000000005e-05 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732585+0000", 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3819000000000001e-05 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138688 3.18 3:181a7d86:::rbd_data.11949f62878b.000000000000000a:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.915397+0000", 2024-09-13T07:03:13.295 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.2515424680000002, 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0082266089999999993, 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138688 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915397+0000", 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915397+0000", 2024-09-13T07:03:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915396+0000", 2024-09-13T07:03:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915402+0000", 2024-09-13T07:03:13.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0970000000000002e-06 2024-09-13T07:03:13.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915418+0000", 2024-09-13T07:03:13.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5766000000000001e-05 2024-09-13T07:03:13.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.915431+0000", 2024-09-13T07:03:13.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2677000000000001e-05 2024-09-13T07:03:13.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.921758+0000", 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0063273670000000004 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.921945+0000", 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000187479 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.922174+0000", 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000228855 2024-09-13T07:03:13.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:13.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.923490+0000", 2024-09-13T07:03:13.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001315891 2024-09-13T07:03:13.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:13.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.923504+0000", 2024-09-13T07:03:13.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3750999999999999e-05 2024-09-13T07:03:13.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:13.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.923569+0000", 2024-09-13T07:03:13.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5480000000000003e-05 2024-09-13T07:03:13.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.923624+0000", 2024-09-13T07:03:13.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4265999999999998e-05 2024-09-13T07:03:13.301 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.301 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.301 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.301 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138786 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.0193633440000003, 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018847452000000001, 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138786 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:13.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147575+0000", 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147583+0000", 2024-09-13T07:03:13.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147601+0000", 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7961e-05 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147610+0000", 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147661+0000", 2024-09-13T07:03:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0447e-05 2024-09-13T07:03:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147754+0000", 2024-09-13T07:03:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2974999999999997e-05 2024-09-13T07:03:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.150580+0000", 2024-09-13T07:03:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002826566 2024-09-13T07:03:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166135+0000", 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015554772 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166225+0000", 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0258000000000002e-05 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166334+0000", 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010816500000000001 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166424+0000", 2024-09-13T07:03:13.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0116000000000003e-05 2024-09-13T07:03:13.307 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.307 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.307 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.307 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:139127 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:13.307 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:13.307 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 3.3923123799999999, 2024-09-13T07:03:13.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019295931999999998, 2024-09-13T07:03:13.307 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.308 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:13.308 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.308 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.308 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.308 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139127 2024-09-13T07:03:13.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.308 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:13.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774626+0000", 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774635+0000", 2024-09-13T07:03:13.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5419999999999994e-06 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774654+0000", 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8359000000000002e-05 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774663+0000", 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774718+0000", 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5315000000000001e-05 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774805+0000", 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7284e-05 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.779319+0000", 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045138310000000003 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:13.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793772+0000", 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014452692 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793821+0000", 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9563e-05 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793878+0000", 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6953999999999997e-05 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793923+0000", 2024-09-13T07:03:13.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4781999999999999e-05 2024-09-13T07:03:13.313 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.313 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.313 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.313 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.313 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.313 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:13.313 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:13.315 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:13.315 INFO:teuthology.orchestra.run.smithi118.stdout:4893 trunc from 0xea3167b to 0x160918e 2024-09-13T07:03:13.316 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:13.206+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:13.316 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:13.314+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:13.316 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:13.316 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:13.316 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:13.317 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:13.317 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:13.317 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:13.317 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:13.318 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:13.318 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:13.318 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:13.318 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:13.318 INFO:teuthology.orchestra.run.smithi118.stdout:4895 write 0xea62600 thru 0xea6e4b4 (0xbeb5 bytes) 2024-09-13T07:03:13.318 INFO:teuthology.orchestra.run.smithi118.stdout:4897 read 0x6613b8 thru 0x6661f2 (0x4e3b bytes) 2024-09-13T07:03:13.318 INFO:teuthology.orchestra.run.smithi118.stdout:4898 writesame 0xdc6f7f3 thru 0xdc7d3af data_size 0x11(0xdbbd bytes) 2024-09-13T07:03:13.333 INFO:teuthology.orchestra.run.smithi118.stdout:4899 read 0xd70d0b thru 0xd70ee5 (0x1db bytes) 2024-09-13T07:03:13.334 INFO:teuthology.orchestra.run.smithi118.stdout:4900 read 0xe5b3718 thru 0xe5c024a (0xcb33 bytes) 2024-09-13T07:03:13.357 INFO:teuthology.orchestra.run.smithi118.stdout:4901 trunc from 0xea6e4b5 to 0x24c8453 2024-09-13T07:03:13.399 INFO:teuthology.orchestra.run.smithi118.stdout:4902 trunc from 0x24c8453 to 0xc86aba2 2024-09-13T07:03:13.410 INFO:teuthology.orchestra.run.smithi118.stdout:4903 punch from 0x83a7374 to 0x83aa630, (0x32bc bytes) 2024-09-13T07:03:13.410 INFO:teuthology.orchestra.run.smithi118.stdout:4904 write 0x923154 thru 0x926653 (0x3500 bytes) 2024-09-13T07:03:13.416 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:13.411+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:13.419 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:13.419 INFO:teuthology.orchestra.run.smithi118.stdout:4907 write 0x635834d thru 0x63654b8 (0xd16c bytes) 2024-09-13T07:03:13.443 INFO:teuthology.orchestra.run.smithi118.stdout:4909 trunc from 0xc86aba2 to 0xc9951e1 2024-09-13T07:03:13.450 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:13.450 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:13.450 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:13.452 INFO:teuthology.orchestra.run.smithi118.stdout:4911 write 0x9cac92e thru 0x9cb3a03 (0x70d6 bytes) 2024-09-13T07:03:13.462 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:13.469 INFO:teuthology.orchestra.run.smithi118.stdout:4913 read 0x5e1b993 thru 0x5e22de0 (0x744e bytes) 2024-09-13T07:03:13.470 INFO:teuthology.orchestra.run.smithi118.stdout:4914 read 0x809950d thru 0x80a93af (0xfea3 bytes) 2024-09-13T07:03:13.473 INFO:teuthology.orchestra.run.smithi118.stdout:4915 trunc from 0xc9951e1 to 0x95fe903 2024-09-13T07:03:13.509 INFO:teuthology.orchestra.run.smithi118.stdout:4917 writesame 0xc2f9199 thru 0xc305282 data_size 0x3(0xc0ea bytes) 2024-09-13T07:03:13.509 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:13.509 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:13.509 INFO:teuthology.orchestra.run.smithi118.stdout:4921 write 0x7d6a7d4 thru 0x7d72f40 (0x876d bytes) 2024-09-13T07:03:13.517 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:13.515+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:13.529 INFO:teuthology.orchestra.run.smithi118.stdout:4924 writesame 0xb761dfc thru 0xb763e8f data_size 0x1e(0x2094 bytes) 2024-09-13T07:03:13.535 INFO:teuthology.orchestra.run.smithi118.stdout:4926 read 0xa6f3a08 thru 0xa6f8643 (0x4c3c bytes) 2024-09-13T07:03:13.546 INFO:teuthology.orchestra.run.smithi118.stdout:4928 trunc from 0xc305283 to 0x5474571 2024-09-13T07:03:13.585 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:13.585 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:13.592 INFO:teuthology.orchestra.run.smithi118.stdout:4929 writesame 0xc3160b6 thru 0xc31ecdb data_size 0x1(0x8c26 bytes) 2024-09-13T07:03:13.595 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:13.598 INFO:teuthology.orchestra.run.smithi118.stdout:4930 trunc from 0xc31ecdc to 0x87a237d 2024-09-13T07:03:13.610 INFO:teuthology.orchestra.run.smithi118.stdout:4931 read 0x6c6bf5f thru 0x6c79cbe (0xdd60 bytes) 2024-09-13T07:03:13.610 INFO:teuthology.orchestra.run.smithi118.stdout:4932 read 0x653cd29 thru 0x6544ecb (0x81a3 bytes) 2024-09-13T07:03:13.618 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:13.615+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:13.621 INFO:teuthology.orchestra.run.smithi118.stdout:4933 trunc from 0x87a237d to 0x15ceb2f 2024-09-13T07:03:13.669 INFO:teuthology.orchestra.run.smithi118.stdout:4934 write 0xcfd0e69 thru 0xcfd0ea3 (0x3b bytes) 2024-09-13T07:03:13.690 INFO:teuthology.orchestra.run.smithi118.stdout:4935 trunc from 0xcfd0ea4 to 0x439ad80 2024-09-13T07:03:13.717 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:13.718 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:13.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:13.718 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:13.718 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.718 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.718 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:13.718 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.595588481, 2024-09-13T07:03:13.718 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:03:13.718 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.718 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.718 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.718 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.719 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.719 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:03:13.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.719 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.719 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:13.719 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.719 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.719 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.719 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:13.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:03:13.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.720 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.720 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:03:13.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:03:13.720 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:03:13.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:03:13.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:03:13.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:03:13.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.722 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.723 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.723 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:13.723 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.595568004, 2024-09-13T07:03:13.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:03:13.723 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.723 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.723 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.723 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.723 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.723 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:13.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:03:13.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:03:13.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.725 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.725 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:03:13.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:03:13.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:03:13.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:03:13.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:03:13.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:03:13.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87036 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 6963200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:13.727 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588883938000002, 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042468831999999998, 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87036 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127964+0000", 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7799999999999999e-06 2024-09-13T07:03:13.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127967+0000", 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9730000000000002e-06 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127971+0000", 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4349999999999999e-06 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170202+0000", 2024-09-13T07:03:13.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042230940000000002 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170293+0000", 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0858000000000003e-05 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170431+0000", 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013821699999999999 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.732 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.732 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:13.732 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588871400999999, 2024-09-13T07:03:13.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:03:13.732 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.732 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.732 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:03:13.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.732 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:13.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:13.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:13.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:13.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:03:13.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:03:13.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:03:13.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:03:13.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:03:13.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588859331999998, 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:03:13.736 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:03:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:03:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:03:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:03:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:03:13.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:03:13.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:03:13.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:03:13.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:03:13.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:03:13.740 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.740 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.740 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.741 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.741 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:13.741 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588846889000003, 2024-09-13T07:03:13.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:03:13.741 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.741 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.741 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.741 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.741 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.741 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:03:13.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:03:13.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:03:13.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:03:13.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:03:13.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:03:13.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:03:13.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:03:13.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:03:13.745 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.745 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.745 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.745 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.745 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:13.745 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588829291000003, 2024-09-13T07:03:13.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:03:13.746 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.746 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.746 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.746 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.746 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.746 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:03:13.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.746 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:13.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:13.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:13.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:03:13.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:03:13.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:03:13.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:03:13.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:03:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:03:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:03:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:03:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:03:13.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:03:13.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:03:13.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:03:13.750 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.750 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.750 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.751 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.751 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:13.751 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588818588000002, 2024-09-13T07:03:13.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:03:13.751 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.751 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.751 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.751 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.751 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.751 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:03:13.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.751 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:13.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:13.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:13.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:03:13.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:03:13.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:03:13.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:03:13.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:03:13.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:03:13.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:03:13.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:03:13.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.756 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.756 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:13.756 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588774655000002, 2024-09-13T07:03:13.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:03:13.756 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.756 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.756 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:03:13.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.756 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:13.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:13.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:03:13.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:03:13.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:03:13.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:03:13.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:03:13.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:03:13.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:03:13.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:03:13.760 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.760 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.760 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.760 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.760 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:13.760 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588754868999999, 2024-09-13T07:03:13.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:03:13.761 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.761 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.761 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.761 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.761 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.761 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:03:13.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.761 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:13.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:13.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:13.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:03:13.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:03:13.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:03:13.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:03:13.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:03:13.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:03:13.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:03:13.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:03:13.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:03:13.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:03:13.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588739119000003, 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.765 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.766 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.766 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.766 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:03:13.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.766 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:13.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:13.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:13.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:13.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:03:13.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:03:13.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.769 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588727867000003, 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.773 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588717268000003, 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.774 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:13.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:13.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:03:13.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:03:13.778 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.778 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.778 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.778 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.778 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:13.778 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588707775000003, 2024-09-13T07:03:13.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:03:13.779 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.779 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.779 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.779 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.779 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.779 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:03:13.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.779 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:13.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:03:13.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:03:13.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:03:13.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:03:13.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:03:13.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:03:13.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:03:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:03:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:03:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:03:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:03:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:03:13.782 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.783 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.783 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.783 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.783 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:13.783 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588698231000002, 2024-09-13T07:03:13.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:03:13.783 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.783 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.783 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.783 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.784 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.784 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:03:13.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.784 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:13.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:13.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:13.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:13.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:03:13.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:03:13.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:03:13.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:03:13.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:03:13.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:03:13.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.787 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.788 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.788 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:13.788 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588665628000001, 2024-09-13T07:03:13.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:03:13.788 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.788 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.788 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:03:13.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.788 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:13.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:13.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:03:13.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:03:13.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:03:13.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.792 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.792 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.792 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:13.792 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.588652605, 2024-09-13T07:03:13.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:03:13.793 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.793 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.793 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:03:13.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.793 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:13.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:13.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:13.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:13.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:03:13.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:03:13.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:03:13.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:03:13.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:03:13.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:03:13.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.797 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.797 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.797 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:13.797 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.585301620000003, 2024-09-13T07:03:13.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:03:13.797 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.797 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:13.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:03:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:03:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:03:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:03:13.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:03:13.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:03:13.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:03:13.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:03:13.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:03:13.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:03:13.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:03:13.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:03:13.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:03:13.801 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.802 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.802 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.802 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:13.802 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.585281342999998, 2024-09-13T07:03:13.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:03:13.802 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.802 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.803 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:03:13.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.803 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:13.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:13.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:03:13.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:13.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:03:13.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:03:13.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:03:13.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:03:13.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:03:13.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:03:13.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:03:13.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:03:13.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:03:13.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:03:13.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.585261142, 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:13.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:13.808 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:13.808 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:03:13.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.808 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:13.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:13.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:13.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:13.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:13.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:13.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:13.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:13.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:13.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:13.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:03:13.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:03:13.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:13.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:03:13.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:03:13.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:13.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:13.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:13.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:03:13.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:03:13.811 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.811 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.811 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.811 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:13.811 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:13.812 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:13.812 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:03:13.813 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:13.810+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:13.814 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:13.815 INFO:teuthology.orchestra.run.smithi118.stdout:4936 write 0xa5b0a61 thru 0xa5b8fec (0x858c bytes) 2024-09-13T07:03:13.815 INFO:teuthology.orchestra.run.smithi118.stdout:4937 write 0x4259c55 thru 0x4264cac (0xb058 bytes) 2024-09-13T07:03:13.815 INFO:teuthology.orchestra.run.smithi118.stdout:4938 read 0x8aea536 thru 0x8af6a85 (0xc550 bytes) 2024-09-13T07:03:13.815 INFO:teuthology.orchestra.run.smithi118.stdout:4940 read 0x56c6d62 thru 0x56c8b2a (0x1dc9 bytes) 2024-09-13T07:03:13.815 INFO:teuthology.orchestra.run.smithi118.stdout:4942 write 0xaf9440d thru 0xafa1962 (0xd556 bytes) 2024-09-13T07:03:13.816 INFO:teuthology.orchestra.run.smithi118.stderr:clear 3.d pg_upmap mapping 2024-09-13T07:03:13.818 INFO:teuthology.orchestra.run.smithi118.stdout:4943 trunc from 0xafa1963 to 0x61d3c5b 2024-09-13T07:03:13.862 INFO:teuthology.orchestra.run.smithi118.stdout:4944 writesame 0xd55811f thru 0xd5596b6 data_size 0x4(0x1598 bytes) 2024-09-13T07:03:13.862 INFO:teuthology.orchestra.run.smithi118.stdout:4947 writesame 0x2b6ed75 thru 0x2b7cd3a data_size 0x1(0xdfc6 bytes) 2024-09-13T07:03:13.901 INFO:teuthology.orchestra.run.smithi118.stdout:4948 trunc from 0xd5596b7 to 0x90a8452 2024-09-13T07:03:13.909 INFO:teuthology.orchestra.run.smithi118.stdout:4950 writesame 0x692e536 thru 0x693529a data_size 0x5(0x6d65 bytes) 2024-09-13T07:03:13.914 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:13.911+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:13.930 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:13.930 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:13.940 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:14.015 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:14.011+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:14.041 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:14.041 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:14.041 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:14.042 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:14.042 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:14.052 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:14.117 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:14.115+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:14.162 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:14.162 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:14.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:14.162 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:14.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.162 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.162 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:14.162 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558923837999998, 2024-09-13T07:03:14.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:03:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:03:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:03:14.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:03:14.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:03:14.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:03:14.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:03:14.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:14.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:03:14.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:03:14.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:03:14.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:03:14.167 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.167 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.167 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.167 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.167 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:14.167 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558892737000001, 2024-09-13T07:03:14.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:03:14.168 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.168 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.168 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:03:14.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.168 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:14.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:03:14.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:03:14.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:03:14.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:03:14.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:03:14.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:03:14.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:03:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:03:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:03:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:03:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558870422999998, 2024-09-13T07:03:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:03:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:03:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:03:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:03:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:03:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:03:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:03:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:03:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:03:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:03:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:03:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:03:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:03:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:03:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:03:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:03:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558847392000001, 2024-09-13T07:03:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:03:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:03:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:03:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:03:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:03:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:03:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:03:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:03:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:03:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:03:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:03:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:03:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:03:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:03:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:03:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558822842000001, 2024-09-13T07:03:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:03:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:03:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:03:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:03:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:03:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:03:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:03:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:03:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:03:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:03:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:03:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:03:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558787132999999, 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:03:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:03:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:03:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:03:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:03:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:03:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:03:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:03:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:03:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:03:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:03:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:03:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:03:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:03:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:03:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558762454, 2024-09-13T07:03:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:03:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:03:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:03:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:03:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:03:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:03:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:03:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:03:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.55872918, 2024-09-13T07:03:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:03:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:03:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:03:14.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:03:14.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:03:14.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:03:14.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:03:14.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:03:14.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.204 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.204 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:14.204 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558702392000001, 2024-09-13T07:03:14.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:03:14.204 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.204 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.204 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.204 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.204 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.204 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:03:14.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.204 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:14.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:14.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:03:14.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:03:14.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:03:14.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:03:14.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:03:14.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:03:14.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:03:14.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:03:14.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:03:14.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:03:14.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:03:14.208 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.208 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.208 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.209 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.209 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:14.209 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558677838000001, 2024-09-13T07:03:14.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:03:14.209 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.209 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.209 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.209 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.209 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.209 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:03:14.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.209 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:03:14.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:03:14.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:03:14.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:03:14.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:03:14.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:03:14.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:03:14.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:03:14.213 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.213 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.213 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.213 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.214 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:14.214 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558648069, 2024-09-13T07:03:14.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:03:14.214 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.214 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.214 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.214 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.214 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.214 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:03:14.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.214 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:14.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:14.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:14.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:03:14.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:03:14.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:03:14.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:03:14.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:03:14.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:03:14.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:03:14.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:03:14.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:03:14.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:03:14.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.218 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.218 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.219 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:14.219 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558588226000001, 2024-09-13T07:03:14.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:03:14.219 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.219 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.219 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:03:14.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.219 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:03:14.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:14.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:03:14.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:03:14.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:03:14.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:03:14.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:03:14.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:03:14.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:03:14.223 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.223 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.223 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.224 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.224 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:14.224 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558521376000002, 2024-09-13T07:03:14.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:03:14.224 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.224 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.224 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.224 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.224 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.224 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:03:14.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.224 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:14.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:14.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:03:14.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:03:14.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:03:14.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:03:14.228 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.228 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.228 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.229 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.229 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:14.229 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558488873999998, 2024-09-13T07:03:14.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:03:14.229 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.229 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.229 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.229 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.229 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.229 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:03:14.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.229 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:14.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:14.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:03:14.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:03:14.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:03:14.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:03:14.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:03:14.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:03:14.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:03:14.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:03:14.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:03:14.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:03:14.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:03:14.233 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.233 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558454253000001, 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:03:14.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:03:14.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:03:14.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:03:14.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:03:14.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:03:14.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:03:14.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:03:14.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:03:14.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:03:14.238 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.238 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.238 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558404022000001, 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:14.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:14.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:14.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558355836, 2024-09-13T07:03:14.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:03:14.244 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.244 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.244 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.244 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.244 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.244 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:03:14.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.244 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:14.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:03:14.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558329197999999, 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.248 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:03:14.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:03:14.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:03:14.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:03:14.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:03:14.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:03:14.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:03:14.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:03:14.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:03:14.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:03:14.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.558282363, 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.253 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:03:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:03:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:03:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:03:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:03:14.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:03:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:03:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:03:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:03:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.554927945999999, 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.258 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:14.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:14.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:03:14.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:03:14.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:03:14.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:03:14.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:03:14.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:03:14.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.262 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.263 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:14.264 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:03:14.264 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:14.262+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:14.365 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:14.362+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:14.374 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:14.374 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:14.374 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:14.374 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:14.389 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:03:14.466 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:14.463+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:14.496 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:14.496 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:14.496 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:14.506 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:03:14.567 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:14.566+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:14.618 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.498191930999994, 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:03:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:03:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:03:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:03:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:03:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:03:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:03:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:03:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:03:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:03:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:03:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:03:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:03:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:03:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:03:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129532 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.001789043999999, 2024-09-13T07:03:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89889205500000002, 2024-09-13T07:03:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129532 2024-09-13T07:03:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614994+0000", 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.044e-06 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615000+0000", 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9229999999999996e-06 2024-09-13T07:03:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615005+0000", 2024-09-13T07:03:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:03:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615060+0000", 2024-09-13T07:03:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4561000000000002e-05 2024-09-13T07:03:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615069+0000", 2024-09-13T07:03:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1540000000000008e-06 2024-09-13T07:03:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513869+0000", 2024-09-13T07:03:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89879977700000002 2024-09-13T07:03:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513883+0000", 2024-09-13T07:03:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4361e-05 2024-09-13T07:03:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129533 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.001693711000001, 2024-09-13T07:03:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.899270177, 2024-09-13T07:03:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129533 2024-09-13T07:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615090+0000", 2024-09-13T07:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.856e-06 2024-09-13T07:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615096+0000", 2024-09-13T07:03:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.703e-06 2024-09-13T07:03:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615102+0000", 2024-09-13T07:03:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.378e-06 2024-09-13T07:03:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615140+0000", 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7966999999999997e-05 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615147+0000", 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8800000000000002e-06 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514342+0000", 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919565300000004 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514357+0000", 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4355000000000001e-05 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129534 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.001620812999999, 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89923080600000005, 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129534 2024-09-13T07:03:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615162+0000", 2024-09-13T07:03:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3569999999999998e-06 2024-09-13T07:03:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615169+0000", 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.455e-06 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615175+0000", 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3679999999999998e-06 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615212+0000", 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6733999999999998e-05 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615218+0000", 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7569999999999998e-06 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514383+0000", 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916408000000003 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514390+0000", 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5429999999999996e-06 2024-09-13T07:03:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129535 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.001547397, 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919205499999999, 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129535 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615232+0000", 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615236+0000", 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615244+0000", 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7369999999999995e-06 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615250+0000", 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2270000000000001e-06 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4627000000000002e-05 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5089999999999993e-06 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514412+0000", 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89907890599999996 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514425+0000", 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3138e-05 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129536 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.001513920000001, 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89918503500000002, 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129536 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615268+0000", 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8370000000000001e-06 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615288+0000", 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9945999999999998e-05 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615294+0000", 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3340000000000001e-06 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615374+0000", 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9988999999999996e-05 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615381+0000", 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514444+0000", 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89906347499999995 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514451+0000", 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7630000000000003e-06 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129537 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.001465368000002, 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916833399999996, 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129537 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615314+0000", 2024-09-13T07:03:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615317+0000", 2024-09-13T07:03:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1410000000000001e-06 2024-09-13T07:03:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3589999999999996e-06 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615329+0000", 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615408+0000", 2024-09-13T07:03:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8678999999999997e-05 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615414+0000", 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514471+0000", 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905719200000001 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514483+0000", 2024-09-13T07:03:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1691999999999999e-05 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129538 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.001446828999999, 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89917603999999995, 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129538 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615335+0000", 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4090000000000001e-06 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615346+0000", 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0866000000000001e-05 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615424+0000", 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7628999999999998e-05 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615459+0000", 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5046999999999998e-05 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615466+0000", 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7610000000000002e-06 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514503+0000", 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89903750299999996 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514509+0000", 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2140000000000001e-06 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129539 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.001302077999998, 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905735799999997, 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129539 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615477+0000", 2024-09-13T07:03:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615481+0000", 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6490000000000001e-06 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615488+0000", 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4359999999999999e-06 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615493+0000", 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3120000000000001e-06 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615529+0000", 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6468999999999999e-05 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615536+0000", 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9789999999999996e-06 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514529+0000", 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89899276400000006 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514535+0000", 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2920000000000001e-06 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129540 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.001231084, 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89901438700000003, 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129540 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615548+0000", 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615552+0000", 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8349999999999997e-06 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615559+0000", 2024-09-13T07:03:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-13T07:03:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615564+0000", 2024-09-13T07:03:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2360000000000004e-06 2024-09-13T07:03:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615600+0000", 2024-09-13T07:03:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6637000000000002e-05 2024-09-13T07:03:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615607+0000", 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.669e-06 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514557+0000", 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89894939399999996 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514563+0000", 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8560000000000001e-06 2024-09-13T07:03:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129541 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.001158472, 2024-09-13T07:03:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89896824600000003, 2024-09-13T07:03:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129541 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615621+0000", 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615625+0000", 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615631+0000", 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3890000000000002e-06 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615636+0000", 2024-09-13T07:03:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3489999999999999e-06 2024-09-13T07:03:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615671+0000", 2024-09-13T07:03:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4570000000000003e-05 2024-09-13T07:03:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615678+0000", 2024-09-13T07:03:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8510000000000001e-06 2024-09-13T07:03:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514583+0000", 2024-09-13T07:03:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89890530599999996 2024-09-13T07:03:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514590+0000", 2024-09-13T07:03:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7469999999999999e-06 2024-09-13T07:03:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129542 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.001084981999998, 2024-09-13T07:03:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89892092400000001, 2024-09-13T07:03:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129542 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615698+0000", 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.664e-06 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615705+0000", 2024-09-13T07:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615711+0000", 2024-09-13T07:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6470000000000003e-06 2024-09-13T07:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615752+0000", 2024-09-13T07:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0389999999999998e-05 2024-09-13T07:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615759+0000", 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9230000000000002e-06 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514609+0000", 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89884984899999998 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514616+0000", 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7299999999999999e-06 2024-09-13T07:03:14.678 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129543 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.001006108999999, 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89886872600000001, 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.679 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129543 2024-09-13T07:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615773+0000", 2024-09-13T07:03:14.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:14.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615777+0000", 2024-09-13T07:03:14.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1860000000000002e-06 2024-09-13T07:03:14.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615785+0000", 2024-09-13T07:03:14.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2710000000000003e-06 2024-09-13T07:03:14.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615790+0000", 2024-09-13T07:03:14.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3619999999999997e-06 2024-09-13T07:03:14.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615828+0000", 2024-09-13T07:03:14.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8059999999999998e-05 2024-09-13T07:03:14.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615836+0000", 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8809999999999996e-06 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514636+0000", 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89880009599999999 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514643+0000", 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.684 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129544 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.684 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:14.684 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.000930586999999, 2024-09-13T07:03:14.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89881970799999999, 2024-09-13T07:03:14.684 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.684 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.684 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129544 2024-09-13T07:03:14.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.685 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:14.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:14.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:14.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615853+0000", 2024-09-13T07:03:14.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8600000000000003e-06 2024-09-13T07:03:14.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615860+0000", 2024-09-13T07:03:14.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1709999999999996e-06 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615867+0000", 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7020000000000004e-06 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615905+0000", 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8265e-05 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615913+0000", 2024-09-13T07:03:14.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514663+0000", 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89874952500000005 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514669+0000", 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129545 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.000852414000001, 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89876792599999999, 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129545 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615927+0000", 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:14.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615931+0000", 2024-09-13T07:03:14.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8010000000000003e-06 2024-09-13T07:03:14.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615939+0000", 2024-09-13T07:03:14.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6310000000000003e-06 2024-09-13T07:03:14.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615947+0000", 2024-09-13T07:03:14.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7090000000000002e-06 2024-09-13T07:03:14.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615985+0000", 2024-09-13T07:03:14.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8389000000000001e-05 2024-09-13T07:03:14.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615993+0000", 2024-09-13T07:03:14.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2209999999999997e-06 2024-09-13T07:03:14.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514689+0000", 2024-09-13T07:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89869535199999995 2024-09-13T07:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514696+0000", 2024-09-13T07:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8179999999999998e-06 2024-09-13T07:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129546 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.000772326, 2024-09-13T07:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89872448199999999, 2024-09-13T07:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129546 2024-09-13T07:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:14.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:14.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616007+0000", 2024-09-13T07:03:14.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616011+0000", 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1659999999999996e-06 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616018+0000", 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0829999999999998e-06 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616024+0000", 2024-09-13T07:03:14.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8170000000000004e-06 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616063+0000", 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8708000000000003e-05 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616070+0000", 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6299999999999998e-06 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514721+0000", 2024-09-13T07:03:14.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89865028800000002 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514732+0000", 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1452e-05 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129547 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.000696997999999, 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89868529799999997, 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.699 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129547 2024-09-13T07:03:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616086+0000", 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.394e-06 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616093+0000", 2024-09-13T07:03:14.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1130000000000003e-06 2024-09-13T07:03:14.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616100+0000", 2024-09-13T07:03:14.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1520000000000003e-06 2024-09-13T07:03:14.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616138+0000", 2024-09-13T07:03:14.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8084000000000003e-05 2024-09-13T07:03:14.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616146+0000", 2024-09-13T07:03:14.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3010000000000001e-06 2024-09-13T07:03:14.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:14.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89861259299999996 2024-09-13T07:03:14.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514768+0000", 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0175e-05 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129548 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.000612939, 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89862847899999998, 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.704 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129548 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616166+0000", 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616170+0000", 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8619999999999996e-06 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616177+0000", 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0450000000000003e-06 2024-09-13T07:03:14.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616183+0000", 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.139e-06 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616221+0000", 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7320000000000002e-05 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616228+0000", 2024-09-13T07:03:14.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2980000000000003e-06 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514789+0000", 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89856044599999996 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514796+0000", 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0809999999999996e-06 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.709 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129549 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.709 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:14.709 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.000539476, 2024-09-13T07:03:14.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89859381900000002, 2024-09-13T07:03:14.709 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.709 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.709 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129549 2024-09-13T07:03:14.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.709 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616240+0000", 2024-09-13T07:03:14.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616244+0000", 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4149999999999999e-06 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616251+0000", 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616256+0000", 2024-09-13T07:03:14.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3299999999999998e-06 2024-09-13T07:03:14.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616302+0000", 2024-09-13T07:03:14.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5646999999999999e-05 2024-09-13T07:03:14.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616309+0000", 2024-09-13T07:03:14.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5530000000000004e-06 2024-09-13T07:03:14.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514825+0000", 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89851528400000003 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514834+0000", 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8349999999999999e-06 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.713 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129550 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:14.714 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:14.714 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.000455097, 2024-09-13T07:03:14.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89854296199999995, 2024-09-13T07:03:14.714 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:14.714 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:14.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:14.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:14.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:14.714 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129550 2024-09-13T07:03:14.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.714 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:14.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616324+0000", 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616328+0000", 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7479999999999999e-06 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616335+0000", 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9280000000000004e-06 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616342+0000", 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2690000000000002e-06 2024-09-13T07:03:14.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616380+0000", 2024-09-13T07:03:14.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:03:14.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:14.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616387+0000", 2024-09-13T07:03:14.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.075e-06 2024-09-13T07:03:14.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:14.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514861+0000", 2024-09-13T07:03:14.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89847450799999995 2024-09-13T07:03:14.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:14.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:14.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:14.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514868+0000", 2024-09-13T07:03:14.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6050000000000003e-06 2024-09-13T07:03:14.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.718 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:14.718 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:14.718 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:14.720 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:14.720 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:14.715+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:14.821 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:14.819+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:14.834 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:14.835 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:14.835 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:14.835 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:14.844 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:14.922 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:14.923+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:14.948 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:14.948 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:14.948 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:14.948 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:14.948 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:14.958 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:15.023 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:15.019+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:15.069 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.008978226, 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:03:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:03:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:03:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:03:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:03:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:03:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:03:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:03:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:03:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:03:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:03:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:03:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:03:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:03:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:03:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:03:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93979 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451218912999998, 2024-09-13T07:03:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042253801000000001, 2024-09-13T07:03:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93979 2024-09-13T07:03:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616683+0000", 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2820000000000001e-06 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616691+0000", 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0779999999999996e-06 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616698+0000", 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658799+0000", 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042100763999999999 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658869+0000", 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0065000000000001e-05 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658934+0000", 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5071000000000001e-05 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451194348000001, 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:03:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:03:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:03:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:03:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:03:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:03:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451171381000002, 2024-09-13T07:03:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:03:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:03:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:03:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:03:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:03:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:03:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:03:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451154203000002, 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:03:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:03:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:03:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:03:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:03:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:03:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:03:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:03:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:03:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451140338000002, 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:03:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:03:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:03:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:03:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451127192999998, 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:03:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:03:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:03:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:03:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:03:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:03:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451113986000003, 2024-09-13T07:03:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:03:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451099919000001, 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451079905, 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:03:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:03:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451060155, 2024-09-13T07:03:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:03:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:03:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:03:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:03:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:03:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:03:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:03:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:03:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:03:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:03:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:03:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451035349000001, 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:03:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:03:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:03:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:03:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:03:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:03:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:03:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451011084000001, 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:03:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.450969350999998, 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:03:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:03:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:03:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:03:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:03:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.450933061999997, 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:03:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:03:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:03:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:03:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:03:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:03:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:03:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:03:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:03:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:03:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:03:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.007829729999997, 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:03:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:03:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:03:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:03:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:03:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:03:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:03:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:03:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:03:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:03:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.007812848, 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:03:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.006762291000001, 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.005087742000001, 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:03:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:03:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:03:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.484505351999999, 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:03:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:03:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:03:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:03:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:03:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:03:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:03:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:03:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:03:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.166 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:15.168 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:03:15.168 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:15.169 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:15.166+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:15.268 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:15.263+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:15.286 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:15.286 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:15.286 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:15.286 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:15.296 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:15.319 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:03:15.369 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:15.366+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:15.400 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:15.400 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:15.400 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:15.400 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:15.401 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:15.410 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:15.470 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:15.466+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:15.471 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:15.490 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:15.513 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:15.530 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:15.551 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:15.570 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:15.571 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:15.567+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:15.575 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:15.575 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:15.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:15.575 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:15.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.575 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:03:15.576 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:15.576 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.519300122, 2024-09-13T07:03:15.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:03:15.576 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.576 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:15.576 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:15.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:15.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:03:15.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:15.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:03:15.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:03:15.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:03:15.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:03:15.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:15.580 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:15.581 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.361418888000003, 2024-09-13T07:03:15.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:03:15.581 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.581 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:15.581 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.581 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.581 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.581 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:03:15.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.581 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:03:15.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:03:15.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:03:15.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:03:15.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:03:15.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:03:15.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:03:15.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:03:15.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:03:15.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:03:15.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:15.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:03:15.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:03:15.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:15.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:03:15.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:03:15.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:15.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:03:15.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:03:15.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:15.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:03:15.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:03:15.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:03:15.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:03:15.586 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.587 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.587 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:15.587 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:15.587 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.335649176999993, 2024-09-13T07:03:15.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:03:15.587 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.587 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:15.587 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.587 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.587 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:03:15.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:15.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:03:15.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:03:15.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:03:15.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:03:15.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:03:15.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:03:15.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:03:15.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:03:15.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:15.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:03:15.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:03:15.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:15.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:03:15.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:03:15.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:03:15.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:03:15.593 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.593 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.593 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.593 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:15.593 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:15.593 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.314498010999998, 2024-09-13T07:03:15.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:03:15.593 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.593 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:15.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.594 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.594 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.594 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:03:15.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.594 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:15.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:15.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:03:15.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:15.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:03:15.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:03:15.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:03:15.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:03:15.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:03:15.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:03:15.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:15.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:03:15.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:03:15.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:15.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:03:15.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:03:15.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:15.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:03:15.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 67.197453248000002, 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:03:15.599 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:03:15.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:03:15.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:03:15.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:03:15.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:03:15.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:03:15.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.605 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.605 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.605 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:15.605 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.912456003000003, 2024-09-13T07:03:15.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:03:15.606 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.606 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.606 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:03:15.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.606 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:15.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:03:15.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:03:15.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:03:15.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:03:15.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:03:15.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:03:15.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:03:15.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:03:15.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:03:15.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:03:15.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:03:15.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:03:15.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.610 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.610 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94598 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:15.610 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:03:15.610 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.912285281999999, 2024-09-13T07:03:15.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044365860999999999, 2024-09-13T07:03:15.610 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.610 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.611 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94598 2024-09-13T07:03:15.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.611 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:03:15.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:03:15.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661034+0000", 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9998465 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661192+0000", 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015813000000000001 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661199+0000", 2024-09-13T07:03:15.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5689999999999999e-06 2024-09-13T07:03:15.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:03:15.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:03:15.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705452+0000", 2024-09-13T07:03:15.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044246273000000003 2024-09-13T07:03:15.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705501+0000", 2024-09-13T07:03:15.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8748000000000002e-05 2024-09-13T07:03:15.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705553+0000", 2024-09-13T07:03:15.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2241999999999999e-05 2024-09-13T07:03:15.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107060 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1003520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.660762992999999, 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044341754999999997, 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.615 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107060 2024-09-13T07:03:15.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.616 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:03:15.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:03:15.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912709+0000", 2024-09-13T07:03:15.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912712+0000", 2024-09-13T07:03:15.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6000000000000001e-06 2024-09-13T07:03:15.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912717+0000", 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3689999999999998e-06 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912722+0000", 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7219999999999999e-06 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956876+0000", 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044154469000000002 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956957+0000", 2024-09-13T07:03:15.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1231999999999997e-05 2024-09-13T07:03:15.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957051+0000", 2024-09-13T07:03:15.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3858999999999997e-05 2024-09-13T07:03:15.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.619 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.620 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:15.620 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:15.620 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.660746865, 2024-09-13T07:03:15.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:03:15.620 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.620 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.621 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.621 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:03:15.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.621 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:15.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:15.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:03:15.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:03:15.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:03:15.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:03:15.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:03:15.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:03:15.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:03:15.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:03:15.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:03:15.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:03:15.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:03:15.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:03:15.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:03:15.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.624 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.625 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:15.625 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:15.625 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.660728831, 2024-09-13T07:03:15.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:03:15.625 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.625 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.625 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:03:15.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.625 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:03:15.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:03:15.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:03:15.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:03:15.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:03:15.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:03:15.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:03:15.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:03:15.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:03:15.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:03:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:03:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:03:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:03:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.660712750999998, 2024-09-13T07:03:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:03:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.630 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:03:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:03:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:03:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:03:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:03:15.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:03:15.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.660697075000002, 2024-09-13T07:03:15.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:03:15.635 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.635 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:15.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.635 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:03:15.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.635 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:15.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:15.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:03:15.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:03:15.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:03:15.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:03:15.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.660674153000002, 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:03:15.639 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.640 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:15.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.640 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:03:15.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.640 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:15.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:15.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:15.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:03:15.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:03:15.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:03:15.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:03:15.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:03:15.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.644 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:15.644 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:15.644 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.660658056000003, 2024-09-13T07:03:15.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:03:15.644 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.644 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:15.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.644 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:03:15.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.644 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:15.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:15.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.648 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107067 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:15.648 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:15.648 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.660098038999998, 2024-09-13T07:03:15.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04441055, 2024-09-13T07:03:15.648 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.648 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:15.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.648 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107067 2024-09-13T07:03:15.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.648 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:15.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:15.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912832+0000", 2024-09-13T07:03:15.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9994574 2024-09-13T07:03:15.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913379+0000", 2024-09-13T07:03:15.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00054719999999999997 2024-09-13T07:03:15.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913386+0000", 2024-09-13T07:03:15.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4549999999999998e-06 2024-09-13T07:03:15.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913393+0000", 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6259999999999997e-06 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957776+0000", 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044383143999999999 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957785+0000", 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9220000000000003e-06 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107068 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.660073607999998, 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044410502999999997, 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107068 2024-09-13T07:03:15.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913401+0000", 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913407+0000", 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7520000000000002e-06 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913412+0000", 2024-09-13T07:03:15.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8550000000000001e-06 2024-09-13T07:03:15.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957802+0000", 2024-09-13T07:03:15.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044389606999999998 2024-09-13T07:03:15.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957809+0000", 2024-09-13T07:03:15.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9160000000000001e-06 2024-09-13T07:03:15.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.656 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.656 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.656 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:15.656 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:15.656 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.660056443999999, 2024-09-13T07:03:15.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:03:15.656 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.656 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:15.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.657 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:03:15.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.657 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:15.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:15.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:03:15.659 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.660 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.660 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.660 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:15.660 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:15.660 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.660013667999998, 2024-09-13T07:03:15.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:03:15.660 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.660 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:15.660 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:15.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:03:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:03:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:03:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:03:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:03:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:03:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.502995898000002, 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.664 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:15.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.665 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:03:15.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.665 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:15.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:03:15.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:03:15.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:03:15.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:03:15.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:03:15.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.6506189679999999, 2024-09-13T07:03:15.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:03:15.669 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:15.669 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:15.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:15.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:15.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:15.669 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:03:15.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.669 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:15.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:15.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:15.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:15.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:15.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:15.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:15.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:03:15.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:15.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:15.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:03:15.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:03:15.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:15.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:03:15.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:03:15.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:15.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:03:15.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:03:15.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:03:15.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:03:15.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:15.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:03:15.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:03:15.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:15.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:03:15.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:03:15.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:15.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:15.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:15.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:03:15.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:03:15.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.673 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:15.673 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:15.674 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:15.675 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:15.675 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:15.676 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:15.677 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:15.677 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:15.677 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:15.677 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:15.677 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:15.677 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:15.677 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:15.677 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:15.678 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:15.671+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:15.775 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:15.771+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:15.835 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:15.835 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:15.845 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:15.876 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:15.874+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:15.949 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:15.950 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:15.959 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:15.977 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:15.974+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:16.065 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:16.065 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:16.075 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:16.077 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:16.075+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:16.079 INFO:teuthology.orchestra.run.smithi118.stdout:4953 write 0x37863fa thru 0x378ef9b (0x8ba2 bytes) 2024-09-13T07:03:16.102 INFO:teuthology.orchestra.run.smithi118.stdout:4955 writesame 0x8158f75 thru 0x81620e2 data_size 0x55(0x916e bytes) 2024-09-13T07:03:16.119 INFO:teuthology.orchestra.run.smithi118.stdout:4957 write 0x6ed4135 thru 0x6edb816 (0x76e2 bytes) 2024-09-13T07:03:16.135 INFO:teuthology.orchestra.run.smithi118.stdout:4958 write 0x157b86d thru 0x1588b2b (0xd2bf bytes) 2024-09-13T07:03:16.158 INFO:teuthology.orchestra.run.smithi118.stdout:4959 trunc from 0x90a8452 to 0x299f20d 2024-09-13T07:03:16.178 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:16.178+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:16.198 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:16.198 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:16.198 INFO:teuthology.orchestra.run.smithi118.stdout:4960 writesame 0x3f2f72e thru 0x3f35104 data_size 0x1(0x59d7 bytes) 2024-09-13T07:03:16.211 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:16.224 INFO:teuthology.orchestra.run.smithi118.stdout:4961 write 0xdaeaaef thru 0xdaf736d (0xc87f bytes) 2024-09-13T07:03:16.249 INFO:teuthology.orchestra.run.smithi118.stdout:4962 read 0x9655c27 thru 0x96594f9 (0x38d3 bytes) 2024-09-13T07:03:16.249 INFO:teuthology.orchestra.run.smithi118.stdout:4963 writesame 0x48c09b4 thru 0x48cd57c data_size 0x1(0xcbc9 bytes) 2024-09-13T07:03:16.279 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:16.275+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:16.280 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:16.280 INFO:teuthology.orchestra.run.smithi118.stdout:4966 writesame 0x8d6cf1c thru 0x8d7a061 data_size 0x3(0xd146 bytes) 2024-09-13T07:03:16.322 INFO:teuthology.orchestra.run.smithi118.stdout:4967 trunc from 0xdaf736e to 0x7f3b91 2024-09-13T07:03:16.337 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:16.337 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:16.351 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:16.361 INFO:teuthology.orchestra.run.smithi118.stdout:4969 read 0x535ff8 thru 0x5408cd (0xa8d6 bytes) 2024-09-13T07:03:16.363 INFO:teuthology.orchestra.run.smithi118.stdout:4970 read 0x35c94f thru 0x36464b (0x7cfd bytes) 2024-09-13T07:03:16.364 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:16.380 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:16.375+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:16.381 INFO:teuthology.orchestra.run.smithi118.stdout:4972 write 0x57cd310 thru 0x57d6df8 (0x9ae9 bytes) 2024-09-13T07:03:16.412 INFO:teuthology.orchestra.run.smithi118.stdout:4973 write 0x6be4bae thru 0x6be998d (0x4de0 bytes) 2024-09-13T07:03:16.426 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:16.426 INFO:teuthology.orchestra.run.smithi118.stdout:4975 read 0x5dbff9e thru 0x5dcd9be (0xda21 bytes) 2024-09-13T07:03:16.426 INFO:teuthology.orchestra.run.smithi118.stdout:4976 punch from 0x22e99f6 to 0x22ef6d3, (0x5cdd bytes) 2024-09-13T07:03:16.426 INFO:teuthology.orchestra.run.smithi118.stdout:4977 punch from 0x842652 to 0x852370, (0xfd1e bytes) 2024-09-13T07:03:16.426 INFO:teuthology.orchestra.run.smithi118.stdout:4978 read 0x1f30340 thru 0x1f3be8c (0xbb4d bytes) 2024-09-13T07:03:16.426 INFO:teuthology.orchestra.run.smithi118.stdout:4979 punch from 0x1e096b4 to 0x1e09bb0, (0x4fc bytes) 2024-09-13T07:03:16.426 INFO:teuthology.orchestra.run.smithi118.stdout:4981 read 0x5fc6638 thru 0x5fd2903 (0xc2cc bytes) 2024-09-13T07:03:16.426 INFO:teuthology.orchestra.run.smithi118.stdout:4982 read 0x52a808 thru 0x52c043 (0x183c bytes) 2024-09-13T07:03:16.435 INFO:teuthology.orchestra.run.smithi118.stdout:4983 trunc from 0x6be998e to 0xbba8c5b 2024-09-13T07:03:16.445 INFO:teuthology.orchestra.run.smithi118.stdout:4984 write 0x40dd8e5 thru 0x40eb36e (0xda8a bytes) 2024-09-13T07:03:16.473 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:16.473 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:16.474 INFO:teuthology.orchestra.run.smithi118.stdout:4987 read 0x46520ff thru 0x465b6f6 (0x95f8 bytes) 2024-09-13T07:03:16.474 INFO:teuthology.orchestra.run.smithi118.stdout:4989 punch from 0x2a8eb39 to 0x2a93df5, (0x52bc bytes) 2024-09-13T07:03:16.475 INFO:teuthology.orchestra.run.smithi118.stdout:4990 punch from 0x5d40618 to 0x5d41376, (0xd5e bytes) 2024-09-13T07:03:16.475 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:16.475 INFO:teuthology.orchestra.run.smithi118.stdout:4992 punch from 0x2498917 to 0x249e467, (0x5b50 bytes) 2024-09-13T07:03:16.475 INFO:teuthology.orchestra.run.smithi118.stdout:4993 read 0x9b168cf thru 0x9b1851b (0x1c4d bytes) 2024-09-13T07:03:16.475 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:16.475 INFO:teuthology.orchestra.run.smithi118.stdout:4996 flatten 2024-09-13T07:03:16.481 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:16.479+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:16.483 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:16.490 INFO:teuthology.orchestra.run.smithi118.stdout:5001 trunc from 0xbba8c5b to 0x759ee6b 2024-09-13T07:03:16.521 INFO:teuthology.orchestra.run.smithi118.stdout:5002 write 0xa59af31 thru 0xa5a4636 (0x9706 bytes) 2024-09-13T07:03:16.558 INFO:teuthology.orchestra.run.smithi118.stdout:5003 trunc from 0xa5a4637 to 0x292d17b 2024-09-13T07:03:16.582 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:16.578+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:16.586 INFO:teuthology.orchestra.run.smithi118.stdout:5005 trunc from 0x292d17b to 0x25e8474 2024-09-13T07:03:16.597 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:16.597 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:16.609 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:16.615 INFO:teuthology.orchestra.run.smithi118.stdout:5006 writesame 0x74bc258 thru 0x74c42a0 data_size 0x59(0x8049 bytes) 2024-09-13T07:03:16.615 INFO:teuthology.orchestra.run.smithi118.stdout:5007 read 0x17e3e2b thru 0x17ee85f (0xaa35 bytes) 2024-09-13T07:03:16.628 INFO:teuthology.orchestra.run.smithi118.stdout:5008 writesame 0x807b5d1 thru 0x8085592 data_size 0x2(0x9fc2 bytes) 2024-09-13T07:03:16.629 INFO:teuthology.orchestra.run.smithi118.stdout:5009 read 0x6afaa3f thru 0x6b08f48 (0xe50a bytes) 2024-09-13T07:03:16.629 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:16.629 INFO:teuthology.orchestra.run.smithi118.stdout:5012 write 0x4cbcc41 thru 0x4cc92b8 (0xc678 bytes) 2024-09-13T07:03:16.655 INFO:teuthology.orchestra.run.smithi118.stdout:5014 read 0x33635fe thru 0x336b0ff (0x7b02 bytes) 2024-09-13T07:03:16.655 INFO:teuthology.orchestra.run.smithi118.stdout:5015 read 0x3578262 thru 0x3584524 (0xc2c3 bytes) 2024-09-13T07:03:16.673 INFO:teuthology.orchestra.run.smithi118.stdout:5017 writesame 0xcb60067 thru 0xcb6dc2e data_size 0xd(0xdbc8 bytes) 2024-09-13T07:03:16.673 INFO:teuthology.orchestra.run.smithi118.stdout:5018 read 0x25b4b6d thru 0x25bf050 (0xa4e4 bytes) 2024-09-13T07:03:16.673 INFO:teuthology.orchestra.run.smithi118.stdout:5021 write 0xc0db4fa thru 0xc0e05e5 (0x50ec bytes) 2024-09-13T07:03:16.682 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:16.679+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:16.690 INFO:teuthology.orchestra.run.smithi118.stdout:5022 read 0x4057eec thru 0x405fb89 (0x7c9e bytes) 2024-09-13T07:03:16.690 INFO:teuthology.orchestra.run.smithi118.stdout:5023 read 0x8f2945f thru 0x8f3812c (0xecce bytes) 2024-09-13T07:03:16.690 INFO:teuthology.orchestra.run.smithi118.stdout:5024 read 0x7ce353b thru 0x7ce5d4d (0x2813 bytes) 2024-09-13T07:03:16.690 INFO:teuthology.orchestra.run.smithi118.stdout:5025 write 0xc740747 thru 0xc74848e (0x7d48 bytes) 2024-09-13T07:03:16.709 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:16.709 INFO:teuthology.orchestra.run.smithi118.stdout:5027 write 0x44d2a3e thru 0x44dd20d (0xa7d0 bytes) 2024-09-13T07:03:16.728 INFO:teuthology.orchestra.run.smithi118.stdout:5028 punch from 0xa279ec1 to 0xa289549, (0xf688 bytes) 2024-09-13T07:03:16.737 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:16.737 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:16.737 INFO:teuthology.orchestra.run.smithi118.stdout:5029 write 0xcfefbd8 thru 0xcff94b7 (0x98e0 bytes) 2024-09-13T07:03:16.747 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:16.758 INFO:teuthology.orchestra.run.smithi118.stdout:5030 read 0x647db91 thru 0x648d288 (0xf6f8 bytes) 2024-09-13T07:03:16.758 INFO:teuthology.orchestra.run.smithi118.stdout:5031 writesame 0x7fc69ab thru 0x7fca581 data_size 0x1(0x3bd7 bytes) 2024-09-13T07:03:16.769 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:16.769 INFO:teuthology.orchestra.run.smithi118.stdout:5033 write 0xa8f404e thru 0xa901341 (0xd2f4 bytes) 2024-09-13T07:03:16.783 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:16.782+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:16.794 INFO:teuthology.orchestra.run.smithi118.stdout:5034 read 0x5970c52 thru 0x59714ad (0x85c bytes) 2024-09-13T07:03:16.794 INFO:teuthology.orchestra.run.smithi118.stdout:5035 writesame 0xb88da3c thru 0xb88fc4b data_size 0x28(0x2210 bytes) 2024-09-13T07:03:16.814 INFO:teuthology.orchestra.run.smithi118.stdout:5037 writesame 0xdaca6c9 thru 0xdad5fc9 data_size 0x1(0xb901 bytes) 2024-09-13T07:03:16.814 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:16.821 INFO:teuthology.orchestra.run.smithi118.stdout:5042 trunc from 0xdad5fca to 0x99ccf4d 2024-09-13T07:03:16.849 INFO:teuthology.orchestra.run.smithi118.stdout:5045 punch from 0x6d44df to 0x6e07b7, (0xc2d8 bytes) 2024-09-13T07:03:16.849 INFO:teuthology.orchestra.run.smithi118.stdout:5046 read 0x7454149 thru 0x7455ea0 (0x1d58 bytes) 2024-09-13T07:03:16.849 INFO:teuthology.orchestra.run.smithi118.stdout:5047 write 0x91b5877 thru 0x91bc87b (0x7005 bytes) 2024-09-13T07:03:16.866 INFO:teuthology.orchestra.run.smithi118.stdout:5048 trunc from 0x99ccf4d to 0x9dd47d7 2024-09-13T07:03:16.870 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:16.870 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:16.875 INFO:teuthology.orchestra.run.smithi118.stdout:5049 write 0x56d607e thru 0x56e5b76 (0xfaf9 bytes) 2024-09-13T07:03:16.880 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:16.910 INFO:teuthology.orchestra.run.smithi118.stdout:5051 trunc from 0x9dd47d7 to 0x531130a 2024-09-13T07:03:16.910 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:16.907+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:16.953 INFO:teuthology.orchestra.run.smithi118.stdout:5052 writesame 0xa4a1a02 thru 0xa4a9de2 data_size 0x35(0x83e1 bytes) 2024-09-13T07:03:16.953 INFO:teuthology.orchestra.run.smithi118.stdout:5053 punch from 0x111d4b2 to 0x112819b, (0xace9 bytes) 2024-09-13T07:03:16.953 INFO:teuthology.orchestra.run.smithi118.stdout:5054 read 0x9fc4bed thru 0x9fcd6e7 (0x8afb bytes) 2024-09-13T07:03:16.953 INFO:teuthology.orchestra.run.smithi118.stdout:5055 read 0x1f4e0cf thru 0x1f54e84 (0x6db6 bytes) 2024-09-13T07:03:16.953 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:16.953 INFO:teuthology.orchestra.run.smithi118.stdout:5057 read 0x5b03a81 thru 0x5b07b64 (0x40e4 bytes) 2024-09-13T07:03:16.953 INFO:teuthology.orchestra.run.smithi118.stdout:5059 punch from 0x961c097 to 0x961cafc, (0xa65 bytes) 2024-09-13T07:03:16.953 INFO:teuthology.orchestra.run.smithi118.stdout:5060 punch from 0x1147dd9 to 0x115432d, (0xc554 bytes) 2024-09-13T07:03:16.953 INFO:teuthology.orchestra.run.smithi118.stdout:5061 write 0x8d6d3f8 thru 0x8d74358 (0x6f61 bytes) 2024-09-13T07:03:16.985 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:16.982+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:16.987 INFO:teuthology.orchestra.run.smithi118.stdout:5062 writesame 0xbfaa114 thru 0xbfb6168 data_size 0x1(0xc055 bytes) 2024-09-13T07:03:16.987 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:16.987 INFO:teuthology.orchestra.run.smithi118.stdout:5064 write 0x6e08770 thru 0x6e0beab (0x373c bytes) 2024-09-13T07:03:16.998 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:16.998 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:17.017 INFO:teuthology.orchestra.run.smithi118.stdout:5072 writesame 0xee33f26 thru 0xee3f79f data_size 0x11(0xb87a bytes) 2024-09-13T07:03:17.021 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:17.021 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:17.035 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:17.037 INFO:teuthology.orchestra.run.smithi118.stdout:5073 trunc from 0xee3f7a0 to 0x25bc31a 2024-09-13T07:03:17.085 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:17.083+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:17.139 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:17.139 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:17.152 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:17.170 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:03:17.217 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:17.215+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:17.260 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:17.260 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:17.270 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:17.314 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:17.314+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:17.387 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:17.387 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:17.388 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:17.383+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:17.397 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:17.488 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:17.486+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:17.499 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:17.499 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:17.509 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:17.589 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:17.587+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:17.661 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:17.661 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:17.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:03:17.679 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:03:17.725 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:17.723+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:17.780 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:17.780 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:17.791 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:03:17.791 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:17.790+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:17.814 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:03:17.843 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:17.857 INFO:teuthology.orchestra.run.smithi118.stdout:5074 writesame 0xa946974 thru 0xa952a93 data_size 0xa0(0xc120 bytes) 2024-09-13T07:03:17.857 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:17.866 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:17.871 INFO:teuthology.orchestra.run.smithi118.stdout:5076 trunc from 0xa952a94 to 0x1d9e277 2024-09-13T07:03:17.885 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:17.892 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:17.887+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:17.903 INFO:teuthology.orchestra.run.smithi118.stdout:5079 write 0x7da155e thru 0x7da301a (0x1abd bytes) 2024-09-13T07:03:17.903 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:17.909 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:17.909 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:17.915 INFO:teuthology.orchestra.run.smithi118.stdout:5080 read 0x71acbfe thru 0x71ae3b3 (0x17b6 bytes) 2024-09-13T07:03:17.915 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:17.915 INFO:teuthology.orchestra.run.smithi118.stdout:5082 read 0xcf034d thru 0xcfd5f9 (0xd2ad bytes) 2024-09-13T07:03:17.920 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:03:17.927 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:17.936 INFO:teuthology.orchestra.run.smithi118.stdout:5085 write 0xe2f809a thru 0xe2fde64 (0x5dcb bytes) 2024-09-13T07:03:17.949 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:17.951 INFO:teuthology.orchestra.run.smithi118.stdout:5086 read 0x88e49c3 thru 0x88ebd5b (0x7399 bytes) 2024-09-13T07:03:17.951 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:17.951 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:17.951 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:17.951 INFO:teuthology.orchestra.run.smithi118.stdout:5090 write 0xe255f1d thru 0xe265848 (0xf92c bytes) 2024-09-13T07:03:17.964 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:17.964 INFO:teuthology.orchestra.run.smithi118.stdout:5092 read 0xdf665b2 thru 0xdf70ca3 (0xa6f2 bytes) 2024-09-13T07:03:17.975 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:17.978 INFO:teuthology.orchestra.run.smithi118.stdout:5093 trunc from 0xe2fde65 to 0x65dbdfd 2024-09-13T07:03:17.992 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:17.990+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:17.998 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:18.000 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:18.000 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:18.000 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:18.000 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:18.000 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:18.000 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:18.000 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:18.000 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:18.001 INFO:teuthology.orchestra.run.smithi118.stdout:5096 read 0x4c5beb6 thru 0x4c6a57f (0xe6ca bytes) 2024-09-13T07:03:18.001 INFO:teuthology.orchestra.run.smithi118.stdout:5097 writesame 0x1b5a632 thru 0x1b5ee94 data_size 0x9(0x4863 bytes) 2024-09-13T07:03:18.012 INFO:teuthology.orchestra.run.smithi118.stdout:5098 read 0x2958247 thru 0x295b9c7 (0x3781 bytes) 2024-09-13T07:03:18.012 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:18.017 INFO:teuthology.orchestra.run.smithi118.stdout:5100 trunc from 0x65dbdfd to 0x449c236 2024-09-13T07:03:18.041 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:18.042 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:18.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:18.042 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:18.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.042 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:03:18.042 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:18.042 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.922283671000002, 2024-09-13T07:03:18.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:03:18.042 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.042 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:18.043 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.043 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.043 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.043 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:03:18.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.043 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:18.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:18.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:03:18.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:18.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:03:18.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:03:18.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:03:18.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:03:18.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:03:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:03:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:03:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:03:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:03:18.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:03:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:03:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:03:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:03:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:03:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:18.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:03:18.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:03:18.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:18.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:03:18.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:03:18.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:18.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:03:18.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:03:18.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:18.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:03:18.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:03:18.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:03:18.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:03:18.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.050 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.050 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.050 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129532 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.050 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:18.050 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.425880784, 2024-09-13T07:03:18.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89889205500000002, 2024-09-13T07:03:18.050 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.050 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.051 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.051 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129532 2024-09-13T07:03:18.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.051 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:18.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:18.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:18.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614994+0000", 2024-09-13T07:03:18.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.044e-06 2024-09-13T07:03:18.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615000+0000", 2024-09-13T07:03:18.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9229999999999996e-06 2024-09-13T07:03:18.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615005+0000", 2024-09-13T07:03:18.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:03:18.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615060+0000", 2024-09-13T07:03:18.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4561000000000002e-05 2024-09-13T07:03:18.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615069+0000", 2024-09-13T07:03:18.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1540000000000008e-06 2024-09-13T07:03:18.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513869+0000", 2024-09-13T07:03:18.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89879977700000002 2024-09-13T07:03:18.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513883+0000", 2024-09-13T07:03:18.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4361e-05 2024-09-13T07:03:18.055 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.055 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.055 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.056 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129533 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.056 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:18.056 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.425785450999999, 2024-09-13T07:03:18.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.899270177, 2024-09-13T07:03:18.056 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.056 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.056 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.056 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.056 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.056 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129533 2024-09-13T07:03:18.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.057 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:18.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:18.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:18.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615090+0000", 2024-09-13T07:03:18.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.856e-06 2024-09-13T07:03:18.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615096+0000", 2024-09-13T07:03:18.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.703e-06 2024-09-13T07:03:18.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615102+0000", 2024-09-13T07:03:18.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.378e-06 2024-09-13T07:03:18.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615140+0000", 2024-09-13T07:03:18.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7966999999999997e-05 2024-09-13T07:03:18.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615147+0000", 2024-09-13T07:03:18.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8800000000000002e-06 2024-09-13T07:03:18.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514342+0000", 2024-09-13T07:03:18.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919565300000004 2024-09-13T07:03:18.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514357+0000", 2024-09-13T07:03:18.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4355000000000001e-05 2024-09-13T07:03:18.061 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.061 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.061 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.061 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129534 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.061 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:18.061 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.425712553, 2024-09-13T07:03:18.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89923080600000005, 2024-09-13T07:03:18.062 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.062 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.062 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129534 2024-09-13T07:03:18.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.062 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:18.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:18.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:18.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615162+0000", 2024-09-13T07:03:18.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3569999999999998e-06 2024-09-13T07:03:18.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615169+0000", 2024-09-13T07:03:18.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.455e-06 2024-09-13T07:03:18.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615175+0000", 2024-09-13T07:03:18.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3679999999999998e-06 2024-09-13T07:03:18.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615212+0000", 2024-09-13T07:03:18.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6733999999999998e-05 2024-09-13T07:03:18.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615218+0000", 2024-09-13T07:03:18.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7569999999999998e-06 2024-09-13T07:03:18.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514383+0000", 2024-09-13T07:03:18.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916408000000003 2024-09-13T07:03:18.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514390+0000", 2024-09-13T07:03:18.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5429999999999996e-06 2024-09-13T07:03:18.066 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.066 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.067 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129535 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.067 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:18.067 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.425639137000001, 2024-09-13T07:03:18.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919205499999999, 2024-09-13T07:03:18.067 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.067 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.067 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.067 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.067 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.068 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129535 2024-09-13T07:03:18.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.068 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:18.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:18.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615232+0000", 2024-09-13T07:03:18.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615236+0000", 2024-09-13T07:03:18.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:18.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615244+0000", 2024-09-13T07:03:18.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7369999999999995e-06 2024-09-13T07:03:18.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615250+0000", 2024-09-13T07:03:18.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2270000000000001e-06 2024-09-13T07:03:18.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:18.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4627000000000002e-05 2024-09-13T07:03:18.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:18.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5089999999999993e-06 2024-09-13T07:03:18.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514412+0000", 2024-09-13T07:03:18.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89907890599999996 2024-09-13T07:03:18.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514425+0000", 2024-09-13T07:03:18.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3138e-05 2024-09-13T07:03:18.072 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.072 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.072 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.072 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129536 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.072 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:18.072 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.425605659999999, 2024-09-13T07:03:18.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89918503500000002, 2024-09-13T07:03:18.072 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.073 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.073 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129536 2024-09-13T07:03:18.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.073 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:18.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615268+0000", 2024-09-13T07:03:18.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8370000000000001e-06 2024-09-13T07:03:18.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615288+0000", 2024-09-13T07:03:18.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9945999999999998e-05 2024-09-13T07:03:18.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615294+0000", 2024-09-13T07:03:18.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3340000000000001e-06 2024-09-13T07:03:18.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615374+0000", 2024-09-13T07:03:18.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9988999999999996e-05 2024-09-13T07:03:18.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615381+0000", 2024-09-13T07:03:18.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:03:18.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514444+0000", 2024-09-13T07:03:18.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89906347499999995 2024-09-13T07:03:18.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514451+0000", 2024-09-13T07:03:18.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7630000000000003e-06 2024-09-13T07:03:18.077 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.077 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.078 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.078 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129537 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.078 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:18.078 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.425557108, 2024-09-13T07:03:18.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916833399999996, 2024-09-13T07:03:18.078 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.078 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.078 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.078 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.078 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.078 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129537 2024-09-13T07:03:18.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.079 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:18.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:18.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615314+0000", 2024-09-13T07:03:18.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615317+0000", 2024-09-13T07:03:18.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1410000000000001e-06 2024-09-13T07:03:18.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:18.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3589999999999996e-06 2024-09-13T07:03:18.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615329+0000", 2024-09-13T07:03:18.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-13T07:03:18.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615408+0000", 2024-09-13T07:03:18.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8678999999999997e-05 2024-09-13T07:03:18.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615414+0000", 2024-09-13T07:03:18.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:03:18.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514471+0000", 2024-09-13T07:03:18.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905719200000001 2024-09-13T07:03:18.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514483+0000", 2024-09-13T07:03:18.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1691999999999999e-05 2024-09-13T07:03:18.083 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.083 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.083 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.084 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129538 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.084 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:18.084 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.425538569, 2024-09-13T07:03:18.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89917603999999995, 2024-09-13T07:03:18.084 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.084 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.084 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129538 2024-09-13T07:03:18.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.084 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:18.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:18.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:18.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615335+0000", 2024-09-13T07:03:18.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4090000000000001e-06 2024-09-13T07:03:18.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615346+0000", 2024-09-13T07:03:18.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0866000000000001e-05 2024-09-13T07:03:18.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615424+0000", 2024-09-13T07:03:18.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7628999999999998e-05 2024-09-13T07:03:18.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615459+0000", 2024-09-13T07:03:18.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5046999999999998e-05 2024-09-13T07:03:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615466+0000", 2024-09-13T07:03:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7610000000000002e-06 2024-09-13T07:03:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514503+0000", 2024-09-13T07:03:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89903750299999996 2024-09-13T07:03:18.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514509+0000", 2024-09-13T07:03:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2140000000000001e-06 2024-09-13T07:03:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129539 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:18.089 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.425393818, 2024-09-13T07:03:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905735799999997, 2024-09-13T07:03:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129539 2024-09-13T07:03:18.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615477+0000", 2024-09-13T07:03:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615481+0000", 2024-09-13T07:03:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6490000000000001e-06 2024-09-13T07:03:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615488+0000", 2024-09-13T07:03:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4359999999999999e-06 2024-09-13T07:03:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615493+0000", 2024-09-13T07:03:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3120000000000001e-06 2024-09-13T07:03:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615529+0000", 2024-09-13T07:03:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6468999999999999e-05 2024-09-13T07:03:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615536+0000", 2024-09-13T07:03:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9789999999999996e-06 2024-09-13T07:03:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514529+0000", 2024-09-13T07:03:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89899276400000006 2024-09-13T07:03:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514535+0000", 2024-09-13T07:03:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2920000000000001e-06 2024-09-13T07:03:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129540 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.425322823999998, 2024-09-13T07:03:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89901438700000003, 2024-09-13T07:03:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.096 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129540 2024-09-13T07:03:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:18.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615548+0000", 2024-09-13T07:03:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615552+0000", 2024-09-13T07:03:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8349999999999997e-06 2024-09-13T07:03:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615559+0000", 2024-09-13T07:03:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-13T07:03:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615564+0000", 2024-09-13T07:03:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2360000000000004e-06 2024-09-13T07:03:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615600+0000", 2024-09-13T07:03:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6637000000000002e-05 2024-09-13T07:03:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615607+0000", 2024-09-13T07:03:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.669e-06 2024-09-13T07:03:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514557+0000", 2024-09-13T07:03:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89894939399999996 2024-09-13T07:03:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514563+0000", 2024-09-13T07:03:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8560000000000001e-06 2024-09-13T07:03:18.101 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129541 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.425250212000002, 2024-09-13T07:03:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89896824600000003, 2024-09-13T07:03:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.102 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129541 2024-09-13T07:03:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615621+0000", 2024-09-13T07:03:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615625+0000", 2024-09-13T07:03:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-13T07:03:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615631+0000", 2024-09-13T07:03:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3890000000000002e-06 2024-09-13T07:03:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615636+0000", 2024-09-13T07:03:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3489999999999999e-06 2024-09-13T07:03:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615671+0000", 2024-09-13T07:03:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4570000000000003e-05 2024-09-13T07:03:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615678+0000", 2024-09-13T07:03:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8510000000000001e-06 2024-09-13T07:03:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514583+0000", 2024-09-13T07:03:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89890530599999996 2024-09-13T07:03:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514590+0000", 2024-09-13T07:03:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7469999999999999e-06 2024-09-13T07:03:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129542 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.425176722, 2024-09-13T07:03:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89892092400000001, 2024-09-13T07:03:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129542 2024-09-13T07:03:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615698+0000", 2024-09-13T07:03:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.664e-06 2024-09-13T07:03:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615705+0000", 2024-09-13T07:03:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615711+0000", 2024-09-13T07:03:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6470000000000003e-06 2024-09-13T07:03:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615752+0000", 2024-09-13T07:03:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0389999999999998e-05 2024-09-13T07:03:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615759+0000", 2024-09-13T07:03:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9230000000000002e-06 2024-09-13T07:03:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514609+0000", 2024-09-13T07:03:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89884984899999998 2024-09-13T07:03:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514616+0000", 2024-09-13T07:03:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7299999999999999e-06 2024-09-13T07:03:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129543 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.425097849, 2024-09-13T07:03:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89886872600000001, 2024-09-13T07:03:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129543 2024-09-13T07:03:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615773+0000", 2024-09-13T07:03:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615777+0000", 2024-09-13T07:03:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1860000000000002e-06 2024-09-13T07:03:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615785+0000", 2024-09-13T07:03:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2710000000000003e-06 2024-09-13T07:03:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615790+0000", 2024-09-13T07:03:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3619999999999997e-06 2024-09-13T07:03:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615828+0000", 2024-09-13T07:03:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8059999999999998e-05 2024-09-13T07:03:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615836+0000", 2024-09-13T07:03:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8809999999999996e-06 2024-09-13T07:03:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514636+0000", 2024-09-13T07:03:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89880009599999999 2024-09-13T07:03:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514643+0000", 2024-09-13T07:03:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129544 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.425022327000001, 2024-09-13T07:03:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89881970799999999, 2024-09-13T07:03:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129544 2024-09-13T07:03:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615853+0000", 2024-09-13T07:03:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8600000000000003e-06 2024-09-13T07:03:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615860+0000", 2024-09-13T07:03:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1709999999999996e-06 2024-09-13T07:03:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615867+0000", 2024-09-13T07:03:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7020000000000004e-06 2024-09-13T07:03:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615905+0000", 2024-09-13T07:03:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8265e-05 2024-09-13T07:03:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615913+0000", 2024-09-13T07:03:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-13T07:03:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514663+0000", 2024-09-13T07:03:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89874952500000005 2024-09-13T07:03:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514669+0000", 2024-09-13T07:03:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-13T07:03:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129545 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.424944153999999, 2024-09-13T07:03:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89876792599999999, 2024-09-13T07:03:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129545 2024-09-13T07:03:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615927+0000", 2024-09-13T07:03:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615931+0000", 2024-09-13T07:03:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8010000000000003e-06 2024-09-13T07:03:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615939+0000", 2024-09-13T07:03:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6310000000000003e-06 2024-09-13T07:03:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615947+0000", 2024-09-13T07:03:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7090000000000002e-06 2024-09-13T07:03:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615985+0000", 2024-09-13T07:03:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8389000000000001e-05 2024-09-13T07:03:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615993+0000", 2024-09-13T07:03:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2209999999999997e-06 2024-09-13T07:03:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514689+0000", 2024-09-13T07:03:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89869535199999995 2024-09-13T07:03:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514696+0000", 2024-09-13T07:03:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8179999999999998e-06 2024-09-13T07:03:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129546 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.424864066000001, 2024-09-13T07:03:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89872448199999999, 2024-09-13T07:03:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129546 2024-09-13T07:03:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616007+0000", 2024-09-13T07:03:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616011+0000", 2024-09-13T07:03:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1659999999999996e-06 2024-09-13T07:03:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616018+0000", 2024-09-13T07:03:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0829999999999998e-06 2024-09-13T07:03:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616024+0000", 2024-09-13T07:03:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8170000000000004e-06 2024-09-13T07:03:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616063+0000", 2024-09-13T07:03:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8708000000000003e-05 2024-09-13T07:03:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616070+0000", 2024-09-13T07:03:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6299999999999998e-06 2024-09-13T07:03:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514721+0000", 2024-09-13T07:03:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89865028800000002 2024-09-13T07:03:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514732+0000", 2024-09-13T07:03:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1452e-05 2024-09-13T07:03:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129547 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.424788738, 2024-09-13T07:03:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89868529799999997, 2024-09-13T07:03:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.138 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129547 2024-09-13T07:03:18.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.138 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:18.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:18.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:18.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616086+0000", 2024-09-13T07:03:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.394e-06 2024-09-13T07:03:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616093+0000", 2024-09-13T07:03:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1130000000000003e-06 2024-09-13T07:03:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616100+0000", 2024-09-13T07:03:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1520000000000003e-06 2024-09-13T07:03:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616138+0000", 2024-09-13T07:03:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8084000000000003e-05 2024-09-13T07:03:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616146+0000", 2024-09-13T07:03:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3010000000000001e-06 2024-09-13T07:03:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89861259299999996 2024-09-13T07:03:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514768+0000", 2024-09-13T07:03:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0175e-05 2024-09-13T07:03:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129548 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.424704679000001, 2024-09-13T07:03:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89862847899999998, 2024-09-13T07:03:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129548 2024-09-13T07:03:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616166+0000", 2024-09-13T07:03:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616170+0000", 2024-09-13T07:03:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8619999999999996e-06 2024-09-13T07:03:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616177+0000", 2024-09-13T07:03:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0450000000000003e-06 2024-09-13T07:03:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616183+0000", 2024-09-13T07:03:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.139e-06 2024-09-13T07:03:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616221+0000", 2024-09-13T07:03:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7320000000000002e-05 2024-09-13T07:03:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616228+0000", 2024-09-13T07:03:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2980000000000003e-06 2024-09-13T07:03:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514789+0000", 2024-09-13T07:03:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89856044599999996 2024-09-13T07:03:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514796+0000", 2024-09-13T07:03:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0809999999999996e-06 2024-09-13T07:03:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129549 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.424631216000002, 2024-09-13T07:03:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89859381900000002, 2024-09-13T07:03:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129549 2024-09-13T07:03:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616240+0000", 2024-09-13T07:03:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616244+0000", 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4149999999999999e-06 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616251+0000", 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616256+0000", 2024-09-13T07:03:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3299999999999998e-06 2024-09-13T07:03:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616302+0000", 2024-09-13T07:03:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5646999999999999e-05 2024-09-13T07:03:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616309+0000", 2024-09-13T07:03:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5530000000000004e-06 2024-09-13T07:03:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514825+0000", 2024-09-13T07:03:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89851528400000003 2024-09-13T07:03:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514834+0000", 2024-09-13T07:03:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8349999999999999e-06 2024-09-13T07:03:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129550 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.424546837000001, 2024-09-13T07:03:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89854296199999995, 2024-09-13T07:03:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129550 2024-09-13T07:03:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616324+0000", 2024-09-13T07:03:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616328+0000", 2024-09-13T07:03:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7479999999999999e-06 2024-09-13T07:03:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616335+0000", 2024-09-13T07:03:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9280000000000004e-06 2024-09-13T07:03:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616342+0000", 2024-09-13T07:03:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2690000000000002e-06 2024-09-13T07:03:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616380+0000", 2024-09-13T07:03:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:03:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616387+0000", 2024-09-13T07:03:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.075e-06 2024-09-13T07:03:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514861+0000", 2024-09-13T07:03:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89847450799999995 2024-09-13T07:03:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514868+0000", 2024-09-13T07:03:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6050000000000003e-06 2024-09-13T07:03:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.159 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:18.162 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:18.162 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:18.158+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:18.162 INFO:teuthology.orchestra.run.smithi118.stdout:5101 write 0x641926a thru 0x642457a (0xb311 bytes) 2024-09-13T07:03:18.162 INFO:teuthology.orchestra.run.smithi118.stdout:5102 punch from 0x4ab2a43 to 0x4ab9a21, (0x6fde bytes) 2024-09-13T07:03:18.163 INFO:teuthology.orchestra.run.smithi118.stdout:5104 punch from 0x399a052 to 0x39a1d62, (0x7d10 bytes) 2024-09-13T07:03:18.163 INFO:teuthology.orchestra.run.smithi118.stdout:5108 write 0x6bf445f thru 0x6bf8a3e (0x45e0 bytes) 2024-09-13T07:03:18.163 INFO:teuthology.orchestra.run.smithi118.stdout:5109 write 0x65f0135 thru 0x65f213f (0x200b bytes) 2024-09-13T07:03:18.163 INFO:teuthology.orchestra.run.smithi118.stdout:5110 write 0x9477d7c thru 0x947e305 (0x658a bytes) 2024-09-13T07:03:18.163 INFO:teuthology.orchestra.run.smithi118.stdout:5112 writesame 0x921192b thru 0x9217d0a data_size 0x11(0x63e0 bytes) 2024-09-13T07:03:18.163 INFO:teuthology.orchestra.run.smithi118.stdout:5113 write 0x41bbb13 thru 0x41bbb3d (0x2b bytes) 2024-09-13T07:03:18.167 INFO:teuthology.orchestra.run.smithi118.stdout:5115 write 0xb5692fa thru 0xb574800 (0xb507 bytes) 2024-09-13T07:03:18.194 INFO:teuthology.orchestra.run.smithi118.stdout:5116 punch from 0x6870eb8 to 0x687aa19, (0x9b61 bytes) 2024-09-13T07:03:18.216 INFO:teuthology.orchestra.run.smithi118.stdout:5119 trunc from 0xb574801 to 0x42d334 2024-09-13T07:03:18.261 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:18.259+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:18.286 INFO:teuthology.orchestra.run.smithi118.stdout:5120 write 0xe51b1fe thru 0xe524010 (0x8e13 bytes) 2024-09-13T07:03:18.324 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:18.324 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:18.324 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:18.327 INFO:teuthology.orchestra.run.smithi118.stdout:5121 writesame 0x7e6efd1 thru 0x7e7063e data_size 0x1d(0x166e bytes) 2024-09-13T07:03:18.334 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:18.350 INFO:teuthology.orchestra.run.smithi118.stdout:5122 trunc from 0xe524011 to 0x92a9c70 2024-09-13T07:03:18.361 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:18.358+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:18.373 INFO:teuthology.orchestra.run.smithi118.stdout:5123 punch from 0x8f3fc25 to 0x8f42c0b, (0x2fe6 bytes) 2024-09-13T07:03:18.384 INFO:teuthology.orchestra.run.smithi118.stdout:5124 trunc from 0x92a9c70 to 0x4513bc4 2024-09-13T07:03:18.405 INFO:teuthology.orchestra.run.smithi118.stdout:5125 read 0x138c23d thru 0x1395290 (0x9054 bytes) 2024-09-13T07:03:18.406 INFO:teuthology.orchestra.run.smithi118.stdout:5126 write 0x14e729a thru 0x14ee389 (0x70f0 bytes) 2024-09-13T07:03:18.444 INFO:teuthology.orchestra.run.smithi118.stdout:5127 write 0x87ce4e3 thru 0x87dcf56 (0xea74 bytes) 2024-09-13T07:03:18.447 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:18.447 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:18.447 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:18.448 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:18.448 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:18.459 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:18.462 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:18.459+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:18.467 INFO:teuthology.orchestra.run.smithi118.stdout:5128 punch from 0x473645b to 0x47428b4, (0xc459 bytes) 2024-09-13T07:03:18.467 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:18.467 INFO:teuthology.orchestra.run.smithi118.stdout:5130 read 0xc3e62e thru 0xc46d7d (0x8750 bytes) 2024-09-13T07:03:18.475 INFO:teuthology.orchestra.run.smithi118.stdout:5132 trunc from 0x87dcf57 to 0x4451e5a 2024-09-13T07:03:18.506 INFO:teuthology.orchestra.run.smithi118.stdout:5133 trunc from 0x4451e5a to 0x884dab4 2024-09-13T07:03:18.519 INFO:teuthology.orchestra.run.smithi118.stdout:5134 punch from 0xd65f42 to 0xd726e8, (0xc7a6 bytes) 2024-09-13T07:03:18.519 INFO:teuthology.orchestra.run.smithi118.stdout:5135 punch from 0x827191e to 0x82793ed, (0x7acf bytes) 2024-09-13T07:03:18.519 INFO:teuthology.orchestra.run.smithi118.stdout:5137 read 0x186724d thru 0x18696df (0x2493 bytes) 2024-09-13T07:03:18.520 INFO:teuthology.orchestra.run.smithi118.stdout:5138 read 0x75b9aac thru 0x75c748a (0xd9df bytes) 2024-09-13T07:03:18.538 INFO:teuthology.orchestra.run.smithi118.stdout:5141 writesame 0xa31130d thru 0xa31405f data_size 0x1(0x2d53 bytes) 2024-09-13T07:03:18.538 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:18.561 INFO:teuthology.orchestra.run.smithi118.stdout:5144 write 0xdefc3b8 thru 0xdefd798 (0x13e1 bytes) 2024-09-13T07:03:18.562 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:18.559+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:18.569 INFO:teuthology.orchestra.run.smithi118.stdout:5145 writesame 0x102eb20 thru 0x103d992 data_size 0x1(0xee73 bytes) 2024-09-13T07:03:18.599 INFO:teuthology.orchestra.run.smithi118.stdout:5148 writesame 0xac860a0 thru 0xac8737f data_size 0x10(0x12e0 bytes) 2024-09-13T07:03:18.604 INFO:teuthology.orchestra.run.smithi118.stdout:5149 read 0x39d9ed4 thru 0x39deb9f (0x4ccc bytes) 2024-09-13T07:03:18.604 INFO:teuthology.orchestra.run.smithi118.stdout:5150 read 0x1a8f16b thru 0x1a8f7c1 (0x657 bytes) 2024-09-13T07:03:18.604 INFO:teuthology.orchestra.run.smithi118.stdout:5151 write 0x5021db3 thru 0x50309a4 (0xebf2 bytes) 2024-09-13T07:03:18.622 INFO:teuthology.orchestra.run.smithi118.stdout:5153 write 0x2291ec7 thru 0x22a00b9 (0xe1f3 bytes) 2024-09-13T07:03:18.633 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.57432300400001, 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:03:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:03:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:03:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:03:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:03:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:03:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93979 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016563691000002, 2024-09-13T07:03:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042253801000000001, 2024-09-13T07:03:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93979 2024-09-13T07:03:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616683+0000", 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2820000000000001e-06 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616691+0000", 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0779999999999996e-06 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616698+0000", 2024-09-13T07:03:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:03:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658799+0000", 2024-09-13T07:03:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042100763999999999 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658869+0000", 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0065000000000001e-05 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658934+0000", 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5071000000000001e-05 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016539125999998, 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:03:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:03:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016516158999998, 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:03:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016498980999998, 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016485115999998, 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:03:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:03:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016471971000001, 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:03:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:03:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016458763999999, 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:03:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016444696999997, 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:03:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016424682999997, 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:03:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:03:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:03:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:03:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:03:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016404932999997, 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:03:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:03:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:03:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:03:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016380126999998, 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:03:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:03:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016355861999998, 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:03:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016314129000001, 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:03:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:03:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:03:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:03:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:03:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:03:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:03:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:03:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.016277840000001, 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:03:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 36.573174508000001, 2024-09-13T07:03:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:03:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:03:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:03:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:03:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:03:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 36.573157625999997, 2024-09-13T07:03:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:03:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:03:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:03:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:03:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 36.572107068999998, 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:03:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:03:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 36.570432519999997, 2024-09-13T07:03:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:03:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:03:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:03:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:03:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:03:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:03:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.049850129999999, 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:03:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:03:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:03:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:18.722 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:18.724 INFO:teuthology.orchestra.run.smithi118.stdout:5154 trunc from 0xdefd799 to 0x1d23b6b 2024-09-13T07:03:18.724 INFO:teuthology.orchestra.run.smithi118.stdout:5155 punch from 0xd5c4ef to 0xd5e961, (0x2472 bytes) 2024-09-13T07:03:18.724 INFO:teuthology.orchestra.run.smithi118.stdout:5156 writesame 0x2e6d265 thru 0x2e7ceb6 data_size 0x1(0xfc52 bytes) 2024-09-13T07:03:18.724 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:18.724 INFO:teuthology.orchestra.run.smithi118.stdout:5159 write 0x3a89d6 thru 0x3b7c3a (0xf265 bytes) 2024-09-13T07:03:18.724 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:18.724 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:18.725 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:18.722+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:18.725 INFO:teuthology.orchestra.run.smithi118.stdout:5161 trunc from 0x2e7ceb7 to 0x4cc5d7d 2024-09-13T07:03:18.735 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:18.755 INFO:teuthology.orchestra.run.smithi118.stdout:5164 writesame 0x8167439 thru 0x8167a2c data_size 0x6(0x5f4 bytes) 2024-09-13T07:03:18.781 INFO:teuthology.orchestra.run.smithi118.stdout:5165 write 0xe427532 thru 0xe42a5c2 (0x3091 bytes) 2024-09-13T07:03:18.796 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:18.796 INFO:teuthology.orchestra.run.smithi118.stdout:5167 write 0x185c5cd thru 0x1867d38 (0xb76c bytes) 2024-09-13T07:03:18.809 INFO:teuthology.orchestra.run.smithi118.stdout:5169 writesame 0xd1ee36d thru 0xd1eebea data_size 0x1(0x87e bytes) 2024-09-13T07:03:18.815 INFO:teuthology.orchestra.run.smithi118.stdout:5170 read 0x1268ba3 thru 0x126eb95 (0x5ff3 bytes) 2024-09-13T07:03:18.815 INFO:teuthology.orchestra.run.smithi118.stdout:5171 writesame 0xda10f95 thru 0xda1396e data_size 0xb(0x29da bytes) 2024-09-13T07:03:18.818 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 7, 2, 0] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:03:18.818 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:03:18.818 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2024-09-13T07:03:18.818 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .001 2024-09-13T07:03:18.841 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:18.841 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:18.841 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:18.841 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:18.846 INFO:teuthology.orchestra.run.smithi118.stdout:5172 trunc from 0xe42a5c3 to 0x33a1a7e 2024-09-13T07:03:18.847 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:18.843+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:18.851 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:18.891 INFO:teuthology.orchestra.run.smithi118.stdout:5173 write 0xba8cfa4 thru 0xba9626f (0x92cc bytes) 2024-09-13T07:03:18.916 INFO:teuthology.orchestra.run.smithi118.stdout:5175 read 0x930c95 thru 0x93eeb6 (0xe222 bytes) 2024-09-13T07:03:18.916 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:18.916 INFO:teuthology.orchestra.run.smithi118.stdout:5177 writesame 0x499382 thru 0x4a111e data_size 0x1b(0x7d9d bytes) 2024-09-13T07:03:18.922 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:18.922 INFO:teuthology.orchestra.run.smithi118.stdout:5179 writesame 0x560d578 thru 0x561c8f8 data_size 0xb(0xf381 bytes) 2024-09-13T07:03:18.924 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:18.919+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:18.943 INFO:teuthology.orchestra.run.smithi118.stdout:5180 write 0xac4008c thru 0xac49af4 (0x9a69 bytes) 2024-09-13T07:03:18.961 INFO:teuthology.orchestra.run.smithi118.stdout:5182 read 0x8471df3 thru 0x847ee71 (0xd07f bytes) 2024-09-13T07:03:18.961 INFO:teuthology.orchestra.run.smithi118.stdout:5183 writesame 0x3ba4d81 thru 0x3baad78 data_size 0x53(0x5ff8 bytes) 2024-09-13T07:03:18.971 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:18.972 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:18.973 INFO:teuthology.orchestra.run.smithi118.stdout:5184 punch from 0x306310d to 0x306366b, (0x55e bytes) 2024-09-13T07:03:18.973 INFO:teuthology.orchestra.run.smithi118.stdout:5185 write 0x1b5b535 thru 0x1b68afa (0xd5c6 bytes) 2024-09-13T07:03:18.983 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:18.996 INFO:teuthology.orchestra.run.smithi118.stdout:5186 read 0x5ba198d thru 0x5bae1d7 (0xc84b bytes) 2024-09-13T07:03:19.006 INFO:teuthology.orchestra.run.smithi118.stdout:5187 writesame 0xc024ea9 thru 0xc030c23 data_size 0x3(0xbd7b bytes) 2024-09-13T07:03:19.006 INFO:teuthology.orchestra.run.smithi118.stdout:5189 read 0x598c15a thru 0x598de39 (0x1ce0 bytes) 2024-09-13T07:03:19.018 INFO:teuthology.orchestra.run.smithi118.stdout:5194 write 0xce4e0c4 thru 0xce56077 (0x7fb4 bytes) 2024-09-13T07:03:19.025 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:19.022+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:19.054 INFO:teuthology.orchestra.run.smithi118.stdout:5196 trunc from 0xce56078 to 0x35820d 2024-09-13T07:03:19.103 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:19.104 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:19.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:19.104 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:19.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.104 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:03:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 113.048802624, 2024-09-13T07:03:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:03:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:03:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:03:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:03:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:03:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.890921390000003, 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:03:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:03:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:03:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:03:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.865151678999993, 2024-09-13T07:03:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:03:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:03:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:03:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:03:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:03:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:03:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:03:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:03:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:03:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:03:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:03:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:03:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:03:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.844000512999997, 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:03:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:03:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:03:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.726955750000002, 2024-09-13T07:03:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:03:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:03:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:03:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:03:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:03:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:03:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:03:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:03:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.441958505000002, 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:03:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:03:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:03:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:03:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:03:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:03:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94598 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.441787783999999, 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044365860999999999, 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94598 2024-09-13T07:03:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661187+0000", 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661034+0000", 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9998465 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661192+0000", 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015813000000000001 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661199+0000", 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5689999999999999e-06 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705452+0000", 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044246273000000003 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705501+0000", 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8748000000000002e-05 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705553+0000", 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2241999999999999e-05 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107060 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1003520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.190265494999998, 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044341754999999997, 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107060 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912710+0000", 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912709+0000", 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912712+0000", 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6000000000000001e-06 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912717+0000", 2024-09-13T07:03:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3689999999999998e-06 2024-09-13T07:03:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912722+0000", 2024-09-13T07:03:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7219999999999999e-06 2024-09-13T07:03:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956876+0000", 2024-09-13T07:03:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044154469000000002 2024-09-13T07:03:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.956957+0000", 2024-09-13T07:03:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1231999999999997e-05 2024-09-13T07:03:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957051+0000", 2024-09-13T07:03:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3858999999999997e-05 2024-09-13T07:03:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.190249367, 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:03:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:03:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:03:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:03:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:03:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.190231333, 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.190215252999998, 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:03:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.190199577000001, 2024-09-13T07:03:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:03:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:03:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:03:19.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.190176655000002, 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:03:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:03:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.190160558000002, 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:03:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107067 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.189600540999997, 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04441055, 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107067 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912832+0000", 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9994574 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913379+0000", 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00054719999999999997 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913386+0000", 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4549999999999998e-06 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913393+0000", 2024-09-13T07:03:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6259999999999997e-06 2024-09-13T07:03:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957776+0000", 2024-09-13T07:03:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044383143999999999 2024-09-13T07:03:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957785+0000", 2024-09-13T07:03:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9220000000000003e-06 2024-09-13T07:03:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107068 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.189576109999997, 2024-09-13T07:03:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044410502999999997, 2024-09-13T07:03:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107068 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913401+0000", 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913407+0000", 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7520000000000002e-06 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913412+0000", 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8550000000000001e-06 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957802+0000", 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044389606999999998 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957809+0000", 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9160000000000001e-06 2024-09-13T07:03:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.189558945999998, 2024-09-13T07:03:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:03:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:03:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:03:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:03:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.189516169999997, 2024-09-13T07:03:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:03:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:03:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:03:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:03:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.032498400000001, 2024-09-13T07:03:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:03:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:03:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:03:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:03:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.1801214699999996, 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:03:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:03:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:03:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:03:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:03:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:03:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:03:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:19.197 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:19.199 INFO:teuthology.orchestra.run.smithi118.stdout:5197 write 0xca92e5d thru 0xca98f5e (0x6102 bytes) 2024-09-13T07:03:19.199 INFO:teuthology.orchestra.run.smithi118.stdout:5198 punch from 0x1073685 to 0x1075239, (0x1bb4 bytes) 2024-09-13T07:03:19.199 INFO:teuthology.orchestra.run.smithi118.stdout:5199 read 0x72a5100 thru 0x72b3856 (0xe757 bytes) 2024-09-13T07:03:19.199 INFO:teuthology.orchestra.run.smithi118.stdout:5204 trunc from 0xca98f5f to 0xc819164 2024-09-13T07:03:19.199 INFO:teuthology.orchestra.run.smithi118.stdout:5205 read 0x2703379 thru 0x2705b0c (0x2794 bytes) 2024-09-13T07:03:19.199 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:19.199 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:19.199 INFO:teuthology.orchestra.run.smithi118.stdout:5208 write 0xb6e4cc9 thru 0xb6ee619 (0x9951 bytes) 2024-09-13T07:03:19.199 INFO:teuthology.orchestra.run.smithi118.stdout:5209 writesame 0x715bd3 thru 0x724ce5 data_size 0x1(0xf113 bytes) 2024-09-13T07:03:19.200 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:19.200 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:19.195+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:19.219 INFO:teuthology.orchestra.run.smithi118.stdout:5210 read 0x931122a thru 0x9317e94 (0x6c6b bytes) 2024-09-13T07:03:19.219 INFO:teuthology.orchestra.run.smithi118.stdout:5211 write 0x9c9375 thru 0x9d1a86 (0x8712 bytes) 2024-09-13T07:03:19.223 INFO:teuthology.orchestra.run.smithi118.stdout:5212 clone 42 order 23 su 4194304 sc 13 2024-09-13T07:03:19.299 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:19.298+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:19.305 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:19.306 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:19.306 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:19.306 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:19.315 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:19.400 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:19.395+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:19.468 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:19.468 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:19.468 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:19.468 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:19.469 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:19.478 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:19.501 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:19.499+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:19.591 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:19.592 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:19.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:19.592 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:19.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.592 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:19.592 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:03:19.592 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.084901497, 2024-09-13T07:03:19.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:03:19.592 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.592 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.592 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.592 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.593 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.593 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:03:19.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.593 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:03:19.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:03:19.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:03:19.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:19.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:03:19.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:03:19.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:03:19.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:03:19.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:03:19.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:03:19.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:03:19.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:03:19.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:03:19.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:03:19.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:19.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:03:19.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:03:19.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:03:19.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:03:19.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:03:19.601 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.602 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.602 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.602 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:19.602 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:03:19.602 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.06007378699999, 2024-09-13T07:03:19.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:03:19.602 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.602 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.602 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.602 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.602 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.603 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:03:19.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.603 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:03:19.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:03:19.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:03:19.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:19.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:03:19.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:03:19.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:03:19.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:03:19.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:03:19.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:03:19.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 113.521401963, 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.608 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:03:19.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:03:19.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:03:19.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:03:19.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:03:19.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:03:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:03:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:03:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:03:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:03:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:03:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:03:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:03:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 113.49177677, 2024-09-13T07:03:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:03:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:03:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:03:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:03:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:03:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:03:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:03:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:03:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:03:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:03:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:03:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:03:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:03:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:03:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:03:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.95906313899999, 2024-09-13T07:03:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:03:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:03:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:03:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:03:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:03:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:03:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:03:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:03:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:03:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:03:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:03:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:03:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:03:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:03:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:03:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:03:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:03:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.95898068299999, 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:03:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:03:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:03:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:03:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:03:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:03:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:03:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:03:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:03:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:03:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:03:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:03:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:03:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:03:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:03:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:03:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:03:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:03:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:03:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:03:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:03:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:03:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:03:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:03:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.958948873, 2024-09-13T07:03:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:03:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:03:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:03:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:03:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:03:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:03:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:03:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:03:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:03:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:03:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:03:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:03:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:03:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:03:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:03:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:03:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:03:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:03:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:03:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:03:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:03:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:03:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:03:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.958893715, 2024-09-13T07:03:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:03:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:03:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:03:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:03:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:03:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:03:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:03:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:03:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:03:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:03:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:03:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:03:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:03:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:03:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:03:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:03:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:03:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:03:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:03:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:03:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:03:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:03:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.958848436, 2024-09-13T07:03:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:03:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:03:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:03:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:03:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:03:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:03:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:03:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:03:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:03:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:03:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:03:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:03:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:03:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:03:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:03:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:03:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:03:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:03:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:03:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:03:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.958809902, 2024-09-13T07:03:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:03:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:03:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:03:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:03:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:03:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:03:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:03:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:03:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:03:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:03:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:03:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:03:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:03:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:03:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:03:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:03:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:03:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:03:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:03:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:03:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:03:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:03:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:03:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:03:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:03:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.95874828300001, 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:03:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:03:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:03:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:03:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.94452560000001, 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:03:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:03:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:03:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:03:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:03:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:03:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:03:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:03:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:03:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:03:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:03:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:03:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 108.932067864, 2024-09-13T07:03:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:03:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:03:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:03:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:03:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:03:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:03:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:03:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:03:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:03:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:03:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 108.932048147, 2024-09-13T07:03:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:03:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:03:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:03:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:03:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:03:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:03:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:03:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 108.932030817, 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:03:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:03:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:03:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:03:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:03:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:03:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:03:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:03:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:03:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:03:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:03:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 108.931735155, 2024-09-13T07:03:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:03:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:03:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:03:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:03:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:03:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:03:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:03:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:03:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:03:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:03:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.370532335999997, 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:03:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:03:19.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:03:19.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:03:19.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:03:19.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:03:19.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:03:19.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:03:19.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:03:19.693 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.693 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.693 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.693 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:03:19.693 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:03:19.694 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.338453432999998, 2024-09-13T07:03:19.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:03:19.694 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.694 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:19.694 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:03:19.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:03:19.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:03:19.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:03:19.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:03:19.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:03:19.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:03:19.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.697 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137474 3.6 3:65c31370:::rbd_data.1194af11889b.000000000000013b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 186185~11715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.845776224, 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0054357140000001, 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137474 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743934+0000", 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743949+0000", 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5705999999999998e-05 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743960+0000", 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0084e-05 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743971+0000", 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0942e-05 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744045+0000", 2024-09-13T07:03:19.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4626e-05 2024-09-13T07:03:19.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:19.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744062+0000", 2024-09-13T07:03:19.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7339000000000001e-05 2024-09-13T07:03:19.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.701 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.701 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746366+0000", 2024-09-13T07:03:19.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.002303457 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746520+0000", 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015385900000000001 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746734+0000", 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000214253 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749128+0000", 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002394129 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749267+0000", 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013878299999999999 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749328+0000", 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0918000000000002e-05 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749371+0000", 2024-09-13T07:03:19.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3188000000000001e-05 2024-09-13T07:03:19.704 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.704 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.704 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.704 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138463 3.17 3:efe7ec01:::rbd_object_map.11949f62878b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:19.704 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.429734+0000", 2024-09-13T07:03:19.704 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 11.159977616999999, 2024-09-13T07:03:19.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092770100000000005, 2024-09-13T07:03:19.704 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:19.704 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:19.704 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:19.705 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:19.705 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:19.705 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138463 2024-09-13T07:03:19.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.705 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:19.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:19.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429734+0000", 2024-09-13T07:03:19.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:19.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429734+0000", 2024-09-13T07:03:19.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:19.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:19.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429734+0000", 2024-09-13T07:03:19.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:19.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.706 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.706 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:19.706 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429736+0000", 2024-09-13T07:03:19.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.621e-06 2024-09-13T07:03:19.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:19.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429740+0000", 2024-09-13T07:03:19.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0350000000000003e-06 2024-09-13T07:03:19.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:19.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429744+0000", 2024-09-13T07:03:19.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3909999999999998e-06 2024-09-13T07:03:19.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:19.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.432092+0000", 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023486330000000001 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.432189+0000", 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6279000000000004e-05 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.432610+0000", 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042100200000000002 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:19.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.435051+0000", 2024-09-13T07:03:19.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024418500000000002 2024-09-13T07:03:19.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:19.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.438887+0000", 2024-09-13T07:03:19.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0038353900000000002 2024-09-13T07:03:19.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:19.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.438959+0000", 2024-09-13T07:03:19.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1880999999999998e-05 2024-09-13T07:03:19.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:19.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:19.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:19.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.439011+0000", 2024-09-13T07:03:19.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2268999999999997e-05 2024-09-13T07:03:19.710 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.710 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.710 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.710 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:19.710 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:19.710 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:19.712 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:19.712 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:19.710+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:19.813 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:19.811+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:19.834 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:03:19.850 INFO:tasks.ceph.mon.a.smithi118.stderr:2024-09-13T07:03:19.847+0000 7fd1eeb1e700 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-13T07:03:19.867 INFO:teuthology.orchestra.run.smithi118.stderr:osd set-full-ratio 0.001 2024-09-13T07:03:19.870 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:19.871 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:19.871 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:19.871 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:19.883 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:19.913 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:19.910+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:20.015 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:03:20.016 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:20.011+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:20.057 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:20.057 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:20.058 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:20.058 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:20.058 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:20.073 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:20.117 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:20.115+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:20.156 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-13T07:03:20.174 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:20.196 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:20.200 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:20.200 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:20.210 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:20.217 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:20.215+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:20.221 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:20.254 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:20.278 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:20.302 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:20.318 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:20.315+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:20.326 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:20.326 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:20.326 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:20.336 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:20.350 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:20.352 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:20.352 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:20.352 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:20.352 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:20.353 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:20.353 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:20.353 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:20.353 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:20.420 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:20.418+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:20.450 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:20.451 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:20.461 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:20.520 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:20.518+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:20.552 INFO:teuthology.orchestra.run.smithi118.stdout:5213 writesame 0xcd0bf6b thru 0xcd16847 data_size 0x1(0xa8dd bytes) 2024-09-13T07:03:20.559 INFO:teuthology.orchestra.run.smithi118.stdout:5215 write 0x7e74106 thru 0x7e7b210 (0x710b bytes) 2024-09-13T07:03:20.568 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:20.568 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:20.579 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:20.582 INFO:teuthology.orchestra.run.smithi118.stdout:5220 write 0xe26c71b thru 0xe272610 (0x5ef6 bytes) 2024-09-13T07:03:20.590 INFO:teuthology.orchestra.run.smithi118.stdout:5221 write 0xc9dc70 thru 0xca06cf (0x2a60 bytes) 2024-09-13T07:03:20.600 INFO:teuthology.orchestra.run.smithi118.stdout:5222 write 0x7b5cfa8 thru 0x7b637ad (0x6806 bytes) 2024-09-13T07:03:20.608 INFO:teuthology.orchestra.run.smithi118.stdout:5223 read 0xd6d2857 thru 0xd6df077 (0xc821 bytes) 2024-09-13T07:03:20.608 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:20.619 INFO:teuthology.orchestra.run.smithi118.stdout:5225 trunc from 0xe272611 to 0x8f5d206 2024-09-13T07:03:20.620 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:20.618+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:20.666 INFO:teuthology.orchestra.run.smithi118.stdout:5227 write 0x4b4f583 thru 0x4b552a7 (0x5d25 bytes) 2024-09-13T07:03:20.673 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:20.673 INFO:teuthology.orchestra.run.smithi118.stdout:5229 punch from 0x70eb93d to 0x70ed342, (0x1a05 bytes) 2024-09-13T07:03:20.685 INFO:teuthology.orchestra.run.smithi118.stdout:5230 writesame 0x9a4db76 thru 0x9a566f1 data_size 0x4(0x8b7c bytes) 2024-09-13T07:03:20.686 INFO:teuthology.orchestra.run.smithi118.stdout:5231 read 0x38409ae thru 0x38453d4 (0x4a27 bytes) 2024-09-13T07:03:20.686 INFO:teuthology.orchestra.run.smithi118.stdout:5236 clone 43 order 24 su 4194304 sc 15 2024-09-13T07:03:20.692 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:20.693 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:20.703 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:20.721 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:20.718+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:20.807 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:20.807 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:20.816 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:20.822 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:20.822+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:20.874 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:03:20.880 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2024-09-13T07:03:20.880 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .95 2024-09-13T07:03:20.929 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:20.927+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:20.930 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:20.931 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:20.941 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:20.990 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone42 from 0x9a566f2 (overlap 0x8f5d206) to 0x0 2024-09-13T07:03:21.024 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:21.019+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:21.070 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:21.070 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:21.080 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:21.124 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-13T07:03:21.125 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:21.123+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:21.180 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:21.180 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:21.190 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:21.227 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:21.226+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:21.301 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:21.302 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:21.311 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:03:21.327 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:21.323+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:21.415 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:21.415 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:21.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:03:21.427 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:21.423+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:21.503 INFO:teuthology.orchestra.run.smithi118.stdout:5237 read 0x7fe9575 thru 0x7ff75fa (0xe086 bytes) 2024-09-13T07:03:21.506 INFO:teuthology.orchestra.run.smithi118.stdout:5238 punch from 0x732d4e6 to 0x73351f9, (0x7d13 bytes) 2024-09-13T07:03:21.528 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:21.526+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:21.531 INFO:teuthology.orchestra.run.smithi118.stdout:5239 write 0xca77c22 thru 0xca793d1 (0x17b0 bytes) 2024-09-13T07:03:21.534 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:21.534 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:21.545 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:03:21.557 INFO:teuthology.orchestra.run.smithi118.stdout:5240 trunc from 0xca793d2 to 0x1a2bd90 2024-09-13T07:03:21.593 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:21.603 INFO:teuthology.orchestra.run.smithi118.stdout:5243 trunc from 0x1a2bd90 to 0x7038fbd 2024-09-13T07:03:21.613 INFO:teuthology.orchestra.run.smithi118.stdout:5244 punch from 0x2cfdfbd to 0x2d05d67, (0x7daa bytes) 2024-09-13T07:03:21.614 INFO:teuthology.orchestra.run.smithi118.stdout:5245 read 0x221b1db thru 0x222536b (0xa191 bytes) 2024-09-13T07:03:21.626 INFO:teuthology.orchestra.run.smithi118.stdout:5246 trunc from 0x7038fbd to 0xe4096ed 2024-09-13T07:03:21.629 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:21.626+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:21.636 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:21.636 INFO:teuthology.orchestra.run.smithi118.stdout:5248 write 0x1df5b71 thru 0x1dfb1bf (0x564f bytes) 2024-09-13T07:03:21.644 INFO:teuthology.orchestra.run.smithi118.stdout:5249 writesame 0x9cd5979 thru 0x9cd69fb data_size 0x1(0x1083 bytes) 2024-09-13T07:03:21.652 INFO:teuthology.orchestra.run.smithi118.stdout:5250 punch from 0x4fb2ac0 to 0x4fc11d0, (0xe710 bytes) 2024-09-13T07:03:21.652 INFO:teuthology.orchestra.run.smithi118.stdout:5252 read 0xcdb69d3 thru 0xcdc03a2 (0x99d0 bytes) 2024-09-13T07:03:21.653 INFO:teuthology.orchestra.run.smithi118.stdout:5254 write 0xac15e8 thru 0xacc16f (0xab88 bytes) 2024-09-13T07:03:21.657 INFO:teuthology.orchestra.run.smithi118.stdout:5255 writesame 0x1e6b0af thru 0x1e6d3c7 data_size 0x5(0x2319 bytes) 2024-09-13T07:03:21.661 INFO:teuthology.orchestra.run.smithi118.stdout:5256 read 0x43ec189 thru 0x43f31d8 (0x7050 bytes) 2024-09-13T07:03:21.661 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:21.661 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:21.661 INFO:teuthology.orchestra.run.smithi118.stdout:5257 writesame 0x3586ffa thru 0x35881fc data_size 0x1d(0x1203 bytes) 2024-09-13T07:03:21.668 INFO:teuthology.orchestra.run.smithi118.stdout:5258 read 0x3c0eebf thru 0x3c1beb2 (0xcff4 bytes) 2024-09-13T07:03:21.669 INFO:teuthology.orchestra.run.smithi118.stdout:5259 write 0x62978c8 thru 0x62994a1 (0x1bda bytes) 2024-09-13T07:03:21.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:21.672 INFO:teuthology.orchestra.run.smithi118.stdout:5261 read 0x7f45345 thru 0x7f492fe (0x3fba bytes) 2024-09-13T07:03:21.673 INFO:teuthology.orchestra.run.smithi118.stdout:5263 writesame 0x816708b thru 0x816c26d data_size 0x1(0x51e3 bytes) 2024-09-13T07:03:21.695 INFO:teuthology.orchestra.run.smithi118.stdout:5264 punch from 0x5928a57 to 0x593429a, (0xb843 bytes) 2024-09-13T07:03:21.695 INFO:teuthology.orchestra.run.smithi118.stdout:5267 writesame 0xd14f6bc thru 0xd15e0b1 data_size 0x1(0xe9f6 bytes) 2024-09-13T07:03:21.730 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:21.727+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:21.749 INFO:teuthology.orchestra.run.smithi118.stdout:5268 read 0xc30ae4d thru 0xc30c60d (0x17c1 bytes) 2024-09-13T07:03:21.750 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:21.750 INFO:teuthology.orchestra.run.smithi118.stdout:5270 punch from 0x36b0020 to 0x36bc88a, (0xc86a bytes) 2024-09-13T07:03:21.750 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:21.750 INFO:teuthology.orchestra.run.smithi118.stdout:5273 read 0x8872b2 thru 0x88831d (0x106c bytes) 2024-09-13T07:03:21.751 INFO:teuthology.orchestra.run.smithi118.stdout:5274 write 0x4c52c99 thru 0x4c60d6d (0xe0d5 bytes) 2024-09-13T07:03:21.775 INFO:teuthology.orchestra.run.smithi118.stdout:5275 trunc from 0xe4096ed to 0x4d55efe 2024-09-13T07:03:21.790 INFO:teuthology.orchestra.run.smithi118.stdout:5276 read 0x41e0847 thru 0x41eb6f2 (0xaeac bytes) 2024-09-13T07:03:21.793 INFO:teuthology.orchestra.run.smithi118.stdout:5280 trunc from 0x4d55efe to 0x64f7eca 2024-09-13T07:03:21.804 INFO:teuthology.orchestra.run.smithi118.stdout:5281 punch from 0x1fa1fb4 to 0x1facac2, (0xab0e bytes) 2024-09-13T07:03:21.826 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:21.827 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:21.830 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:21.826+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:21.831 INFO:teuthology.orchestra.run.smithi118.stdout:5284 write 0xdecfc92 thru 0xded385b (0x3bca bytes) 2024-09-13T07:03:21.837 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:21.838 INFO:teuthology.orchestra.run.smithi118.stdout:5285 write 0x16c23c7 thru 0x16c3dfd (0x1a37 bytes) 2024-09-13T07:03:21.841 INFO:teuthology.orchestra.run.smithi118.stdout:5286 writesame 0x82c48f7 thru 0x82d31b4 data_size 0x1f(0xe8be bytes) 2024-09-13T07:03:21.846 INFO:teuthology.orchestra.run.smithi118.stdout:5287 write 0x256189 thru 0x25e134 (0x7fac bytes) 2024-09-13T07:03:21.849 INFO:teuthology.orchestra.run.smithi118.stdout:5288 read 0x7ecfca1 thru 0x7edbe44 (0xc1a4 bytes) 2024-09-13T07:03:21.850 INFO:teuthology.orchestra.run.smithi118.stdout:5289 writesame 0x958c8d4 thru 0x9591cd3 data_size 0x70(0x5400 bytes) 2024-09-13T07:03:21.854 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:21.861 INFO:teuthology.orchestra.run.smithi118.stdout:5291 trunc from 0xded385c to 0xa90bcd4 2024-09-13T07:03:21.877 INFO:teuthology.orchestra.run.smithi118.stderr:osd set-full-ratio 0.95 2024-09-13T07:03:21.886 INFO:teuthology.orchestra.run.smithi118.stdout:5292 writesame 0x4e47f57 thru 0x4e4aca1 data_size 0x5(0x2d4b bytes) 2024-09-13T07:03:21.890 INFO:teuthology.orchestra.run.smithi118.stdout:5296 read 0x71ce50a thru 0x71dc0e0 (0xdbd7 bytes) 2024-09-13T07:03:21.893 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:21.893 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:21.893 INFO:teuthology.orchestra.run.smithi118.stdout:5303 write 0x59a3fde thru 0x59ae721 (0xa744 bytes) 2024-09-13T07:03:21.895 INFO:teuthology.orchestra.run.smithi118.stdout:5304 write 0x9396277 thru 0x93a0be1 (0xa96b bytes) 2024-09-13T07:03:21.897 INFO:teuthology.orchestra.run.smithi118.stdout:5305 writesame 0xa5fc08b thru 0xa60a7e9 data_size 0x1(0xe75f bytes) 2024-09-13T07:03:21.931 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:21.927+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:21.931 INFO:teuthology.orchestra.run.smithi118.stdout:5306 writesame 0x5c86345 thru 0x5c8f630 data_size 0x2(0x92ec bytes) 2024-09-13T07:03:21.944 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:21.945 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:21.952 INFO:teuthology.orchestra.run.smithi118.stdout:5307 trunc from 0xa90bcd4 to 0x8037be8 2024-09-13T07:03:21.955 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:21.965 INFO:teuthology.orchestra.run.smithi118.stdout:5310 write 0x3fc314d thru 0x3fcb109 (0x7fbd bytes) 2024-09-13T07:03:21.988 INFO:teuthology.orchestra.run.smithi118.stdout:5311 writesame 0xc6e6ca3 thru 0xc6e770a data_size 0x24(0xa68 bytes) 2024-09-13T07:03:22.009 INFO:teuthology.orchestra.run.smithi118.stdout:5312 trunc from 0xc6e770b to 0x1354b9e 2024-09-13T07:03:22.031 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:22.027+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:22.046 INFO:teuthology.orchestra.run.smithi118.stdout:5313 write 0x378fcc thru 0x386c7c (0xdcb1 bytes) 2024-09-13T07:03:22.048 INFO:teuthology.orchestra.run.smithi118.stdout:5314 read 0xc858ee thru 0xc8c81f (0x6f32 bytes) 2024-09-13T07:03:22.050 INFO:teuthology.orchestra.run.smithi118.stdout:5315 write 0x79b986 thru 0x79f247 (0x38c2 bytes) 2024-09-13T07:03:22.056 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:22.056 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:22.066 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:22.081 INFO:teuthology.orchestra.run.smithi118.stdout:5316 write 0xb65e152 thru 0xb66276c (0x461b bytes) 2024-09-13T07:03:22.084 INFO:teuthology.orchestra.run.smithi118.stdout:5317 read 0x4212952 thru 0x421d61d (0xaccc bytes) 2024-09-13T07:03:22.086 INFO:teuthology.orchestra.run.smithi118.stdout:5318 write 0x8cc25da thru 0x8cce120 (0xbb47 bytes) 2024-09-13T07:03:22.092 INFO:teuthology.orchestra.run.smithi118.stdout:5319 trunc from 0xb66276d to 0xb399291 2024-09-13T07:03:22.105 INFO:teuthology.orchestra.run.smithi118.stdout:5321 clone 44 order 22 su 131072 sc 3 2024-09-13T07:03:22.133 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:22.131+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:22.169 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:22.169 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:22.178 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:22.234 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:22.235+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:22.286 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:22.286 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:22.296 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:22.335 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:22.334+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:22.364 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:03:22.404 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:22.404 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:22.413 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:22.436 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:22.431+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:22.524 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:22.536 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:22.531+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:22.539 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:22.539 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:22.539 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:22.539 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:22.546 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:22.549 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:22.564 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:22.583 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:22.606 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:22.629 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:22.638 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:22.634+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:22.653 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:22.678 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:22.678 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:22.678 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:22.678 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:22.678 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:22.688 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:22.689 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:22.690 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:22.690 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:22.690 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:22.690 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:22.690 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:22.690 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:22.691 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:22.691 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:22.739 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:22.738+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:22.798 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:971 3.1 3:82a6caa3:::rbd_data.1194d22ad098.000000000000003d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.293829958, 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091827720000000005, 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.799 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 971 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504810+0000", 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504809+0000", 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504824+0000", 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4397e-05 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504840+0000", 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6382e-05 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504849+0000", 2024-09-13T07:03:22.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5620000000000004e-06 2024-09-13T07:03:22.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504901+0000", 2024-09-13T07:03:22.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2710000000000002e-05 2024-09-13T07:03:22.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.504973+0000", 2024-09-13T07:03:22.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1567000000000001e-05 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.505192+0000", 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000219089 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.506014+0000", 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082227999999999999 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513910+0000", 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078953919999999993 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513958+0000", 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8038000000000003e-05 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.513993+0000", 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5271999999999997e-05 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.804 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1023 3.17 3:ee0121e1:::rbd_data.1194d22ad098.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.26900224800001, 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010059435, 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1023 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.805 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:03:22.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529638+0000", 2024-09-13T07:03:22.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529637+0000", 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529643+0000", 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6309999999999997e-06 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529661+0000", 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7492000000000001e-05 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529670+0000", 2024-09-13T07:03:22.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6339999999999996e-06 2024-09-13T07:03:22.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529731+0000", 2024-09-13T07:03:22.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0319000000000003e-05 2024-09-13T07:03:22.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.529835+0000", 2024-09-13T07:03:22.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104448 2024-09-13T07:03:22.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:22.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530032+0000", 2024-09-13T07:03:22.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00019714900000000001 2024-09-13T07:03:22.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.530961+0000", 2024-09-13T07:03:22.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000929074 2024-09-13T07:03:22.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539601+0000", 2024-09-13T07:03:22.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0086394939999999993 2024-09-13T07:03:22.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539658+0000", 2024-09-13T07:03:22.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6857999999999997e-05 2024-09-13T07:03:22.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:25.539697+0000", 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9464e-05 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1535 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 116.730330424, 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029411100999999999, 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.811 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1535 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068310+0000", 2024-09-13T07:03:22.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068307+0000", 2024-09-13T07:03:22.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:03:22.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068324+0000", 2024-09-13T07:03:22.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7784000000000001e-05 2024-09-13T07:03:22.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068357+0000", 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2441000000000003e-05 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068371+0000", 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4439e-05 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068448+0000", 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6684e-05 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.068577+0000", 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000128756 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.069070+0000", 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00049383900000000004 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075109+0000", 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0060388050000000004 2024-09-13T07:03:22.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097568+0000", 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.022458500999999999 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097659+0000", 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1656999999999993e-05 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097721+0000", 2024-09-13T07:03:22.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1289000000000002e-05 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1536 3.17 3:e98072c3:::rbd_data.11943a018f34.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 14680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 116.700705231, 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094483299999999996, 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.817 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1536 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097935+0000", 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097933+0000", 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097942+0000", 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3340000000000002e-06 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097957+0000", 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4837999999999999e-05 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.097966+0000", 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4499999999999993e-06 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098057+0000", 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0635000000000002e-05 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098288+0000", 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00023125500000000001 2024-09-13T07:03:22.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.098568+0000", 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028047800000000002 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.099391+0000", 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00082278400000000002 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107230+0000", 2024-09-13T07:03:22.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0078385420000000004 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107314+0000", 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4701000000000001e-05 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.107383+0000", 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8775000000000005e-05 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1873 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 116.16799159999999, 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010171005, 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1873 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630648+0000", 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630655+0000", 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9600000000000003e-06 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630697+0000", 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2345000000000002e-05 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630714+0000", 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6929e-05 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.633938+0000", 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0032241219999999998 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634050+0000", 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112229 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.634474+0000", 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042370800000000002 2024-09-13T07:03:22.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638761+0000", 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042867929999999997 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640679+0000", 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0019186400000000001 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640781+0000", 2024-09-13T07:03:22.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101338 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640819+0000", 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8819000000000002e-05 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1874 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 116.16790914400001, 2024-09-13T07:03:22.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010256395, 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1874 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630731+0000", 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630730+0000", 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630741+0000", 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0796e-05 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630751+0000", 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0624e-05 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630757+0000", 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9699999999999996e-06 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635040+0000", 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.004282943 2024-09-13T07:03:22.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635308+0000", 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00026790599999999999 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.635837+0000", 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000529058 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638926+0000", 2024-09-13T07:03:22.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.003088529 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640869+0000", 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001943438 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640951+0000", 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2176999999999996e-05 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.640987+0000", 2024-09-13T07:03:22.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6137000000000003e-05 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1875 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 116.167877334, 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010388464, 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1875 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630763+0000", 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630762+0000", 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630766+0000", 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2220000000000004e-06 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630796+0000", 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9297000000000001e-05 2024-09-13T07:03:22.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630807+0000", 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1745e-05 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636725+0000", 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.005917935 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.636885+0000", 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016000899999999999 2024-09-13T07:03:22.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637275+0000", 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00038998100000000002 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638931+0000", 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001656058 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641031+0000", 2024-09-13T07:03:22.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0020998470000000002 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641104+0000", 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2297000000000003e-05 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641151+0000", 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7590999999999999e-05 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1876 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 116.167822176, 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010441181000000001, 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1876 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630818+0000", 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630817+0000", 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630827+0000", 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7659999999999993e-06 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630842+0000", 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5746000000000001e-05 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630853+0000", 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0851e-05 2024-09-13T07:03:22.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637800+0000", 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069463529999999997 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.637893+0000", 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3286000000000001e-05 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.638306+0000", 2024-09-13T07:03:22.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041314699999999999 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641184+0000", 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0028775369999999999 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641191+0000", 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.272e-06 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641230+0000", 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9236999999999997e-05 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641259+0000", 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8983e-05 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1877 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:03:22.845 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 116.167776897, 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015077164000000001, 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1877 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630863+0000", 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630862+0000", 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630868+0000", 2024-09-13T07:03:22.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0229999999999998e-06 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630883+0000", 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4256999999999999e-05 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630893+0000", 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0207000000000001e-05 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641287+0000", 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010393749000000001 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641363+0000", 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6564999999999996e-05 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641649+0000", 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00028597399999999998 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.643988+0000", 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023391430000000001 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645853+0000", 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018647099999999999 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645906+0000", 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2685999999999997e-05 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645940+0000", 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4681000000000002e-05 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1878 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 116.167738363, 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015212707000000001, 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1878 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630902+0000", 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630901+0000", 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630907+0000", 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8499999999999999e-06 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630938+0000", 2024-09-13T07:03:22.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0849999999999998e-05 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630953+0000", 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4976e-05 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641872+0000", 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010919427000000001 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.641945+0000", 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3112000000000001e-05 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642163+0000", 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00021759000000000001 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645762+0000", 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0035996869999999999 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646010+0000", 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00024797099999999999 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646071+0000", 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0789999999999999e-05 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646114+0000", 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3248999999999999e-05 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.856 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1879 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 116.167676744, 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015811899000000001, 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1879 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.857 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:03:22.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630963+0000", 2024-09-13T07:03:22.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630962+0000", 2024-09-13T07:03:22.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:22.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630969+0000", 2024-09-13T07:03:22.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.9820000000000002e-06 2024-09-13T07:03:22.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.630986+0000", 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7428e-05 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.631001+0000", 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5034999999999999e-05 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642547+0000", 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01154556 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642600+0000", 2024-09-13T07:03:22.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3319999999999998e-05 2024-09-13T07:03:22.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:22.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.642823+0000", 2024-09-13T07:03:22.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00022294400000000001 2024-09-13T07:03:22.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645795+0000", 2024-09-13T07:03:22.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0029721359999999998 2024-09-13T07:03:22.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646690+0000", 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00089451999999999999 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646740+0000", 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9583000000000003e-05 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.646775+0000", 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5673999999999998e-05 2024-09-13T07:03:22.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:1892 3.12 3:4d1f9422:::rbd_object_map.11943a018f34:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 116.15345406100001, 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0094804439999999993, 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.863 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 1892 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645186+0000", 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645185+0000", 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645190+0000", 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0479999999999998e-06 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645197+0000", 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0530000000000001e-06 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.645201+0000", 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2030000000000002e-06 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649457+0000", 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0042551339999999998 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649554+0000", 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7503999999999993e-05 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.649969+0000", 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041506699999999998 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.651783+0000", 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001814314 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654573+0000", 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0027892720000000002 2024-09-13T07:03:22.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654626+0000", 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3418000000000003e-05 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.654666+0000", 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0249e-05 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.869 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4002 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 12189696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:22.869 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:03:22.869 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.140996325, 2024-09-13T07:03:22.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092370879999999992, 2024-09-13T07:03:22.869 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.869 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:22.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.869 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4002 2024-09-13T07:03:22.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.869 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657644+0000", 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657643+0000", 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657646+0000", 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.481e-06 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657652+0000", 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657656+0000", 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:03:22.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666742+0000", 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090859809999999999 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666809+0000", 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6055000000000004e-05 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666881+0000", 2024-09-13T07:03:22.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2304000000000004e-05 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4003 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.140976608, 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093834090000000005, 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.873 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4003 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657663+0000", 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657665+0000", 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1849999999999998e-06 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657671+0000", 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657676+0000", 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8110000000000002e-06 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666905+0000", 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092287140000000007 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.666971+0000", 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6663999999999997e-05 2024-09-13T07:03:22.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667047+0000", 2024-09-13T07:03:22.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5395999999999994e-05 2024-09-13T07:03:22.877 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.877 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.877 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.877 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4004 3.1f 3:f96129b9:::rbd_data.11945d91e99c.0000000000000003:8 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:22.877 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:03:22.877 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.140959278, 2024-09-13T07:03:22.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095403220000000004, 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4004 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657681+0000", 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657680+0000", 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657687+0000", 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7220000000000002e-06 2024-09-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657693+0000", 2024-09-13T07:03:22.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9549999999999997e-06 2024-09-13T07:03:22.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657698+0000", 2024-09-13T07:03:22.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9930000000000003e-06 2024-09-13T07:03:22.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667073+0000", 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0093753029999999998 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667144+0000", 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0879999999999999e-05 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667221+0000", 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6843999999999998e-05 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.881 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:4010 3.1b 3:d801a9a5:::rbd_data.11945d91e99c.0000000000000004:8 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.140663616, 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.009213242, 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 4010 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.882 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657976+0000", 2024-09-13T07:03:22.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657979+0000", 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5189999999999999e-06 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657985+0000", 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8759999999999999e-06 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.657991+0000", 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8189999999999997e-06 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667026+0000", 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090349499999999999 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667106+0000", 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9337000000000003e-05 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:30.667190+0000", 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4117999999999997e-05 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12386 3.12 3:49c9e7d4:::rbd_data.11945fc173dd.0000000000000007:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 102.579460797, 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.041224072, 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.886 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12386 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219179+0000", 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219177+0000", 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219191+0000", 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3699999999999999e-05 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219220+0000", 2024-09-13T07:03:22.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9524000000000001e-05 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219235+0000", 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4633e-05 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219346+0000", 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011097100000000001 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.219589+0000", 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000242678 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.229698+0000", 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.01010906 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260149+0000", 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.030451734000000001 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260257+0000", 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000108078 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260346+0000", 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8634000000000006e-05 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260403+0000", 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.719e-05 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop_reply(client.4500.0:12386 3.12 e41/22 ondisk, result = 0)", 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 102.547381894, 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0091663089999999992, 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251258+0000", 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251257+0000", 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251263+0000", 2024-09-13T07:03:22.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5610000000000004e-06 2024-09-13T07:03:22.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251283+0000", 2024-09-13T07:03:22.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9405000000000001e-05 2024-09-13T07:03:22.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.251292+0000", 2024-09-13T07:03:22.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1309999999999995e-06 2024-09-13T07:03:22.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260230+0000", 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089377950000000001 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260248+0000", 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8723999999999999e-05 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:40.260424+0000", 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017614700000000001 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.895 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137474 3.6 3:65c31370:::rbd_data.1194af11889b.000000000000013b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 186185~11715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.054704685000001, 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0054357140000001, 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137474 2024-09-13T07:03:22.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743934+0000", 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743949+0000", 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5705999999999998e-05 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743960+0000", 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0084e-05 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743971+0000", 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0942e-05 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744045+0000", 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4626e-05 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744062+0000", 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7339000000000001e-05 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746366+0000", 2024-09-13T07:03:22.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.002303457 2024-09-13T07:03:22.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746520+0000", 2024-09-13T07:03:22.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015385900000000001 2024-09-13T07:03:22.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:22.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746734+0000", 2024-09-13T07:03:22.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000214253 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749128+0000", 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002394129 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749267+0000", 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013878299999999999 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749328+0000", 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0918000000000002e-05 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749371+0000", 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3188000000000001e-05 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138463 3.17 3:efe7ec01:::rbd_object_map.11949f62878b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.429734+0000", 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.368906078, 2024-09-13T07:03:22.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0092770100000000005, 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138463 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429734+0000", 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:22.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429734+0000", 2024-09-13T07:03:22.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:22.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:22.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429734+0000", 2024-09-13T07:03:22.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:22.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:22.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429736+0000", 2024-09-13T07:03:22.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.621e-06 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429740+0000", 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0350000000000003e-06 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.429744+0000", 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3909999999999998e-06 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:22.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.432092+0000", 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0023486330000000001 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.432189+0000", 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6279000000000004e-05 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.432610+0000", 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00042100200000000002 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:22.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.435051+0000", 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0024418500000000002 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.438887+0000", 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0038353900000000002 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.438959+0000", 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1880999999999998e-05 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:22.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:22.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.439011+0000", 2024-09-13T07:03:22.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2268999999999997e-05 2024-09-13T07:03:22.908 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.908 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.908 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.908 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:22.908 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:22.908 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:22.909 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:03:22.910 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:22.906+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:22.910 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:23.010 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:23.007+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:23.033 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:23.033 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:23.033 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:23.033 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:23.036 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone43 from 0xb399291 (overlap 0x1354b9e) to 0x1eedf6 2024-09-13T07:03:23.044 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:23.111 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:23.110+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:23.147 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:23.147 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:23.147 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:23.147 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:23.148 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:23.149 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-13T07:03:23.157 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:23.167 INFO:teuthology.orchestra.run.smithi118.stdout:5322 trunc from 0xb399291 to 0x35f0d93 2024-09-13T07:03:23.195 INFO:teuthology.orchestra.run.smithi118.stdout:5323 punch from 0x2acb223 to 0x2acb3b5, (0x192 bytes) 2024-09-13T07:03:23.195 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:23.195 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:23.195 INFO:teuthology.orchestra.run.smithi118.stdout:5326 read 0x9400a4 thru 0x9428ec (0x2849 bytes) 2024-09-13T07:03:23.196 INFO:teuthology.orchestra.run.smithi118.stdout:5327 punch from 0x27cff1b to 0x27dca11, (0xcaf6 bytes) 2024-09-13T07:03:23.197 INFO:teuthology.orchestra.run.smithi118.stdout:5328 trunc from 0x35f0d93 to 0x3d385af 2024-09-13T07:03:23.207 INFO:teuthology.orchestra.run.smithi118.stdout:5329 read 0x1e1c81c thru 0x1e25269 (0x8a4e bytes) 2024-09-13T07:03:23.212 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:23.207+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:23.229 INFO:teuthology.orchestra.run.smithi118.stdout:5330 writesame 0xa6ee318 thru 0xa6f23c6 data_size 0x1(0x40af bytes) 2024-09-13T07:03:23.229 INFO:teuthology.orchestra.run.smithi118.stdout:5331 write 0x5f4429c thru 0x5f49551 (0x52b6 bytes) 2024-09-13T07:03:23.235 INFO:teuthology.orchestra.run.smithi118.stdout:5332 write 0x88820ce thru 0x888fa50 (0xd983 bytes) 2024-09-13T07:03:23.242 INFO:teuthology.orchestra.run.smithi118.stdout:5333 read 0x4e070c thru 0x4ecd7b (0xc670 bytes) 2024-09-13T07:03:23.243 INFO:teuthology.orchestra.run.smithi118.stdout:5334 punch from 0x1c8b478 to 0x1c94033, (0x8bbb bytes) 2024-09-13T07:03:23.243 INFO:teuthology.orchestra.run.smithi118.stdout:5335 write 0x2f53c19 thru 0x2f5d61a (0x9a02 bytes) 2024-09-13T07:03:23.263 INFO:teuthology.orchestra.run.smithi118.stdout:5336 write 0xb923668 thru 0xb92eb7c (0xb515 bytes) 2024-09-13T07:03:23.270 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:23.270 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.776890976999994, 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:03:23.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.272 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:23.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:23.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:03:23.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:23.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:03:23.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:03:23.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:03:23.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:23.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.290407412999997, 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.276 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.277 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.277 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:03:23.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.277 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:23.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.280 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.290337484000005, 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:23.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:23.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:03:23.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:03:23.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:23.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:03:23.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:03:23.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:03:23.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:03:23.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.290321612, 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.287 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.288 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.288 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.288 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.288 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:03:23.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.288 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:23.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:03:23.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:03:23.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.290251154999993, 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:03:23.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:03:23.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286899117000004, 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.299 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.300 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.300 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.300 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:23.300 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286882816000002, 2024-09-13T07:03:23.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:03:23.300 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.300 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.300 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.300 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.300 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.300 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.303 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286863030999996, 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:03:23.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.307 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.308 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.308 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:23.308 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286846757999996, 2024-09-13T07:03:23.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:03:23.308 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.308 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.308 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.308 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.308 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.308 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:03:23.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:03:23.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:03:23.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:03:23.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:03:23.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:03:23.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.311 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.312 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.312 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:23.312 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286828932999995, 2024-09-13T07:03:23.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:03:23.312 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.312 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.312 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.312 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.312 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.312 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:03:23.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.315 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.316 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.316 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:23.316 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286812839999996, 2024-09-13T07:03:23.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:03:23.316 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.316 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.316 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.316 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.316 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.316 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:03:23.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:03:23.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.319 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.320 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.320 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:23.320 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286791653999998, 2024-09-13T07:03:23.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:03:23.320 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.320 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.320 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:03:23.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.324 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.324 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:23.324 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286775621000004, 2024-09-13T07:03:23.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:03:23.324 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.324 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.324 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:03:23.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.325 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:23.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:23.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:03:23.327 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286726381999998, 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.328 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:03:23.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:23.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:03:23.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:03:23.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:03:23.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286708321999996, 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.332 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.333 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:03:23.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.333 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:23.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:23.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:03:23.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286691806999997, 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.336 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.337 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:03:23.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.337 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:23.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:23.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:03:23.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:03:23.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:03:23.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286673441999994, 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.340 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.341 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.341 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:03:23.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.341 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:23.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:23.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:03:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286657344000005, 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:03:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:03:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:03:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286641408999998, 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:03:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.286624911999994, 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:03:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.356 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:23.357 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:23.358 INFO:teuthology.orchestra.run.smithi118.stdout:5338 write 0xecd359e thru 0xecdc18f (0x8bf2 bytes) 2024-09-13T07:03:23.359 INFO:teuthology.orchestra.run.smithi118.stdout:5339 punch from 0x45cac0e to 0x45cbc90, (0x1082 bytes) 2024-09-13T07:03:23.359 INFO:teuthology.orchestra.run.smithi118.stdout:5341 write 0xa05e393 thru 0xa05f729 (0x1397 bytes) 2024-09-13T07:03:23.359 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:23.359 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:23.359 INFO:teuthology.orchestra.run.smithi118.stdout:5345 read 0x2543a61 thru 0x254fd85 (0xc325 bytes) 2024-09-13T07:03:23.359 INFO:teuthology.orchestra.run.smithi118.stdout:5347 write 0xa8cd5f thru 0xa97b17 (0xadb9 bytes) 2024-09-13T07:03:23.359 INFO:teuthology.orchestra.run.smithi118.stdout:5349 punch from 0x6a708d8 to 0x6a71f23, (0x164b bytes) 2024-09-13T07:03:23.359 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:23.359 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:23.359 INFO:teuthology.orchestra.run.smithi118.stdout:5355 read 0x6dfa69a thru 0x6e09abd (0xf424 bytes) 2024-09-13T07:03:23.359 INFO:teuthology.orchestra.run.smithi118.stdout:5356 write 0x1fedc22 thru 0x1ff638e (0x876d bytes) 2024-09-13T07:03:23.359 INFO:teuthology.orchestra.run.smithi118.stdout:5357 write 0xb985a0a thru 0xb989eb0 (0x44a7 bytes) 2024-09-13T07:03:23.359 INFO:teuthology.orchestra.run.smithi118.stdout:5358 writesame 0x65671e4 thru 0x656dbe4 data_size 0x1(0x6a01 bytes) 2024-09-13T07:03:23.360 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:23.354+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:23.362 INFO:teuthology.orchestra.run.smithi118.stdout:5359 read 0x997f859 thru 0x99878f3 (0x809b bytes) 2024-09-13T07:03:23.382 INFO:teuthology.orchestra.run.smithi118.stdout:5360 trunc from 0xecdc190 to 0x55771f8 2024-09-13T07:03:23.405 INFO:teuthology.orchestra.run.smithi118.stdout:5361 read 0x434cd8a thru 0x434ef25 (0x219c bytes) 2024-09-13T07:03:23.422 INFO:teuthology.orchestra.run.smithi118.stdout:5363 writesame 0x8260b77 thru 0x82679ab data_size 0x59(0x6e35 bytes) 2024-09-13T07:03:23.422 INFO:teuthology.orchestra.run.smithi118.stdout:5364 punch from 0x7348033 to 0x7351ea6, (0x9e73 bytes) 2024-09-13T07:03:23.427 INFO:teuthology.orchestra.run.smithi118.stdout:5365 trunc from 0x82679ac to 0xb1acad3 2024-09-13T07:03:23.439 INFO:teuthology.orchestra.run.smithi118.stdout:5366 clone 45 order 20 su 32768 sc 7 2024-09-13T07:03:23.458 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:23.455+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:23.472 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:23.472 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:23.472 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:23.472 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:23.482 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:23.559 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:23.558+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:23.632 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:23.633 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:23.633 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:23.642 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:23.660 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:23.655+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:23.754 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:23.755 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:23.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:23.755 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:23.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.755 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:03:23.755 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:23.755 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.69679465199999, 2024-09-13T07:03:23.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:03:23.755 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.755 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:23.755 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:03:23.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:03:23.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:03:23.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:03:23.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:03:23.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:03:23.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:03:23.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.759 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133476 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.879034789000002, 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0096071349999999993, 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133476 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874350+0000", 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874356+0000", 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5999999999999997e-06 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874369+0000", 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3125000000000001e-05 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874380+0000", 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1396999999999999e-05 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882884+0000", 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.008503666 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883050+0000", 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000165926 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883958+0000", 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00090819200000000005 2024-09-13T07:03:23.763 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133477 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1310720~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.878995570000001, 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010531547, 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133477 2024-09-13T07:03:23.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874394+0000", 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7829999999999998e-06 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874407+0000", 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3050999999999999e-05 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874418+0000", 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0974e-05 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883998+0000", 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095798310000000005 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884148+0000", 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014957499999999999 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884922+0000", 2024-09-13T07:03:23.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000774127 2024-09-13T07:03:23.768 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.768 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.768 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.768 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133484 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:23.768 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:03:23.768 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.878912415999999, 2024-09-13T07:03:23.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011150092, 2024-09-13T07:03:23.768 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.769 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.769 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.769 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133484 2024-09-13T07:03:23.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.769 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:03:23.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874472+0000", 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874479+0000", 2024-09-13T07:03:23.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5700000000000004e-06 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874497+0000", 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7385999999999999e-05 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874510+0000", 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3628999999999999e-05 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884968+0000", 2024-09-13T07:03:23.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010458053 2024-09-13T07:03:23.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885119+0000", 2024-09-13T07:03:23.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000150932 2024-09-13T07:03:23.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885624+0000", 2024-09-13T07:03:23.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050432500000000002 2024-09-13T07:03:23.772 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.772 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.772 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133485 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.878860723999999, 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011776926, 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133485 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.773 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874524+0000", 2024-09-13T07:03:23.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874530+0000", 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8599999999999998e-06 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874545+0000", 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5186e-05 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874558+0000", 2024-09-13T07:03:23.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3040000000000001e-05 2024-09-13T07:03:23.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885700+0000", 2024-09-13T07:03:23.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011141514999999999 2024-09-13T07:03:23.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885859+0000", 2024-09-13T07:03:23.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015918599999999999 2024-09-13T07:03:23.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886302+0000", 2024-09-13T07:03:23.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044316899999999998 2024-09-13T07:03:23.777 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.777 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.777 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.777 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133494 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:23.777 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:03:23.777 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.878724927, 2024-09-13T07:03:23.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012188055999999999, 2024-09-13T07:03:23.777 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.777 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133494 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874660+0000", 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874666+0000", 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7300000000000002e-06 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874701+0000", 2024-09-13T07:03:23.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5095999999999997e-05 2024-09-13T07:03:23.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874714+0000", 2024-09-13T07:03:23.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2833000000000001e-05 2024-09-13T07:03:23.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886333+0000", 2024-09-13T07:03:23.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011619078 2024-09-13T07:03:23.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886405+0000", 2024-09-13T07:03:23.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2525000000000001e-05 2024-09-13T07:03:23.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886849+0000", 2024-09-13T07:03:23.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000443697 2024-09-13T07:03:23.781 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.781 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.782 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.782 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133498 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:23.782 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:03:23.782 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.878660164999999, 2024-09-13T07:03:23.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012651943000000001, 2024-09-13T07:03:23.782 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.782 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.783 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133498 2024-09-13T07:03:23.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.783 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:03:23.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:03:23.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874725+0000", 2024-09-13T07:03:23.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:23.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874731+0000", 2024-09-13T07:03:23.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0159999999999997e-06 2024-09-13T07:03:23.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874788+0000", 2024-09-13T07:03:23.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7021000000000002e-05 2024-09-13T07:03:23.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874797+0000", 2024-09-13T07:03:23.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3430000000000002e-06 2024-09-13T07:03:23.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886880+0000", 2024-09-13T07:03:23.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012083287 2024-09-13T07:03:23.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886959+0000", 2024-09-13T07:03:23.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8582000000000001e-05 2024-09-13T07:03:23.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887378+0000", 2024-09-13T07:03:23.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041888400000000001 2024-09-13T07:03:23.786 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.786 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.878578294999997, 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:03:23.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.788 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:23.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:23.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:03:23.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:03:23.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:03:23.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:03:23.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:03:23.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:03:23.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.878528907000003, 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:03:23.791 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:03:23.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:03:23.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:03:23.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:03:23.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:03:23.795 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.795 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.795 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.796 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:23.796 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:23.796 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.878506199999997, 2024-09-13T07:03:23.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:03:23.796 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.796 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.796 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:03:23.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.796 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:03:23.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.878462007000003, 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.800 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.801 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:03:23.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.801 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:23.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:23.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:03:23.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.804 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133516 3.0 3:07bbab57:::rbd_data.11946d7ee8e1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.867626825999999, 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010861226999999999, 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133516 2024-09-13T07:03:23.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.806 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:03:23.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:03:23.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885757+0000", 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886801+0000", 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010439550000000001 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886827+0000", 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5803e-05 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886839+0000", 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2136e-05 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886894+0000", 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4434000000000001e-05 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887043+0000", 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014903200000000001 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887376+0000", 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00033327399999999999 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.894299+0000", 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069232620000000003 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896556+0000", 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022571309999999999 2024-09-13T07:03:23.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896599+0000", 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2620999999999997e-05 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896620+0000", 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1367999999999999e-05 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.810 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.847482917999997, 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:23.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:03:23.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:03:23.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:03:23.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:03:23.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:03:23.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:03:23.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.814 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137457 3.18 3:1a596dcc:::rbd_data.1194af11889b.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.101861108000001, 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033734403000000003, 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137457 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.815 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651523+0000", 2024-09-13T07:03:23.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:23.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651531+0000", 2024-09-13T07:03:23.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.328e-06 2024-09-13T07:03:23.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651545+0000", 2024-09-13T07:03:23.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4270000000000001e-05 2024-09-13T07:03:23.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651555+0000", 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5210000000000008e-06 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651626+0000", 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1915999999999994e-05 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651761+0000", 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000134838 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.658577+0000", 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068151820000000004 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685181+0000", 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026604200000000001 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685193+0000", 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2128e-05 2024-09-13T07:03:23.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685226+0000", 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3504999999999999e-05 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685259+0000", 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2866e-05 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.821 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138565 3.0 3:002d0425:::rbd_data.11949f62878b.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:23.821 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:23.821 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.088490616, 2024-09-13T07:03:23.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033394489999999999, 2024-09-13T07:03:23.821 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.821 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:23.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.821 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.821 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138565 2024-09-13T07:03:23.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.821 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664893+0000", 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:23.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664906+0000", 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2996000000000001e-05 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664936+0000", 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9609999999999999e-05 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664949+0000", 2024-09-13T07:03:23.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383e-05 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665046+0000", 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6459000000000006e-05 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665259+0000", 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000213141 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:23.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.672522+0000", 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072632110000000003 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.697982+0000", 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025459699999999998 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698018+0000", 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6375999999999999e-05 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698166+0000", 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014818700000000001 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698290+0000", 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012333999999999999 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138587 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:23.826 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 15.040024948999999, 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019224502000000001, 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138587 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713360+0000", 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713368+0000", 2024-09-13T07:03:23.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713383+0000", 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5176e-05 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713390+0000", 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8669999999999996e-06 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713449+0000", 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8938999999999999e-05 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713549+0000", 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010009 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.716087+0000", 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025383820000000001 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732351+0000", 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016263564000000001 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732450+0000", 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9529000000000002e-05 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732532+0000", 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1526000000000005e-05 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732585+0000", 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3819000000000001e-05 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138786 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 14.605809694, 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018847452000000001, 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.832 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138786 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:23.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147575+0000", 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147583+0000", 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147601+0000", 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7961e-05 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147610+0000", 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147661+0000", 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0447e-05 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147754+0000", 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2974999999999997e-05 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.150580+0000", 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002826566 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166135+0000", 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015554772 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:23.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166225+0000", 2024-09-13T07:03:23.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0258000000000002e-05 2024-09-13T07:03:23.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:23.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166334+0000", 2024-09-13T07:03:23.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010816500000000001 2024-09-13T07:03:23.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166424+0000", 2024-09-13T07:03:23.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0116000000000003e-05 2024-09-13T07:03:23.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:139127 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.978758729999999, 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019295931999999998, 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.838 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139127 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774626+0000", 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774635+0000", 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5419999999999994e-06 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774654+0000", 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8359000000000002e-05 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774663+0000", 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774718+0000", 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5315000000000001e-05 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774805+0000", 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7284e-05 2024-09-13T07:03:23.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.779319+0000", 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045138310000000003 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793772+0000", 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014452692 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793821+0000", 2024-09-13T07:03:23.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9563e-05 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793878+0000", 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6953999999999997e-05 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793923+0000", 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4781999999999999e-05 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:145375 3.4 3:23664b2c:::rbd_data.119421a2279e.00000000000001d4:head [set-alloc-hint object_size 524288 write_size 524288,write 32768~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:13.294016+0000", 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.459369579000001, 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090666410000000003, 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 145375 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:13.294016+0000", 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:13.294016+0000", 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:13.294013+0000", 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:13.294040+0000", 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6364999999999999e-05 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:13.294092+0000", 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2431e-05 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:13.294113+0000", 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1155999999999999e-05 2024-09-13T07:03:23.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:13.294209+0000", 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5488999999999993e-05 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:13.294363+0000", 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000153707 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:13.294513+0000", 2024-09-13T07:03:23.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014994699999999999 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:13.295486+0000", 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00097330699999999997 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:13.302927+0000", 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0074414900000000003 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:23.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:13.302991+0000", 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3863999999999997e-05 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:13.303083+0000", 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1807999999999999e-05 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:147368 3.4 3:26bd7659:::rbd_data.119421a2279e.000000000000001d:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:18.583791+0000", 2024-09-13T07:03:23.849 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.1695949719999996, 2024-09-13T07:03:23.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011299273, 2024-09-13T07:03:23.850 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:23.850 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:23.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:23.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:23.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:23.850 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 147368 2024-09-13T07:03:23.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.850 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:23.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:23.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583791+0000", 2024-09-13T07:03:23.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583791+0000", 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583788+0000", 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583803+0000", 2024-09-13T07:03:23.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5135e-05 2024-09-13T07:03:23.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:23.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583832+0000", 2024-09-13T07:03:23.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8741000000000001e-05 2024-09-13T07:03:23.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:23.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583851+0000", 2024-09-13T07:03:23.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8552000000000001e-05 2024-09-13T07:03:23.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:23.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583983+0000", 2024-09-13T07:03:23.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013209699999999999 2024-09-13T07:03:23.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:23.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.584149+0000", 2024-09-13T07:03:23.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000166117 2024-09-13T07:03:23.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:23.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.585549+0000", 2024-09-13T07:03:23.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001399682 2024-09-13T07:03:23.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.591604+0000", 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006055146 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.594934+0000", 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033300309999999998 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.595023+0000", 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8839999999999996e-05 2024-09-13T07:03:23.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:23.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:23.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:23.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.595090+0000", 2024-09-13T07:03:23.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7614999999999996e-05 2024-09-13T07:03:23.855 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.855 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.855 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.855 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:23.855 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:23.855 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:23.857 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:23.857 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:23.851+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:23.901 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:03:23.961 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:23.959+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:24.008 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:24.008 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:24.008 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:24.018 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:24.046 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone44 from 0xb1acad3 (overlap 0x35f0d93) to 0x0 2024-09-13T07:03:24.058 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:24.055+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:24.132 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:24.132 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:24.132 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:24.132 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:24.132 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:24.142 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:24.185 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:24.183+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:24.238 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-13T07:03:24.251 INFO:teuthology.orchestra.run.smithi118.stdout:5367 write 0x788aefc thru 0x7898eea (0xdfef bytes) 2024-09-13T07:03:24.260 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:24.255+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:24.270 INFO:teuthology.orchestra.run.smithi118.stdout:5369 write 0x744d291 thru 0x7454e5f (0x7bcf bytes) 2024-09-13T07:03:24.278 INFO:teuthology.orchestra.run.smithi118.stdout:5370 read 0x544e6e3 thru 0x545d7e3 (0xf101 bytes) 2024-09-13T07:03:24.278 INFO:teuthology.orchestra.run.smithi118.stdout:5372 writesame 0xd9f8ae thru 0xda6034 data_size 0x1(0x6787 bytes) 2024-09-13T07:03:24.304 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:24.304 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:24.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:24.305 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:24.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.305 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.305 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:24.305 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.183766706999997, 2024-09-13T07:03:24.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:03:24.305 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.305 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.305 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:03:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:03:24.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:03:24.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:03:24.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:03:24.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:03:24.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:03:24.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:03:24.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:03:24.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:03:24.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.309 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.309 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.309 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:24.310 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.183746229999997, 2024-09-13T07:03:24.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:03:24.310 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.310 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.310 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.310 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.310 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.310 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:03:24.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.310 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:03:24.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:03:24.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:03:24.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:03:24.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:03:24.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87036 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 6963200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.177062163999999, 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042468831999999998, 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.314 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.315 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87036 2024-09-13T07:03:24.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.315 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:24.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:24.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:24.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127964+0000", 2024-09-13T07:03:24.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7799999999999999e-06 2024-09-13T07:03:24.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127967+0000", 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9730000000000002e-06 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127971+0000", 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4349999999999999e-06 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170202+0000", 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042230940000000002 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170293+0000", 2024-09-13T07:03:24.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0858000000000003e-05 2024-09-13T07:03:24.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170431+0000", 2024-09-13T07:03:24.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013821699999999999 2024-09-13T07:03:24.318 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.318 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.318 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.319 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.319 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:24.319 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.177049627000002, 2024-09-13T07:03:24.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:03:24.319 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.319 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.319 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.319 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.319 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.319 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:03:24.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.319 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:24.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:24.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:24.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:24.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:03:24.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:03:24.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:03:24.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:03:24.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:03:24.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:03:24.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:03:24.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.323 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.323 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.324 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:24.324 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.177037558000002, 2024-09-13T07:03:24.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:03:24.324 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.324 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.324 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:03:24.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.324 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:03:24.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:03:24.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:03:24.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:03:24.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:03:24.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:03:24.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:03:24.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:03:24.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:03:24.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:03:24.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.177025114999999, 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:03:24.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:24.332 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.177007517, 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:03:24.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.337 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.337 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:24.337 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.176996813999999, 2024-09-13T07:03:24.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:03:24.337 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.337 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.337 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.337 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:24.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:03:24.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:03:24.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:03:24.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:03:24.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:03:24.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:03:24.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:03:24.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:03:24.341 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.341 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.341 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.341 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.341 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:24.341 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.176952880999998, 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:24.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:03:24.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:03:24.345 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.176933095000003, 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.346 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:24.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:03:24.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:03:24.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:03:24.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:03:24.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:03:24.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:03:24.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:03:24.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:03:24.350 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.350 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.350 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.350 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.350 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:24.351 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.176917345, 2024-09-13T07:03:24.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:03:24.351 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.351 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.351 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.351 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.351 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.351 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:03:24.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.351 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:24.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:24.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:24.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:24.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:03:24.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:03:24.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:03:24.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:03:24.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:03:24.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:03:24.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:03:24.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:03:24.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:03:24.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:03:24.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:03:24.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:03:24.355 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.355 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.355 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.355 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.355 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:24.355 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.176906092999999, 2024-09-13T07:03:24.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:03:24.356 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.356 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.356 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.356 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.356 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.356 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:03:24.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.356 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:24.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:24.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:03:24.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:24.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:03:24.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:03:24.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:03:24.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:03:24.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:03:24.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:03:24.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:03:24.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:03:24.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:03:24.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:03:24.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:03:24.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:03:24.360 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.360 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.360 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.361 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.361 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:24.361 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.176895494, 2024-09-13T07:03:24.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:03:24.361 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.361 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.361 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.361 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.361 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.361 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:03:24.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.362 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:24.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:24.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:03:24.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:03:24.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:03:24.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:03:24.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:03:24.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:03:24.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:03:24.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:03:24.365 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.365 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.365 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.366 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.366 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:24.366 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.176886001, 2024-09-13T07:03:24.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:03:24.366 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.366 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.366 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:03:24.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.367 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:24.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:24.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:03:24.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:24.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:03:24.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:03:24.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:03:24.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:03:24.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:03:24.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:03:24.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:03:24.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:03:24.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:03:24.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:03:24.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:03:24.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:03:24.371 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.371 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.371 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.372 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.372 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:24.372 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.176876456999999, 2024-09-13T07:03:24.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:03:24.372 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.372 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.372 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.372 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.372 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.372 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:03:24.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.373 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:24.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:24.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:24.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:24.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:03:24.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:03:24.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:03:24.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:03:24.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:03:24.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:03:24.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:03:24.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:03:24.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:03:24.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:03:24.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:03:24.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:03:24.377 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.377 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.377 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.377 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.377 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:24.377 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.176843853999998, 2024-09-13T07:03:24.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:03:24.378 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.378 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.378 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:03:24.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.378 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:24.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:24.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:03:24.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:03:24.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:03:24.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:03:24.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:03:24.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:03:24.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:03:24.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:03:24.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:03:24.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:03:24.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:03:24.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:03:24.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:03:24.382 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.382 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.383 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.383 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.383 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:24.383 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.176830830999997, 2024-09-13T07:03:24.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:03:24.383 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.383 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.384 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.384 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.384 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.384 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:03:24.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.384 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:24.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:24.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:24.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:24.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:03:24.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:03:24.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:03:24.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:03:24.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:03:24.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:03:24.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:03:24.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:03:24.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:03:24.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:03:24.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:03:24.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:03:24.388 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.389 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.389 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.389 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.389 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:24.389 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.173479845999999, 2024-09-13T07:03:24.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:03:24.389 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.389 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.390 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.390 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.390 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.390 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:03:24.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.390 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:24.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:24.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:03:24.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:03:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:03:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:03:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:03:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:03:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:03:24.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:03:24.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:03:24.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:03:24.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:03:24.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:03:24.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:03:24.394 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.394 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.394 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.394 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.394 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:24.394 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.173459569000002, 2024-09-13T07:03:24.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:03:24.394 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.394 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.394 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:24.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:03:24.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:03:24.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:03:24.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:03:24.398 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.398 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.398 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.173439367999997, 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:24.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:03:24.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:03:24.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:24.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:03:24.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:03:24.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:03:24.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:24.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:03:24.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:03:24.403 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.403 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.403 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.403 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:24.403 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:24.403 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:24.404 INFO:teuthology.orchestra.run.smithi118.stdout:5373 punch from 0x7d97b99 to 0x7da671b, (0xeb82 bytes) 2024-09-13T07:03:24.404 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:24.405 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:24.406 INFO:teuthology.orchestra.run.smithi118.stdout:5376 trunc from 0xb1acad3 to 0x4de51ad 2024-09-13T07:03:24.406 INFO:teuthology.orchestra.run.smithi118.stdout:5377 trunc from 0x4de51ad to 0xcd4495d 2024-09-13T07:03:24.406 INFO:teuthology.orchestra.run.smithi118.stdout:5378 read 0x6afd2ee thru 0x6b04229 (0x6f3c bytes) 2024-09-13T07:03:24.406 INFO:teuthology.orchestra.run.smithi118.stdout:5379 writesame 0xa7f68c2 thru 0xa8019c8 data_size 0x1(0xb107 bytes) 2024-09-13T07:03:24.406 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:24.402+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:24.414 INFO:teuthology.orchestra.run.smithi118.stdout:5382 read 0xcc8613f thru 0xcc8d764 (0x7626 bytes) 2024-09-13T07:03:24.414 INFO:teuthology.orchestra.run.smithi118.stdout:5383 write 0x2c48f69 thru 0x2c51c7a (0x8d12 bytes) 2024-09-13T07:03:24.428 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:24.428 INFO:teuthology.orchestra.run.smithi118.stdout:5387 write 0x6dcdcb7 thru 0x6ddb959 (0xdca3 bytes) 2024-09-13T07:03:24.436 INFO:teuthology.orchestra.run.smithi118.stdout:5388 writesame 0x3b14df7 thru 0x3b1bf46 data_size 0x38(0x7150 bytes) 2024-09-13T07:03:24.446 INFO:teuthology.orchestra.run.smithi118.stdout:5389 punch from 0xa62c8db to 0xa63039e, (0x3ac3 bytes) 2024-09-13T07:03:24.446 INFO:teuthology.orchestra.run.smithi118.stdout:5390 write 0x950dd7a thru 0x95101a8 (0x242f bytes) 2024-09-13T07:03:24.467 INFO:teuthology.orchestra.run.smithi118.stdout:5391 writesame 0xe3f49cb thru 0xe3fc11b data_size 0x29(0x7751 bytes) 2024-09-13T07:03:24.468 INFO:teuthology.orchestra.run.smithi118.stdout:5392 punch from 0x8dc0df2 to 0x8dcf724, (0xe932 bytes) 2024-09-13T07:03:24.468 INFO:teuthology.orchestra.run.smithi118.stdout:5394 punch from 0xa769f7f to 0xa771fd6, (0x8057 bytes) 2024-09-13T07:03:24.468 INFO:teuthology.orchestra.run.smithi118.stdout:5396 read 0xa1b8a6b thru 0xa1c409e (0xb634 bytes) 2024-09-13T07:03:24.481 INFO:teuthology.orchestra.run.smithi118.stdout:5399 writesame 0xe803314 thru 0xe8118c2 data_size 0x1(0xe5af bytes) 2024-09-13T07:03:24.481 INFO:teuthology.orchestra.run.smithi118.stdout:5401 punch from 0x14b3c0d to 0x14c2e07, (0xf1fa bytes) 2024-09-13T07:03:24.481 INFO:teuthology.orchestra.run.smithi118.stdout:5402 read 0x14043d3 thru 0x1414269 (0xfe97 bytes) 2024-09-13T07:03:24.482 INFO:teuthology.orchestra.run.smithi118.stdout:5403 read 0x13cd23c thru 0x13d611c (0x8ee1 bytes) 2024-09-13T07:03:24.483 INFO:teuthology.orchestra.run.smithi118.stdout:5404 read 0xe7412e6 thru 0xe74582b (0x4546 bytes) 2024-09-13T07:03:24.483 INFO:teuthology.orchestra.run.smithi118.stdout:5405 read 0x67eccc9 thru 0x67ee62c (0x1964 bytes) 2024-09-13T07:03:24.484 INFO:teuthology.orchestra.run.smithi118.stdout:5406 punch from 0xace35cd to 0xace933e, (0x5d71 bytes) 2024-09-13T07:03:24.484 INFO:teuthology.orchestra.run.smithi118.stdout:5412 punch from 0xc91de5d to 0xc9275ef, (0x9792 bytes) 2024-09-13T07:03:24.484 INFO:teuthology.orchestra.run.smithi118.stdout:5413 writesame 0x17674ef thru 0x176a317 data_size 0x1(0x2e29 bytes) 2024-09-13T07:03:24.505 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:24.503+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:24.523 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:24.523 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:24.523 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:24.523 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:24.526 INFO:teuthology.orchestra.run.smithi118.stdout:5415 trunc from 0xe8118c3 to 0xcba55b 2024-09-13T07:03:24.534 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:24.569 INFO:teuthology.orchestra.run.smithi118.stdout:5416 punch from 0xbea575 to 0xbefe69, (0x58f4 bytes) 2024-09-13T07:03:24.572 INFO:teuthology.orchestra.run.smithi118.stdout:5417 trunc from 0xcba55b to 0x25d020c 2024-09-13T07:03:24.583 INFO:teuthology.orchestra.run.smithi118.stdout:5418 write 0x176ce81 thru 0x176dafb (0xc7b bytes) 2024-09-13T07:03:24.588 INFO:teuthology.orchestra.run.smithi118.stdout:5420 read 0xe746f5 thru 0xe7be07 (0x7713 bytes) 2024-09-13T07:03:24.588 INFO:teuthology.orchestra.run.smithi118.stdout:5421 read 0xba78ba thru 0xba8f9b (0x16e2 bytes) 2024-09-13T07:03:24.593 INFO:teuthology.orchestra.run.smithi118.stdout:5423 trunc from 0x25d020c to 0xd812b5 2024-09-13T07:03:24.606 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:24.603+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:24.633 INFO:teuthology.orchestra.run.smithi118.stdout:5424 write 0x269e74b thru 0x26a7095 (0x894b bytes) 2024-09-13T07:03:24.640 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:24.640 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:24.642 INFO:teuthology.orchestra.run.smithi118.stdout:5426 punch from 0x707c1b to 0x70f661, (0x7a46 bytes) 2024-09-13T07:03:24.642 INFO:teuthology.orchestra.run.smithi118.stdout:5427 read 0x2190790 thru 0x219911c (0x898d bytes) 2024-09-13T07:03:24.642 INFO:teuthology.orchestra.run.smithi118.stdout:5428 punch from 0x20c6c02 to 0x20d6ba5, (0xffa3 bytes) 2024-09-13T07:03:24.642 INFO:teuthology.orchestra.run.smithi118.stdout:5429 read 0xc29c65 thru 0xc32206 (0x85a2 bytes) 2024-09-13T07:03:24.643 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:24.650 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:24.665 INFO:teuthology.orchestra.run.smithi118.stdout:5431 write 0x68faac5 thru 0x6904001 (0x953d bytes) 2024-09-13T07:03:24.673 INFO:teuthology.orchestra.run.smithi118.stdout:5432 read 0x5219525 thru 0x5219703 (0x1df bytes) 2024-09-13T07:03:24.673 INFO:teuthology.orchestra.run.smithi118.stdout:5433 clone 46 order 19 su 4096 sc 15 2024-09-13T07:03:24.701 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:03:24.709 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:24.706+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:24.755 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152574215999998, 2024-09-13T07:03:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:03:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:03:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:03:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:03:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:03:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:03:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:03:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:03:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:03:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:03:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:03:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:03:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:03:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152543115, 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:03:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152520801000001, 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:03:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:03:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:03:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:03:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:03:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152497769999997, 2024-09-13T07:03:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:03:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:03:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:03:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:03:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:03:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:03:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:03:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:03:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:03:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152473219999997, 2024-09-13T07:03:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:03:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:03:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:03:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:03:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:03:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:03:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152437511000002, 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:03:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:03:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:03:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:03:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:03:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:03:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:03:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152412832000003, 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:03:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:03:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:03:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:03:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:03:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:03:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:03:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152379558, 2024-09-13T07:03:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:03:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:03:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:03:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:03:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:03:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:03:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:03:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:03:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.15235277, 2024-09-13T07:03:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:03:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:03:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:03:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:03:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152328216000001, 2024-09-13T07:03:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:03:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:03:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:03:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:03:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:03:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:03:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:03:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:03:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152298447, 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:03:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:03:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:03:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:03:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:03:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:03:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152238603999997, 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:03:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:03:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:03:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:03:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:03:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:03:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:03:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152171754000001, 2024-09-13T07:03:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:03:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:03:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:03:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:03:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:03:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:03:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:03:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:03:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152139251999998, 2024-09-13T07:03:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:03:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:03:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:03:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:03:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:03:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:03:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152104631, 2024-09-13T07:03:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:03:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:03:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:03:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:03:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:03:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:03:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:03:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:03:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:03:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:03:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:03:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:03:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152054399999997, 2024-09-13T07:03:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:03:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:03:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:03:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:03:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:03:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:03:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:03:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.152006213999996, 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.151979576000002, 2024-09-13T07:03:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:03:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:03:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:03:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:03:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:03:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:03:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:03:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:03:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:03:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.151932741000003, 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:03:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:03:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.148578323999999, 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:03:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:03:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:24.855 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:24.856 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:03:24.856 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:24.854+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:24.857 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:24.884 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:24.905 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:24.924 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:24.945 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:24.957 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:24.955+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:24.968 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:24.990 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:25.007 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:25.007 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:25.014 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:25.016 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:25.016 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:25.016 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:25.016 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:25.016 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:25.017 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:25.017 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:25.017 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:25.018 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:03:25.058 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:25.058+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:25.062 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone45 from 0x6904002 (overlap 0xcba55b) to 0x65d2ad 2024-09-13T07:03:25.133 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:25.133 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:25.144 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:03:25.159 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:25.155+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:25.260 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:25.255+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:25.280 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-13T07:03:25.282 INFO:teuthology.orchestra.run.smithi118.stdout:5434 read 0x380e9e6 thru 0x3819dce (0xb3e9 bytes) 2024-09-13T07:03:25.297 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:25.297 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:25.308 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:25.319 INFO:teuthology.orchestra.run.smithi118.stdout:5435 writesame 0xc7eef2d thru 0xc7fcd51 data_size 0x25(0xde25 bytes) 2024-09-13T07:03:25.319 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:25.319 INFO:teuthology.orchestra.run.smithi118.stdout:5437 write 0x68719c5 thru 0x68814ca (0xfb06 bytes) 2024-09-13T07:03:25.361 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:25.359+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:25.388 INFO:teuthology.orchestra.run.smithi118.stdout:5438 trunc from 0xc7fcd52 to 0x523a9cd 2024-09-13T07:03:25.431 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:25.432 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:25.441 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:25.445 INFO:teuthology.orchestra.run.smithi118.stdout:5439 trunc from 0x523a9cd to 0xc15fe0e 2024-09-13T07:03:25.457 INFO:teuthology.orchestra.run.smithi118.stdout:5440 write 0x1dbe260 thru 0x1dc46bd (0x645e bytes) 2024-09-13T07:03:25.461 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:25.458+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:25.476 INFO:teuthology.orchestra.run.smithi118.stdout:5441 read 0x6c92dcd thru 0x6c93017 (0x24b bytes) 2024-09-13T07:03:25.487 INFO:teuthology.orchestra.run.smithi118.stdout:5442 write 0xcc9bdb8 thru 0xccaa949 (0xeb92 bytes) 2024-09-13T07:03:25.529 INFO:teuthology.orchestra.run.smithi118.stdout:5444 trunc from 0xccaa94a to 0x3a11580 2024-09-13T07:03:25.601 INFO:teuthology.orchestra.run.smithi118.stdout:5445 read 0x17a139b thru 0x17a6526 (0x518c bytes) 2024-09-13T07:03:25.601 INFO:teuthology.orchestra.run.smithi118.stdout:5446 punch from 0xdf907e to 0xe079ae, (0xe930 bytes) 2024-09-13T07:03:25.601 INFO:teuthology.orchestra.run.smithi118.stdout:5447 writesame 0x409fcea thru 0x40a036f data_size 0x5(0x686 bytes) 2024-09-13T07:03:25.602 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:25.602 INFO:teuthology.orchestra.run.smithi118.stdout:5449 read 0x1972117 thru 0x197a156 (0x8040 bytes) 2024-09-13T07:03:25.602 INFO:teuthology.orchestra.run.smithi118.stdout:5450 trunc from 0x40a0370 to 0x720657a 2024-09-13T07:03:25.602 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:25.602 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:25.602 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:25.595+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:25.602 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:25.609 INFO:teuthology.orchestra.run.smithi118.stdout:5451 trunc from 0x720657a to 0xa2b11c0 2024-09-13T07:03:25.622 INFO:teuthology.orchestra.run.smithi118.stdout:5452 writesame 0x922fca3 thru 0x92311b4 data_size 0x3a(0x1512 bytes) 2024-09-13T07:03:25.630 INFO:teuthology.orchestra.run.smithi118.stdout:5453 read 0x1ed0e8e thru 0x1ed6829 (0x599c bytes) 2024-09-13T07:03:25.631 INFO:teuthology.orchestra.run.smithi118.stdout:5454 write 0x1e0131e thru 0x1e08370 (0x7053 bytes) 2024-09-13T07:03:25.661 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:25.661 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:25.661 INFO:teuthology.orchestra.run.smithi118.stdout:5457 read 0x649ed74 thru 0x64a8c1d (0x9eaa bytes) 2024-09-13T07:03:25.673 INFO:teuthology.orchestra.run.smithi118.stdout:5459 trunc from 0xa2b11c0 to 0x46c0940 2024-09-13T07:03:25.692 INFO:teuthology.orchestra.run.smithi118.stdout:5461 clone 47 order 23 su 32768 sc 6 2024-09-13T07:03:25.701 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:25.698+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:25.744 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:25.745 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:25.755 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:25.802 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:25.799+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:25.861 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:25.861 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:25.870 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:25.902 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:25.902+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:25.916 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:03:25.982 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:25.982 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:25.992 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:26.003 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:26.002+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:26.083 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone46 from 0x46c0940 (overlap 0x3a11580) to 0x2e74466 2024-09-13T07:03:26.104 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:26.104 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:26.104 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:26.099+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:26.115 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:26.205 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:26.202+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:26.219 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:26.219 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:26.231 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:26.306 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:26.306+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:26.329 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:26.329 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:26.339 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:26.360 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-13T07:03:26.386 INFO:teuthology.orchestra.run.smithi118.stdout:5462 read 0x437dfee thru 0x438a355 (0xc368 bytes) 2024-09-13T07:03:26.388 INFO:teuthology.orchestra.run.smithi118.stdout:5466 write 0x17f7138 thru 0x1800a22 (0x98eb bytes) 2024-09-13T07:03:26.410 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:26.407+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:26.479 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:26.479 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:26.495 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:26.538 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:26.535+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:26.609 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:26.607+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:26.643 INFO:teuthology.orchestra.run.smithi118.stdout:5467 read 0x3f57ce9 thru 0x3f5acf7 (0x300f bytes) 2024-09-13T07:03:26.644 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:26.644 INFO:teuthology.orchestra.run.smithi118.stdout:5469 read 0x12d446a thru 0x12e3d98 (0xf92f bytes) 2024-09-13T07:03:26.644 INFO:teuthology.orchestra.run.smithi118.stdout:5470 write 0x2dd3e43 thru 0x2ddf47e (0xb63c bytes) 2024-09-13T07:03:26.671 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:26.671 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:26.684 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:26.730 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:26.727+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:26.801 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:26.802 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:26.811 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:26.811 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:26.807+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:26.829 INFO:teuthology.orchestra.run.smithi118.stdout:5471 trunc from 0x46c0940 to 0xda88276 2024-09-13T07:03:26.842 INFO:teuthology.orchestra.run.smithi118.stdout:5472 write 0x478ecf8 thru 0x479023c (0x1545 bytes) 2024-09-13T07:03:26.890 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 7, 2, 0] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:03:26.890 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:03:26.891 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-13T07:03:26.891 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T07:03:26.891 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T07:03:26.891 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-13T07:03:26.894 INFO:teuthology.orchestra.run.smithi118.stdout:5473 trunc from 0xda88276 to 0x6ac49aa 2024-09-13T07:03:26.909 INFO:teuthology.orchestra.run.smithi118.stdout:5475 trunc from 0x6ac49aa to 0x4643b3c 2024-09-13T07:03:26.915 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:26.907+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:26.935 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:03:26.941 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:26.941 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:26.958 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:27.012 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:27.007+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:27.018 INFO:teuthology.orchestra.run.smithi118.stdout:5476 write 0xed8ffad thru 0xed936a0 (0x36f4 bytes) 2024-09-13T07:03:27.027 INFO:teuthology.orchestra.run.smithi118.stdout:5477 punch from 0x4fd31bb to 0x4fd591f, (0x2764 bytes) 2024-09-13T07:03:27.027 INFO:teuthology.orchestra.run.smithi118.stdout:5478 read 0xc19fd48 thru 0xc1a5611 (0x58ca bytes) 2024-09-13T07:03:27.027 INFO:teuthology.orchestra.run.smithi118.stdout:5479 punch from 0x59098dd to 0x590a774, (0xe97 bytes) 2024-09-13T07:03:27.027 INFO:teuthology.orchestra.run.smithi118.stdout:5480 write 0x5e3391b thru 0x5e36abb (0x31a1 bytes) 2024-09-13T07:03:27.029 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:27.029 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T07:03:27.029 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:27.030 INFO:teuthology.orchestra.run.smithi118.stdout:5481 punch from 0x3a24174 to 0x3a32bec, (0xea78 bytes) 2024-09-13T07:03:27.030 INFO:teuthology.orchestra.run.smithi118.stdout:5483 punch from 0xc605c37 to 0xc612090, (0xc459 bytes) 2024-09-13T07:03:27.030 INFO:teuthology.orchestra.run.smithi118.stdout:5484 read 0x411081a thru 0x4113ae5 (0x32cc bytes) 2024-09-13T07:03:27.030 INFO:teuthology.orchestra.run.smithi118.stdout:5486 clone 48 order 24 su 131072 sc 4 2024-09-13T07:03:27.032 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:03:27.111 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:27.111 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:27.113 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:27.114+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:27.121 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:27.179 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:27.201 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:27.214 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:27.211+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:27.223 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:27.239 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:27.239 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:27.239 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:27.239 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:27.239 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:27.243 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:27.248 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:27.265 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:27.289 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:27.313 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:27.315 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:27.314+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:27.336 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:27.339 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:27.339 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:27.339 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:27.339 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:27.339 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:27.339 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:27.339 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:27.339 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:27.352 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:27.352 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:27.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:27.352 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:27.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.352 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:03:27.352 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:27.352 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.29412895199999, 2024-09-13T07:03:27.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:03:27.353 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.353 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:27.353 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:27.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:27.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:03:27.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:27.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:03:27.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:03:27.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:03:27.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:03:27.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:03:27.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:03:27.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:03:27.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:03:27.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:03:27.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:03:27.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:27.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:03:27.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:03:27.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:03:27.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:03:27.357 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.357 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.357 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.357 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133476 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:27.357 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:03:27.357 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.476369089000002, 2024-09-13T07:03:27.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0096071349999999993, 2024-09-13T07:03:27.357 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.357 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.357 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133476 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874351+0000", 2024-09-13T07:03:27.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874350+0000", 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874356+0000", 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5999999999999997e-06 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874369+0000", 2024-09-13T07:03:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3125000000000001e-05 2024-09-13T07:03:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874380+0000", 2024-09-13T07:03:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1396999999999999e-05 2024-09-13T07:03:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.882884+0000", 2024-09-13T07:03:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.008503666 2024-09-13T07:03:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883050+0000", 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000165926 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883958+0000", 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00090819200000000005 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.361 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133477 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1310720~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.476329870000001, 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010531547, 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133477 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874390+0000", 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874394+0000", 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7829999999999998e-06 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874407+0000", 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3050999999999999e-05 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874418+0000", 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0974e-05 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.883998+0000", 2024-09-13T07:03:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0095798310000000005 2024-09-13T07:03:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884148+0000", 2024-09-13T07:03:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014957499999999999 2024-09-13T07:03:27.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884922+0000", 2024-09-13T07:03:27.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000774127 2024-09-13T07:03:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133484 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:03:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.476246715999999, 2024-09-13T07:03:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011150092, 2024-09-13T07:03:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.369 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133484 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874473+0000", 2024-09-13T07:03:27.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874472+0000", 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874479+0000", 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5700000000000004e-06 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874497+0000", 2024-09-13T07:03:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7385999999999999e-05 2024-09-13T07:03:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874510+0000", 2024-09-13T07:03:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3628999999999999e-05 2024-09-13T07:03:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.884968+0000", 2024-09-13T07:03:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010458053 2024-09-13T07:03:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885119+0000", 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000150932 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885624+0000", 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00050432500000000002 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.373 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133485 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.476195023999999, 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011776926, 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133485 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874525+0000", 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874524+0000", 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874530+0000", 2024-09-13T07:03:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8599999999999998e-06 2024-09-13T07:03:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874545+0000", 2024-09-13T07:03:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5186e-05 2024-09-13T07:03:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874558+0000", 2024-09-13T07:03:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3040000000000001e-05 2024-09-13T07:03:27.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885700+0000", 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011141514999999999 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885859+0000", 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015918599999999999 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886302+0000", 2024-09-13T07:03:27.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044316899999999998 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133494 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.476059227, 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012188055999999999, 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.378 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133494 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874661+0000", 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874660+0000", 2024-09-13T07:03:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874666+0000", 2024-09-13T07:03:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7300000000000002e-06 2024-09-13T07:03:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874701+0000", 2024-09-13T07:03:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5095999999999997e-05 2024-09-13T07:03:27.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874714+0000", 2024-09-13T07:03:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2833000000000001e-05 2024-09-13T07:03:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886333+0000", 2024-09-13T07:03:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011619078 2024-09-13T07:03:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886405+0000", 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2525000000000001e-05 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886849+0000", 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000443697 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133498 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:03:27.382 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.475994464999999, 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012651943000000001, 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133498 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874726+0000", 2024-09-13T07:03:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874725+0000", 2024-09-13T07:03:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874731+0000", 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0159999999999997e-06 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874788+0000", 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7021000000000002e-05 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874797+0000", 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3430000000000002e-06 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886880+0000", 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012083287 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886959+0000", 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8582000000000001e-05 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887378+0000", 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00041888400000000001 2024-09-13T07:03:27.386 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.475912594999997, 2024-09-13T07:03:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:03:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:03:27.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:03:27.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.475863207000003, 2024-09-13T07:03:27.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:03:27.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.475840499999997, 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:03:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:03:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:03:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:03:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:03:27.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:03:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:03:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.475796307000003, 2024-09-13T07:03:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:03:27.400 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:03:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:03:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:03:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:03:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:03:27.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133516 3.0 3:07bbab57:::rbd_data.11946d7ee8e1.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.464961125999999, 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010861226999999999, 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133516 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:03:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885759+0000", 2024-09-13T07:03:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.885757+0000", 2024-09-13T07:03:27.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886801+0000", 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0010439550000000001 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886827+0000", 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5803e-05 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886839+0000", 2024-09-13T07:03:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2136e-05 2024-09-13T07:03:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.886894+0000", 2024-09-13T07:03:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4434000000000001e-05 2024-09-13T07:03:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887043+0000", 2024-09-13T07:03:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014903200000000001 2024-09-13T07:03:27.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887376+0000", 2024-09-13T07:03:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00033327399999999999 2024-09-13T07:03:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.894299+0000", 2024-09-13T07:03:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0069232620000000003 2024-09-13T07:03:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:27.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896556+0000", 2024-09-13T07:03:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022571309999999999 2024-09-13T07:03:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896599+0000", 2024-09-13T07:03:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2620999999999997e-05 2024-09-13T07:03:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.896620+0000", 2024-09-13T07:03:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1367999999999999e-05 2024-09-13T07:03:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.410 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:03:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.444817217999997, 2024-09-13T07:03:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:03:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:03:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:03:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:03:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:03:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:03:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:03:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:03:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:03:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:03:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:03:27.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:03:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:03:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:03:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:03:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137457 3.18 3:1a596dcc:::rbd_data.1194af11889b.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T07:03:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.699195408000001, 2024-09-13T07:03:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033734403000000003, 2024-09-13T07:03:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137457 2024-09-13T07:03:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.416 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651523+0000", 2024-09-13T07:03:27.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651531+0000", 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.328e-06 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651545+0000", 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4270000000000001e-05 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651555+0000", 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5210000000000008e-06 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651626+0000", 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1915999999999994e-05 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651761+0000", 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000134838 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:27.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.658577+0000", 2024-09-13T07:03:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068151820000000004 2024-09-13T07:03:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685181+0000", 2024-09-13T07:03:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026604200000000001 2024-09-13T07:03:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685193+0000", 2024-09-13T07:03:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2128e-05 2024-09-13T07:03:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685226+0000", 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3504999999999999e-05 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685259+0000", 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2866e-05 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138565 3.0 3:002d0425:::rbd_data.11949f62878b.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:27.421 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.685824916000001, 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033394489999999999, 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138565 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:27.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664893+0000", 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664906+0000", 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2996000000000001e-05 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664936+0000", 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9609999999999999e-05 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664949+0000", 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383e-05 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665046+0000", 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6459000000000006e-05 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665259+0000", 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000213141 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.672522+0000", 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072632110000000003 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.697982+0000", 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025459699999999998 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698018+0000", 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6375999999999999e-05 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698166+0000", 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014818700000000001 2024-09-13T07:03:27.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698290+0000", 2024-09-13T07:03:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012333999999999999 2024-09-13T07:03:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138587 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.637359248999999, 2024-09-13T07:03:27.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019224502000000001, 2024-09-13T07:03:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138587 2024-09-13T07:03:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713360+0000", 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713368+0000", 2024-09-13T07:03:27.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713383+0000", 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5176e-05 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713390+0000", 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8669999999999996e-06 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713449+0000", 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8938999999999999e-05 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713549+0000", 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010009 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.716087+0000", 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025383820000000001 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732351+0000", 2024-09-13T07:03:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016263564000000001 2024-09-13T07:03:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732450+0000", 2024-09-13T07:03:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9529000000000002e-05 2024-09-13T07:03:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732532+0000", 2024-09-13T07:03:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1526000000000005e-05 2024-09-13T07:03:27.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732585+0000", 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3819000000000001e-05 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138786 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 18.203143994000001, 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018847452000000001, 2024-09-13T07:03:27.433 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138786 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147575+0000", 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147583+0000", 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147601+0000", 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7961e-05 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147610+0000", 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147661+0000", 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0447e-05 2024-09-13T07:03:27.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147754+0000", 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2974999999999997e-05 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.150580+0000", 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002826566 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166135+0000", 2024-09-13T07:03:27.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015554772 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166225+0000", 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0258000000000002e-05 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166334+0000", 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010816500000000001 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166424+0000", 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0116000000000003e-05 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:139127 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 17.576093029999999, 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019295931999999998, 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.439 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139127 2024-09-13T07:03:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:27.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774626+0000", 2024-09-13T07:03:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774635+0000", 2024-09-13T07:03:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5419999999999994e-06 2024-09-13T07:03:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774654+0000", 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8359000000000002e-05 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774663+0000", 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774718+0000", 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5315000000000001e-05 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774805+0000", 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7284e-05 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.779319+0000", 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045138310000000003 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793772+0000", 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014452692 2024-09-13T07:03:27.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793821+0000", 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9563e-05 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793878+0000", 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6953999999999997e-05 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793923+0000", 2024-09-13T07:03:27.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4781999999999999e-05 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:147368 3.4 3:26bd7659:::rbd_data.119421a2279e.000000000000001d:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:18.583791+0000", 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.7669292720000005, 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.011299273, 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.445 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 147368 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583791+0000", 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583791+0000", 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583788+0000", 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999971 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583803+0000", 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5135e-05 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583832+0000", 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8741000000000001e-05 2024-09-13T07:03:27.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583851+0000", 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8552000000000001e-05 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.583983+0000", 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013209699999999999 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.584149+0000", 2024-09-13T07:03:27.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000166117 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.585549+0000", 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001399682 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.591604+0000", 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.006055146 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:27.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.594934+0000", 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033300309999999998 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.595023+0000", 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8839999999999996e-05 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:18.595090+0000", 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7614999999999996e-05 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:150355 3.0 3:043405b9:::rbd_data.119453884147.0000000000000002:30 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:24.546292+0000", 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 2.8044278600000001, 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0090966889999999998, 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 150355 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:24.546292+0000", 2024-09-13T07:03:27.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:24.546292+0000", 2024-09-13T07:03:27.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:24.546292+0000", 2024-09-13T07:03:27.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:24.546295+0000", 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.326e-06 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:24.546303+0000", 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3679999999999996e-06 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.453 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:24.546312+0000", 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1370000000000008e-06 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:24.555275+0000", 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0089637420000000002 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:24.555338+0000", 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2155000000000004e-05 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.454 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:24.555389+0000", 2024-09-13T07:03:27.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1434000000000003e-05 2024-09-13T07:03:27.455 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.455 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.455 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.455 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.455 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.455 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:27.457 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:27.457 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:27.455+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:27.558 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:27.554+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:27.615 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:27.615 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:27.615 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:27.625 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:27.658 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:27.658+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:27.734 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:27.734 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:27.734 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:27.734 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:27.734 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:27.744 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:27.759 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:27.758+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:27.854 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:27.854 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.732165698000003, 2024-09-13T07:03:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:03:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:03:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.855 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:03:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:03:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:03:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:03:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:03:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:03:27.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:03:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:03:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.732145220999996, 2024-09-13T07:03:27.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:03:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:03:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:03:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:03:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:03:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:03:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:03:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:03:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:03:27.863 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87036 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 6963200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725461155000005, 2024-09-13T07:03:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042468831999999998, 2024-09-13T07:03:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.864 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87036 2024-09-13T07:03:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127962+0000", 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127964+0000", 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7799999999999999e-06 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127967+0000", 2024-09-13T07:03:27.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9730000000000002e-06 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127971+0000", 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4349999999999999e-06 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170202+0000", 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042230940000000002 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170293+0000", 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0858000000000003e-05 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170431+0000", 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013821699999999999 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:27.868 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725448618000001, 2024-09-13T07:03:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:03:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:03:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:27.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:03:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:03:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:03:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:03:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:03:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:03:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:03:27.872 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725436548999994, 2024-09-13T07:03:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:03:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:03:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:03:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:03:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:03:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:03:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:03:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:03:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725424106000006, 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725406508000006, 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:03:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:03:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:03:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:03:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:03:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:03:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:03:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725395805000005, 2024-09-13T07:03:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:03:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:03:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:03:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:03:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:03:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:03:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:03:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725351872000005, 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:03:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:03:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:03:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:03:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:03:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725332085999995, 2024-09-13T07:03:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:03:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:03:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:03:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:03:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:03:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:03:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:03:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:03:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:03:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:03:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:03:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725316336000006, 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:03:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:03:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:03:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:03:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:03:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:03:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725305083999999, 2024-09-13T07:03:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:03:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:03:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:03:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:03:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725294485000006, 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:03:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:03:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:03:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:03:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:03:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:03:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:03:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:03:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725284991999999, 2024-09-13T07:03:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:03:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:03:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:03:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:03:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:03:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:03:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:03:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:03:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:03:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:03:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:03:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:03:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:03:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:03:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725275448000005, 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:03:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:03:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:03:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:03:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:03:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:03:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725242844999997, 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:03:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:03:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:03:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:03:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.725229822000003, 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:03:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:03:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:03:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:03:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:03:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:03:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.721878837000006, 2024-09-13T07:03:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:03:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:03:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:03:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:03:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:03:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:03:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:03:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:03:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:03:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:03:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:03:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:03:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:03:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:03:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.721858560000001, 2024-09-13T07:03:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:03:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:03:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:03:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:03:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.721838359000003, 2024-09-13T07:03:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:03:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:03:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:03:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:03:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:03:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:03:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:03:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:27.946 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:27.947 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:27.948 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:27.943+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:28.048 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:28.046+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:28.060 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:28.060 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:28.061 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:28.061 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:28.071 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:28.140 INFO:teuthology.orchestra.run.smithi118.stdout:leaving image image_client.0-clone47 intact 2024-09-13T07:03:28.148 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:28.146+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:28.179 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:28.179 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:28.179 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:28.179 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:28.179 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:28.189 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:28.249 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:28.246+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:28.298 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:28.298 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:28.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:28.299 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:28.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.299 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.299 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:28.299 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696930668, 2024-09-13T07:03:28.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:03:28.299 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.299 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.300 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:03:28.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.300 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:28.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:28.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:28.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:03:28.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:03:28.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:03:28.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:03:28.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:03:28.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:28.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:03:28.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:03:28.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:03:28.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:03:28.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:03:28.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:03:28.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:03:28.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:03:28.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.305 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.305 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.305 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:28.305 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696899567000003, 2024-09-13T07:03:28.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:03:28.305 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.305 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.306 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:03:28.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.306 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:28.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:28.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:28.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:03:28.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:03:28.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:03:28.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:03:28.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:03:28.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:03:28.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:03:28.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:03:28.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:03:28.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:03:28.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:28.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:03:28.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:03:28.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:03:28.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.310 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.311 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.311 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:28.311 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696877252999997, 2024-09-13T07:03:28.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:03:28.311 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.311 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.311 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:03:28.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.312 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:28.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:28.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:03:28.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:03:28.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:03:28.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:03:28.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:03:28.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:03:28.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:03:28.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:03:28.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:03:28.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:03:28.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:03:28.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:03:28.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:03:28.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:03:28.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:03:28.316 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.316 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.316 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.316 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.316 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:28.317 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696854221999999, 2024-09-13T07:03:28.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:03:28.317 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.317 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.317 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:03:28.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.317 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:28.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:28.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:03:28.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:03:28.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:03:28.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:03:28.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:03:28.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:03:28.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:03:28.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:28.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:03:28.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:03:28.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:03:28.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:03:28.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:03:28.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:03:28.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:03:28.322 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.322 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.322 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.322 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.323 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:28.323 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696829672, 2024-09-13T07:03:28.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:03:28.323 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.323 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.323 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:03:28.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.323 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:28.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:28.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:28.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:03:28.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:03:28.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:03:28.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:03:28.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:03:28.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:03:28.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:03:28.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:03:28.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:03:28.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:03:28.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:03:28.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:03:28.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:03:28.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:03:28.328 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.328 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.328 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.328 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.328 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:28.329 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696793962999998, 2024-09-13T07:03:28.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:03:28.329 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.329 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.329 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:03:28.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.329 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:28.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:28.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:03:28.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:03:28.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:03:28.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:03:28.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:03:28.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:03:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:03:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:03:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:03:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:03:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:03:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:03:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:03:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:03:28.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:03:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696769283999998, 2024-09-13T07:03:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:03:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.334 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.335 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:03:28.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.335 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:28.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:28.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:03:28.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:03:28.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:28.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:03:28.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:03:28.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:03:28.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:03:28.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:03:28.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:03:28.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:03:28.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:03:28.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:03:28.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:03:28.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:03:28.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:28.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.339 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.340 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.340 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:28.340 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696736010000002, 2024-09-13T07:03:28.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:03:28.340 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.340 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.340 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:03:28.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.341 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:28.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:28.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:03:28.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:03:28.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:03:28.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:03:28.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:03:28.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:03:28.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:03:28.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:03:28.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:03:28.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:03:28.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:03:28.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:03:28.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:03:28.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:03:28.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:03:28.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.345 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.345 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.346 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:28.346 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696709222000003, 2024-09-13T07:03:28.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:03:28.346 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.346 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.346 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:03:28.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.346 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:28.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:28.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:03:28.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:28.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:03:28.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:03:28.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:03:28.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:03:28.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:03:28.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:03:28.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:03:28.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:03:28.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:03:28.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:03:28.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:03:28.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:03:28.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:03:28.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:03:28.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.351 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.351 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.351 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:28.351 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696684668000003, 2024-09-13T07:03:28.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:03:28.352 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.352 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.352 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:03:28.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.352 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:28.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:28.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:28.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:03:28.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:03:28.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:03:28.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:03:28.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:03:28.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:03:28.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:03:28.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:03:28.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:03:28.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:03:28.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:03:28.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:03:28.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:03:28.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:03:28.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.357 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.357 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.357 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:28.357 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696654899000002, 2024-09-13T07:03:28.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:03:28.357 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.357 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.358 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.358 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:03:28.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.358 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:28.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:28.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:28.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:03:28.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:03:28.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:03:28.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:03:28.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:03:28.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:03:28.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:03:28.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:03:28.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:03:28.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:03:28.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:03:28.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:03:28.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:03:28.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:03:28.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.362 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.363 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.363 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:28.363 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696595056, 2024-09-13T07:03:28.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:03:28.363 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.363 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.363 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:03:28.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.364 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:28.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:28.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:03:28.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:28.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:03:28.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:28.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:03:28.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:03:28.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:03:28.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:03:28.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:03:28.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:03:28.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:03:28.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:03:28.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:03:28.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:03:28.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:03:28.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:03:28.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.368 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.368 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.368 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:28.369 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696528205999996, 2024-09-13T07:03:28.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:03:28.369 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.369 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.369 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.369 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.369 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.369 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:03:28.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.369 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:28.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:28.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:03:28.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:03:28.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:03:28.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:03:28.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:03:28.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:03:28.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:03:28.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:03:28.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:03:28.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:03:28.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:03:28.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:03:28.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:03:28.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:03:28.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:03:28.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.374 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.374 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.374 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.374 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:28.374 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696495704, 2024-09-13T07:03:28.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:03:28.374 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.374 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.374 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.375 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:03:28.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.375 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:28.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:28.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:28.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:03:28.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:03:28.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:03:28.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:03:28.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:03:28.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:03:28.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:03:28.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:03:28.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:03:28.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:03:28.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:03:28.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:03:28.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:03:28.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:03:28.379 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.379 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.379 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.380 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.380 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:28.380 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696461083000003, 2024-09-13T07:03:28.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:03:28.380 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.380 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.380 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.380 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.380 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.380 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:03:28.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.381 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:28.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:28.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:03:28.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:03:28.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:03:28.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:03:28.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:03:28.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:03:28.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:03:28.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:03:28.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:03:28.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:03:28.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:03:28.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:03:28.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:03:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:03:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:03:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.385 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696410852, 2024-09-13T07:03:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:03:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:03:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.386 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:03:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:03:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:03:28.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:03:28.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:03:28.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:03:28.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:03:28.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:03:28.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:03:28.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:03:28.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:03:28.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:03:28.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:03:28.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:03:28.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.391 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.392 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.392 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:28.392 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696362665999999, 2024-09-13T07:03:28.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:03:28.392 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.392 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.392 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:03:28.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.393 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:28.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:28.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:28.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:03:28.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:03:28.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:03:28.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:03:28.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:03:28.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:03:28.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:03:28.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:03:28.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:03:28.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:03:28.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:03:28.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:03:28.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:03:28.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:03:28.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.397 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.397 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.397 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:28.397 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696336027999997, 2024-09-13T07:03:28.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:03:28.398 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.398 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.398 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:03:28.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.398 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:28.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:28.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:03:28.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:03:28.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:03:28.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:03:28.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:03:28.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:03:28.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:03:28.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:03:28.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:03:28.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:03:28.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:03:28.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:03:28.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:03:28.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:03:28.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:03:28.402 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.403 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.403 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.403 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.403 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:28.403 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.696289192999998, 2024-09-13T07:03:28.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:03:28.403 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.403 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.403 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.404 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.404 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.404 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:03:28.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.404 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:28.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:28.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:03:28.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:03:28.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:03:28.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:03:28.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:03:28.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:03:28.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:03:28.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:03:28.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:03:28.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:03:28.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:03:28.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:03:28.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:03:28.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:03:28.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:28.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.408 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.408 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:28.409 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:28.409 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.692934776000001, 2024-09-13T07:03:28.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:03:28.409 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.409 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.409 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:03:28.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.409 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:28.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:28.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:28.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:03:28.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:03:28.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:03:28.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:03:28.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:03:28.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:03:28.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:03:28.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:03:28.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:03:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:03:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:03:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:03:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:03:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:03:28.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.414 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.414 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.414 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.414 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.414 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:28.415 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:03:28.416 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:28.411+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:28.516 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:28.514+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:28.557 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-09-13T07:03:28.587 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:28.588 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:28.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.588 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159361 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.588 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577772+0000", 2024-09-13T07:03:28.588 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0021712120000000001, 2024-09-13T07:03:28.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021934699999999999, 2024-09-13T07:03:28.588 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.588 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.589 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159361 2024-09-13T07:03:28.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.589 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577772+0000", 2024-09-13T07:03:28.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577772+0000", 2024-09-13T07:03:28.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577771+0000", 2024-09-13T07:03:28.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577774+0000", 2024-09-13T07:03:28.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7489999999999999e-06 2024-09-13T07:03:28.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577780+0000", 2024-09-13T07:03:28.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6949999999999997e-06 2024-09-13T07:03:28.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577785+0000", 2024-09-13T07:03:28.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2229999999999998e-06 2024-09-13T07:03:28.592 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.592 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.592 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.592 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159491 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.592 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578560+0000", 2024-09-13T07:03:28.592 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0013831710000000001, 2024-09-13T07:03:28.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001465025, 2024-09-13T07:03:28.593 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.593 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.593 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159491 2024-09-13T07:03:28.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.593 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578560+0000", 2024-09-13T07:03:28.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578560+0000", 2024-09-13T07:03:28.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578559+0000", 2024-09-13T07:03:28.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578562+0000", 2024-09-13T07:03:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-13T07:03:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578568+0000", 2024-09-13T07:03:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2090000000000002e-06 2024-09-13T07:03:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578574+0000", 2024-09-13T07:03:28.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1869999999999996e-06 2024-09-13T07:03:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159585 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579599+0000", 2024-09-13T07:03:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00034402, 2024-09-13T07:03:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047767099999999999, 2024-09-13T07:03:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.598 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159585 2024-09-13T07:03:28.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.598 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579599+0000", 2024-09-13T07:03:28.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579599+0000", 2024-09-13T07:03:28.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579599+0000", 2024-09-13T07:03:28.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579601+0000", 2024-09-13T07:03:28.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.554e-06 2024-09-13T07:03:28.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579607+0000", 2024-09-13T07:03:28.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-13T07:03:28.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579614+0000", 2024-09-13T07:03:28.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3159999999999999e-06 2024-09-13T07:03:28.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.601 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.601 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159362 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.601 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577790+0000", 2024-09-13T07:03:28.601 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0021530989999999999, 2024-09-13T07:03:28.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023387479999999999, 2024-09-13T07:03:28.602 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.602 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.602 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159362 2024-09-13T07:03:28.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.602 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577790+0000", 2024-09-13T07:03:28.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577790+0000", 2024-09-13T07:03:28.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577789+0000", 2024-09-13T07:03:28.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577792+0000", 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9399999999999998e-06 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577798+0000", 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8549999999999998e-06 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577803+0000", 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9629999999999997e-06 2024-09-13T07:03:28.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.606 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.606 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.606 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159492 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.606 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578579+0000", 2024-09-13T07:03:28.606 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0013635570000000001, 2024-09-13T07:03:28.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016079449999999999, 2024-09-13T07:03:28.606 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.606 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159492 2024-09-13T07:03:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578579+0000", 2024-09-13T07:03:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578579+0000", 2024-09-13T07:03:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578579+0000", 2024-09-13T07:03:28.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578582+0000", 2024-09-13T07:03:28.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4339999999999999e-06 2024-09-13T07:03:28.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578587+0000", 2024-09-13T07:03:28.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6949999999999997e-06 2024-09-13T07:03:28.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578592+0000", 2024-09-13T07:03:28.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-13T07:03:28.609 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.609 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.609 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.610 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159587 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.610 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579618+0000", 2024-09-13T07:03:28.610 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00032463799999999998, 2024-09-13T07:03:28.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00062315899999999997, 2024-09-13T07:03:28.610 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.610 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.611 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159587 2024-09-13T07:03:28.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.611 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579618+0000", 2024-09-13T07:03:28.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579618+0000", 2024-09-13T07:03:28.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579618+0000", 2024-09-13T07:03:28.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579621+0000", 2024-09-13T07:03:28.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6220000000000001e-06 2024-09-13T07:03:28.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579626+0000", 2024-09-13T07:03:28.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-06 2024-09-13T07:03:28.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579631+0000", 2024-09-13T07:03:28.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9910000000000002e-06 2024-09-13T07:03:28.613 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.613 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.613 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.613 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159363 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.613 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577807+0000", 2024-09-13T07:03:28.614 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0021356119999999998, 2024-09-13T07:03:28.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024902489999999999, 2024-09-13T07:03:28.614 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.614 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.614 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159363 2024-09-13T07:03:28.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.614 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577807+0000", 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577807+0000", 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577807+0000", 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577810+0000", 2024-09-13T07:03:28.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7089999999999998e-06 2024-09-13T07:03:28.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577816+0000", 2024-09-13T07:03:28.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1179999999999997e-06 2024-09-13T07:03:28.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577821+0000", 2024-09-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6970000000000001e-06 2024-09-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159493 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578596+0000", 2024-09-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001346603, 2024-09-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017537710000000001, 2024-09-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159493 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578596+0000", 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578596+0000", 2024-09-13T07:03:28.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578596+0000", 2024-09-13T07:03:28.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578599+0000", 2024-09-13T07:03:28.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-13T07:03:28.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578605+0000", 2024-09-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3929999999999996e-06 2024-09-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578610+0000", 2024-09-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0730000000000004e-06 2024-09-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159589 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579644+0000", 2024-09-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00029876000000000002, 2024-09-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075641200000000003, 2024-09-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159589 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579644+0000", 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579644+0000", 2024-09-13T07:03:28.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579644+0000", 2024-09-13T07:03:28.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579647+0000", 2024-09-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9059999999999998e-06 2024-09-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579652+0000", 2024-09-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5990000000000001e-06 2024-09-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579657+0000", 2024-09-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0950000000000003e-06 2024-09-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159364 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577825+0000", 2024-09-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0021182470000000002, 2024-09-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026340370000000001, 2024-09-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159364 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577825+0000", 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577825+0000", 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577824+0000", 2024-09-13T07:03:28.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577827+0000", 2024-09-13T07:03:28.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5639999999999999e-06 2024-09-13T07:03:28.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577833+0000", 2024-09-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3350000000000003e-06 2024-09-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577838+0000", 2024-09-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1630000000000004e-06 2024-09-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159494 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578618+0000", 2024-09-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0013245609999999999, 2024-09-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001889389, 2024-09-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159494 2024-09-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578618+0000", 2024-09-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578618+0000", 2024-09-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578618+0000", 2024-09-13T07:03:28.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578621+0000", 2024-09-13T07:03:28.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7350000000000001e-06 2024-09-13T07:03:28.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578627+0000", 2024-09-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8710000000000002e-06 2024-09-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578632+0000", 2024-09-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2719999999999997e-06 2024-09-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.633 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159593 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.633 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579661+0000", 2024-09-13T07:03:28.633 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00028155299999999998, 2024-09-13T07:03:28.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089671100000000001, 2024-09-13T07:03:28.633 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.633 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.633 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.633 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.633 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159593 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579661+0000", 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579661+0000", 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579661+0000", 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579664+0000", 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5119999999999998e-06 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579669+0000", 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.84e-06 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579674+0000", 2024-09-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9309999999999999e-06 2024-09-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159365 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577843+0000", 2024-09-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.002100106, 2024-09-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027688909999999999, 2024-09-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159365 2024-09-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577843+0000", 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577843+0000", 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577843+0000", 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577845+0000", 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.621e-06 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577850+0000", 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3179999999999997e-06 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577855+0000", 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8749999999999999e-06 2024-09-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159495 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578636+0000", 2024-09-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001306544, 2024-09-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020253189999999998, 2024-09-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159495 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578636+0000", 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578636+0000", 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578636+0000", 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578638+0000", 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.322e-06 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578644+0000", 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4670000000000003e-06 2024-09-13T07:03:28.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578649+0000", 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6890000000000003e-06 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159596 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579679+0000", 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00026434499999999998, 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010325569999999999, 2024-09-13T07:03:28.643 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159596 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579679+0000", 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579679+0000", 2024-09-13T07:03:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579678+0000", 2024-09-13T07:03:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579681+0000", 2024-09-13T07:03:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9170000000000002e-06 2024-09-13T07:03:28.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579687+0000", 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7030000000000003e-06 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579692+0000", 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0560000000000004e-06 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.646 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159367 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577859+0000", 2024-09-13T07:03:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0020836019999999999, 2024-09-13T07:03:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002906021, 2024-09-13T07:03:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159367 2024-09-13T07:03:28.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577859+0000", 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577859+0000", 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577859+0000", 2024-09-13T07:03:28.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577862+0000", 2024-09-13T07:03:28.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6589999999999999e-06 2024-09-13T07:03:28.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577867+0000", 2024-09-13T07:03:28.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7250000000000002e-06 2024-09-13T07:03:28.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577874+0000", 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1380000000000004e-06 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159496 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578652+0000", 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0012905010000000001, 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021626079999999999, 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.650 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.651 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.651 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.651 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159496 2024-09-13T07:03:28.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.651 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578652+0000", 2024-09-13T07:03:28.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578652+0000", 2024-09-13T07:03:28.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578652+0000", 2024-09-13T07:03:28.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578655+0000", 2024-09-13T07:03:28.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3980000000000002e-06 2024-09-13T07:03:28.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578660+0000", 2024-09-13T07:03:28.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4070000000000001e-06 2024-09-13T07:03:28.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578665+0000", 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7210000000000002e-06 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159598 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579696+0000", 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00024681200000000001, 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001169504, 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.655 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.655 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159598 2024-09-13T07:03:28.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.655 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579696+0000", 2024-09-13T07:03:28.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579696+0000", 2024-09-13T07:03:28.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579696+0000", 2024-09-13T07:03:28.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579698+0000", 2024-09-13T07:03:28.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4679999999999999e-06 2024-09-13T07:03:28.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579704+0000", 2024-09-13T07:03:28.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4539999999999998e-06 2024-09-13T07:03:28.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579708+0000", 2024-09-13T07:03:28.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6879999999999998e-06 2024-09-13T07:03:28.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.657 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.658 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.658 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159368 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.658 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577878+0000", 2024-09-13T07:03:28.658 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.002065145, 2024-09-13T07:03:28.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030478469999999998, 2024-09-13T07:03:28.658 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.658 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.658 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159368 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577878+0000", 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577878+0000", 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577877+0000", 2024-09-13T07:03:28.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577881+0000", 2024-09-13T07:03:28.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1609999999999999e-06 2024-09-13T07:03:28.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577887+0000", 2024-09-13T07:03:28.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0800000000000002e-06 2024-09-13T07:03:28.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577892+0000", 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7929999999999997e-06 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159497 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578668+0000", 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0012746140000000001, 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023089740000000001, 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.661 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.662 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159497 2024-09-13T07:03:28.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.662 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578668+0000", 2024-09-13T07:03:28.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578668+0000", 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578668+0000", 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578670+0000", 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2740000000000002e-06 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578675+0000", 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1039999999999998e-06 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578680+0000", 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5190000000000003e-06 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.664 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159599 3.16 3.233fd676 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.665 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579713+0000", 2024-09-13T07:03:28.665 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.000229929, 2024-09-13T07:03:28.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001313643, 2024-09-13T07:03:28.665 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.665 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.665 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159599 2024-09-13T07:03:28.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.665 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579713+0000", 2024-09-13T07:03:28.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579713+0000", 2024-09-13T07:03:28.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579713+0000", 2024-09-13T07:03:28.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579715+0000", 2024-09-13T07:03:28.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6749999999999998e-06 2024-09-13T07:03:28.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579721+0000", 2024-09-13T07:03:28.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6439999999999997e-06 2024-09-13T07:03:28.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579726+0000", 2024-09-13T07:03:28.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0000000000000004e-06 2024-09-13T07:03:28.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.668 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.669 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.669 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159369 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.669 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577896+0000", 2024-09-13T07:03:28.669 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0020473689999999998, 2024-09-13T07:03:28.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031839260000000001, 2024-09-13T07:03:28.670 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.670 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.670 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.670 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.670 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.670 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159369 2024-09-13T07:03:28.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.670 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577896+0000", 2024-09-13T07:03:28.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577896+0000", 2024-09-13T07:03:28.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577895+0000", 2024-09-13T07:03:28.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577898+0000", 2024-09-13T07:03:28.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.43e-06 2024-09-13T07:03:28.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577903+0000", 2024-09-13T07:03:28.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0100000000000003e-06 2024-09-13T07:03:28.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577907+0000", 2024-09-13T07:03:28.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5170000000000001e-06 2024-09-13T07:03:28.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.675 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.676 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159498 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.676 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578684+0000", 2024-09-13T07:03:28.676 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001259015, 2024-09-13T07:03:28.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024455620000000001, 2024-09-13T07:03:28.677 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.677 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.677 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.677 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.677 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.677 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159498 2024-09-13T07:03:28.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.678 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578684+0000", 2024-09-13T07:03:28.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578684+0000", 2024-09-13T07:03:28.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578684+0000", 2024-09-13T07:03:28.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578686+0000", 2024-09-13T07:03:28.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5720000000000001e-06 2024-09-13T07:03:28.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578692+0000", 2024-09-13T07:03:28.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7200000000000003e-06 2024-09-13T07:03:28.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578697+0000", 2024-09-13T07:03:28.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8060000000000002e-06 2024-09-13T07:03:28.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.683 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.684 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159600 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.684 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579730+0000", 2024-09-13T07:03:28.684 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.000212709, 2024-09-13T07:03:28.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014521270000000001, 2024-09-13T07:03:28.684 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.685 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.685 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.685 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.685 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.685 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159600 2024-09-13T07:03:28.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.685 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579730+0000", 2024-09-13T07:03:28.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579730+0000", 2024-09-13T07:03:28.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579730+0000", 2024-09-13T07:03:28.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579738+0000", 2024-09-13T07:03:28.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5059999999999995e-06 2024-09-13T07:03:28.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579745+0000", 2024-09-13T07:03:28.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.348e-06 2024-09-13T07:03:28.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579750+0000", 2024-09-13T07:03:28.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4879999999999998e-06 2024-09-13T07:03:28.690 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.691 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.691 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159370 3.1d 3.f20203bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.691 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577911+0000", 2024-09-13T07:03:28.692 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0020316890000000002, 2024-09-13T07:03:28.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033263849999999998, 2024-09-13T07:03:28.692 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.692 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.693 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159370 2024-09-13T07:03:28.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.693 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577911+0000", 2024-09-13T07:03:28.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577911+0000", 2024-09-13T07:03:28.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577911+0000", 2024-09-13T07:03:28.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577914+0000", 2024-09-13T07:03:28.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:03:28.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577919+0000", 2024-09-13T07:03:28.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8760000000000002e-06 2024-09-13T07:03:28.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577924+0000", 2024-09-13T07:03:28.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.758e-06 2024-09-13T07:03:28.698 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.698 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.698 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.699 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159499 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.699 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578701+0000", 2024-09-13T07:03:28.699 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0012422939999999999, 2024-09-13T07:03:28.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025880970000000001, 2024-09-13T07:03:28.699 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.699 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.700 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159499 2024-09-13T07:03:28.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.700 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578701+0000", 2024-09-13T07:03:28.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578701+0000", 2024-09-13T07:03:28.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578700+0000", 2024-09-13T07:03:28.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578703+0000", 2024-09-13T07:03:28.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.491e-06 2024-09-13T07:03:28.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578708+0000", 2024-09-13T07:03:28.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4299999999999997e-06 2024-09-13T07:03:28.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578713+0000", 2024-09-13T07:03:28.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1080000000000001e-06 2024-09-13T07:03:28.705 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.705 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.706 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159603 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.706 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579803+0000", 2024-09-13T07:03:28.706 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00014049099999999999, 2024-09-13T07:03:28.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001537329, 2024-09-13T07:03:28.707 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.707 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.708 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159603 2024-09-13T07:03:28.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.708 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579803+0000", 2024-09-13T07:03:28.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579803+0000", 2024-09-13T07:03:28.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579802+0000", 2024-09-13T07:03:28.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:28.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579806+0000", 2024-09-13T07:03:28.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2180000000000001e-06 2024-09-13T07:03:28.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579815+0000", 2024-09-13T07:03:28.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9600000000000006e-06 2024-09-13T07:03:28.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579821+0000", 2024-09-13T07:03:28.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6810000000000003e-06 2024-09-13T07:03:28.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.713 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.714 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159371 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.714 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577928+0000", 2024-09-13T07:03:28.714 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.002014872, 2024-09-13T07:03:28.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034667740000000002, 2024-09-13T07:03:28.715 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.715 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.715 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.715 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.715 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.715 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159371 2024-09-13T07:03:28.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.716 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577928+0000", 2024-09-13T07:03:28.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577928+0000", 2024-09-13T07:03:28.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577928+0000", 2024-09-13T07:03:28.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577930+0000", 2024-09-13T07:03:28.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3819999999999998e-06 2024-09-13T07:03:28.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577936+0000", 2024-09-13T07:03:28.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5029999999999997e-06 2024-09-13T07:03:28.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577940+0000", 2024-09-13T07:03:28.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.707e-06 2024-09-13T07:03:28.721 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.721 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.721 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.722 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159500 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.722 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578718+0000", 2024-09-13T07:03:28.722 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0012253819999999999, 2024-09-13T07:03:28.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002727957, 2024-09-13T07:03:28.722 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.722 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.723 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159500 2024-09-13T07:03:28.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.723 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578718+0000", 2024-09-13T07:03:28.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578718+0000", 2024-09-13T07:03:28.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578717+0000", 2024-09-13T07:03:28.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578720+0000", 2024-09-13T07:03:28.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6390000000000001e-06 2024-09-13T07:03:28.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578726+0000", 2024-09-13T07:03:28.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0290000000000002e-06 2024-09-13T07:03:28.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578731+0000", 2024-09-13T07:03:28.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.994e-06 2024-09-13T07:03:28.728 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.729 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.729 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.729 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159606 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.729 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579825+0000", 2024-09-13T07:03:28.729 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.000118108, 2024-09-13T07:03:28.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016688289999999999, 2024-09-13T07:03:28.729 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.729 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.729 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159606 2024-09-13T07:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579825+0000", 2024-09-13T07:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579825+0000", 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579825+0000", 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579827+0000", 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6840000000000001e-06 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579832+0000", 2024-09-13T07:03:28.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2009999999999998e-06 2024-09-13T07:03:28.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579837+0000", 2024-09-13T07:03:28.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:03:28.732 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.732 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.732 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.732 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159372 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.733 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577944+0000", 2024-09-13T07:03:28.733 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0019987820000000002, 2024-09-13T07:03:28.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003599625, 2024-09-13T07:03:28.733 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.733 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.733 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.733 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.733 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.733 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159372 2024-09-13T07:03:28.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.733 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577944+0000", 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577944+0000", 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577944+0000", 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577958+0000", 2024-09-13T07:03:28.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3833e-05 2024-09-13T07:03:28.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577964+0000", 2024-09-13T07:03:28.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.122e-06 2024-09-13T07:03:28.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577969+0000", 2024-09-13T07:03:28.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6890000000000001e-06 2024-09-13T07:03:28.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.736 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.736 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159501 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.736 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578735+0000", 2024-09-13T07:03:28.736 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00120758, 2024-09-13T07:03:28.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028581180000000002, 2024-09-13T07:03:28.736 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.736 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159501 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578735+0000", 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578735+0000", 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578735+0000", 2024-09-13T07:03:28.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578738+0000", 2024-09-13T07:03:28.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6089999999999999e-06 2024-09-13T07:03:28.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578744+0000", 2024-09-13T07:03:28.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578749+0000", 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2839999999999997e-06 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159609 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579841+0000", 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.000102205, 2024-09-13T07:03:28.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001804197, 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159609 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579841+0000", 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579841+0000", 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579840+0000", 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579843+0000", 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6429999999999999e-06 2024-09-13T07:03:28.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579848+0000", 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0180000000000001e-06 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579853+0000", 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7040000000000002e-06 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159373 3.16 3.42ea76f6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577974+0000", 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001968858, 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003725866, 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159373 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577974+0000", 2024-09-13T07:03:28.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577974+0000", 2024-09-13T07:03:28.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577974+0000", 2024-09-13T07:03:28.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577977+0000", 2024-09-13T07:03:28.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9299999999999999e-06 2024-09-13T07:03:28.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577983+0000", 2024-09-13T07:03:28.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8939999999999998e-06 2024-09-13T07:03:28.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577992+0000", 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5950000000000005e-06 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159502 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578753+0000", 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001189985, 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00299717, 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159502 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578753+0000", 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578753+0000", 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578753+0000", 2024-09-13T07:03:28.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578755+0000", 2024-09-13T07:03:28.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:03:28.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578761+0000", 2024-09-13T07:03:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.032e-06 2024-09-13T07:03:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578770+0000", 2024-09-13T07:03:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8470000000000002e-06 2024-09-13T07:03:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.749 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159610 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579857+0000", 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6416999999999996e-05, 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019930519999999999, 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159610 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579857+0000", 2024-09-13T07:03:28.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579857+0000", 2024-09-13T07:03:28.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579856+0000", 2024-09-13T07:03:28.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579859+0000", 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7099999999999999e-06 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579864+0000", 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0769999999999998e-06 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579875+0000", 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:03:28.752 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159376 3.c 3.f0cb4f2c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577997+0000", 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001946143, 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039076579999999996, 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.753 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159376 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577997+0000", 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577997+0000", 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577996+0000", 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577999+0000", 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7520000000000001e-06 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578005+0000", 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8490000000000002e-06 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578010+0000", 2024-09-13T07:03:28.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9969999999999998e-06 2024-09-13T07:03:28.756 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.756 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.756 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.756 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159503 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.756 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579045+0000", 2024-09-13T07:03:28.756 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00089756000000000005, 2024-09-13T07:03:28.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029138200000000001, 2024-09-13T07:03:28.756 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159503 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579045+0000", 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579045+0000", 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579045+0000", 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579050+0000", 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6720000000000001e-06 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579061+0000", 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1066e-05 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579069+0000", 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6140000000000003e-06 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159611 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.759 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579880+0000", 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 6.2980999999999999e-05, 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021305249999999999, 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159611 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579880+0000", 2024-09-13T07:03:28.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579880+0000", 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579879+0000", 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579883+0000", 2024-09-13T07:03:28.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4070000000000002e-06 2024-09-13T07:03:28.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579889+0000", 2024-09-13T07:03:28.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6209999999999998e-06 2024-09-13T07:03:28.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579895+0000", 2024-09-13T07:03:28.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7540000000000003e-06 2024-09-13T07:03:28.762 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.762 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159381 3.1d 3.b1216bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578023+0000", 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0019203989999999999, 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040477519999999999, 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159381 2024-09-13T07:03:28.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.764 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578023+0000", 2024-09-13T07:03:28.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578023+0000", 2024-09-13T07:03:28.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578022+0000", 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578025+0000", 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0730000000000001e-06 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578032+0000", 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0650000000000001e-06 2024-09-13T07:03:28.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578038+0000", 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3669999999999996e-06 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159505 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579074+0000", 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00086884500000000003, 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030444949999999999, 2024-09-13T07:03:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.767 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.767 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159505 2024-09-13T07:03:28.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.767 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579074+0000", 2024-09-13T07:03:28.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579074+0000", 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579074+0000", 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579076+0000", 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.711e-06 2024-09-13T07:03:28.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579082+0000", 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7640000000000002e-06 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579087+0000", 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8359999999999999e-06 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159612 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579899+0000", 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4289000000000002e-05, 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022493460000000002, 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159612 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579899+0000", 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579899+0000", 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579898+0000", 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579900+0000", 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.029e-06 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579905+0000", 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5249999999999999e-06 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579911+0000", 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6620000000000002e-06 2024-09-13T07:03:28.772 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159401 3.5 3.93665ee5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578061+0000", 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001881948, 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041197769999999998, 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.773 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159401 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578061+0000", 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578061+0000", 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578061+0000", 2024-09-13T07:03:28.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578064+0000", 2024-09-13T07:03:28.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2650000000000001e-06 2024-09-13T07:03:28.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578071+0000", 2024-09-13T07:03:28.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8299999999999998e-06 2024-09-13T07:03:28.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578075+0000", 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159506 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579091+0000", 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00085169499999999997, 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031269639999999999, 2024-09-13T07:03:28.776 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.777 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.777 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159506 2024-09-13T07:03:28.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.777 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579091+0000", 2024-09-13T07:03:28.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579091+0000", 2024-09-13T07:03:28.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579091+0000", 2024-09-13T07:03:28.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579093+0000", 2024-09-13T07:03:28.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2589999999999999e-06 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579098+0000", 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2020000000000003e-06 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579104+0000", 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.784e-06 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.780 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159613 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.780 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579914+0000", 2024-09-13T07:03:28.780 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.8600000000000001e-05, 2024-09-13T07:03:28.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002333613, 2024-09-13T07:03:28.780 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.780 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.780 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159613 2024-09-13T07:03:28.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.780 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579914+0000", 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579914+0000", 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579914+0000", 2024-09-13T07:03:28.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579916+0000", 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0470000000000001e-06 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579921+0000", 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.673e-06 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579925+0000", 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159507 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579108+0000", 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00083459099999999998, 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031737279999999998, 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.784 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.784 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159507 2024-09-13T07:03:28.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.784 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579108+0000", 2024-09-13T07:03:28.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579108+0000", 2024-09-13T07:03:28.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579108+0000", 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579131+0000", 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2810999999999999e-05 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579139+0000", 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5779999999999999e-06 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579145+0000", 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7640000000000002e-06 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159614 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579929+0000", 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.3776e-05, 2024-09-13T07:03:28.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023891429999999998, 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159614 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579929+0000", 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579929+0000", 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579929+0000", 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579931+0000", 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5689999999999999e-06 2024-09-13T07:03:28.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579937+0000", 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1839999999999998e-06 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579941+0000", 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5739999999999997e-06 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159415 3.c 3.b6612c0c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578108+0000", 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0018349340000000001, 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042427319999999999, 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159415 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578108+0000", 2024-09-13T07:03:28.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578108+0000", 2024-09-13T07:03:28.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578108+0000", 2024-09-13T07:03:28.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578110+0000", 2024-09-13T07:03:28.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6850000000000001e-06 2024-09-13T07:03:28.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578116+0000", 2024-09-13T07:03:28.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9789999999999998e-06 2024-09-13T07:03:28.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578121+0000", 2024-09-13T07:03:28.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7770000000000002e-06 2024-09-13T07:03:28.793 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.793 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.793 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.793 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159509 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.793 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579151+0000", 2024-09-13T07:03:28.793 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00079248999999999997, 2024-09-13T07:03:28.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032364719999999998, 2024-09-13T07:03:28.793 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.793 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159509 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579151+0000", 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579151+0000", 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579150+0000", 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579153+0000", 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579158+0000", 2024-09-13T07:03:28.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:03:28.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579163+0000", 2024-09-13T07:03:28.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9960000000000001e-06 2024-09-13T07:03:28.796 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.796 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.796 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.796 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159615 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.796 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579945+0000", 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9999986, 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024683769999999999, 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159615 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579945+0000", 2024-09-13T07:03:28.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579945+0000", 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579944+0000", 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579947+0000", 2024-09-13T07:03:28.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4820000000000001e-06 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579952+0000", 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.135e-06 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579956+0000", 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2910000000000001e-06 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.800 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159419 3.c 3.87cb7d2c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.800 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578125+0000", 2024-09-13T07:03:28.800 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001817976, 2024-09-13T07:03:28.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043231190000000003, 2024-09-13T07:03:28.800 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.800 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.800 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.800 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.800 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.800 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159419 2024-09-13T07:03:28.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.800 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578125+0000", 2024-09-13T07:03:28.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578125+0000", 2024-09-13T07:03:28.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578125+0000", 2024-09-13T07:03:28.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578127+0000", 2024-09-13T07:03:28.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3649999999999998e-06 2024-09-13T07:03:28.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578132+0000", 2024-09-13T07:03:28.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0089999999999998e-06 2024-09-13T07:03:28.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578138+0000", 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1650000000000003e-06 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159510 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579169+0000", 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00077425300000000003, 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033136110000000002, 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.803 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159510 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579169+0000", 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579169+0000", 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579168+0000", 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579171+0000", 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.954e-06 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579179+0000", 2024-09-13T07:03:28.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9060000000000002e-06 2024-09-13T07:03:28.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579184+0000", 2024-09-13T07:03:28.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:03:28.806 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.806 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.806 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.806 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159618 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579959+0000", 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9999838, 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00255404, 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159618 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579959+0000", 2024-09-13T07:03:28.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579959+0000", 2024-09-13T07:03:28.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579959+0000", 2024-09-13T07:03:28.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579961+0000", 2024-09-13T07:03:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1600000000000001e-06 2024-09-13T07:03:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579966+0000", 2024-09-13T07:03:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:03:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579971+0000", 2024-09-13T07:03:28.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3270000000000002e-06 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159343 3.16 3:6bd96f03:::rbd_data.1194eff4793a.0000000000000001:31 [read 700416~4096 out=4096b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577411+0000", 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.002532014, 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051398909999999997, 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159343 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577411+0000", 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577411+0000", 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577411+0000", 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577430+0000", 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9029e-05 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577437+0000", 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6000000000000001e-06 2024-09-13T07:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577443+0000", 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9359999999999996e-06 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.582230+0000", 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047867559999999997 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.582395+0000", 2024-09-13T07:03:28.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016514000000000001 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159428 3.2 3:4304edb7:::rbd_data.119485e97dc3.0000000000000056:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578143+0000", 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0018004150000000001, 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044219580000000001, 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.814 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159428 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578143+0000", 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578143+0000", 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578142+0000", 2024-09-13T07:03:28.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578145+0000", 2024-09-13T07:03:28.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.615e-06 2024-09-13T07:03:28.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578150+0000", 2024-09-13T07:03:28.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6690000000000003e-06 2024-09-13T07:03:28.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578155+0000", 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9960000000000001e-06 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.582495+0000", 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043399429999999998 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.582565+0000", 2024-09-13T07:03:28.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9109000000000005e-05 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159511 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579188+0000", 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00075464499999999997, 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034606659999999998, 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159511 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579188+0000", 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579188+0000", 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579188+0000", 2024-09-13T07:03:28.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579191+0000", 2024-09-13T07:03:28.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5550000000000001e-06 2024-09-13T07:03:28.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579197+0000", 2024-09-13T07:03:28.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9649999999999996e-06 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579202+0000", 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2179999999999998e-06 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159620 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579975+0000", 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9999681, 2024-09-13T07:03:28.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027281440000000001, 2024-09-13T07:03:28.822 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.822 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.822 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159620 2024-09-13T07:03:28.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.822 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579975+0000", 2024-09-13T07:03:28.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579975+0000", 2024-09-13T07:03:28.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579975+0000", 2024-09-13T07:03:28.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579977+0000", 2024-09-13T07:03:28.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.305e-06 2024-09-13T07:03:28.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579982+0000", 2024-09-13T07:03:28.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0839999999999999e-06 2024-09-13T07:03:28.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579987+0000", 2024-09-13T07:03:28.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7520000000000004e-06 2024-09-13T07:03:28.824 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.824 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.825 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159344 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.825 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577449+0000", 2024-09-13T07:03:28.825 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0024944709999999998, 2024-09-13T07:03:28.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053114870000000002, 2024-09-13T07:03:28.825 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.825 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.825 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.825 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.825 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.825 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159344 2024-09-13T07:03:28.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.826 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577449+0000", 2024-09-13T07:03:28.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577449+0000", 2024-09-13T07:03:28.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577448+0000", 2024-09-13T07:03:28.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577451+0000", 2024-09-13T07:03:28.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0989999999999999e-06 2024-09-13T07:03:28.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577457+0000", 2024-09-13T07:03:28.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1120000000000001e-06 2024-09-13T07:03:28.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577462+0000", 2024-09-13T07:03:28.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:03:28.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.828 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.828 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159443 3.5 3.5b07b465 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.828 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578167+0000", 2024-09-13T07:03:28.829 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0017764040000000001, 2024-09-13T07:03:28.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046479399999999997, 2024-09-13T07:03:28.829 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.829 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.829 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159443 2024-09-13T07:03:28.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.829 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578167+0000", 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578167+0000", 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578166+0000", 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578169+0000", 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0479999999999999e-06 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578176+0000", 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4819999999999999e-06 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578181+0000", 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.95e-06 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.831 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159513 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579206+0000", 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00073671999999999995, 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036589719999999999, 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159513 2024-09-13T07:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579206+0000", 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579206+0000", 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579206+0000", 2024-09-13T07:03:28.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579209+0000", 2024-09-13T07:03:28.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9340000000000002e-06 2024-09-13T07:03:28.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579216+0000", 2024-09-13T07:03:28.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4490000000000002e-06 2024-09-13T07:03:28.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579221+0000", 2024-09-13T07:03:28.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0749999999999997e-06 2024-09-13T07:03:28.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.835 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.835 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159623 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.835 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579991+0000", 2024-09-13T07:03:28.835 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9999523, 2024-09-13T07:03:28.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029279409999999999, 2024-09-13T07:03:28.836 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.836 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.836 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159623 2024-09-13T07:03:28.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.836 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579991+0000", 2024-09-13T07:03:28.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579991+0000", 2024-09-13T07:03:28.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579990+0000", 2024-09-13T07:03:28.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579998+0000", 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0840000000000003e-06 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580004+0000", 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0129999999999999e-06 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580009+0000", 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0649999999999998e-06 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.838 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159346 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577466+0000", 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.002476565, 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055142230000000004, 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159346 2024-09-13T07:03:28.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.840 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577466+0000", 2024-09-13T07:03:28.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577466+0000", 2024-09-13T07:03:28.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577466+0000", 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577469+0000", 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577475+0000", 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1500000000000004e-06 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577480+0000", 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8940000000000001e-06 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159448 3.5 3.8ee17c25 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578185+0000", 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0017578380000000001, 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048466280000000004, 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.842 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159448 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578185+0000", 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578185+0000", 2024-09-13T07:03:28.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578185+0000", 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578187+0000", 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6510000000000001e-06 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578193+0000", 2024-09-13T07:03:28.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6559999999999997e-06 2024-09-13T07:03:28.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578198+0000", 2024-09-13T07:03:28.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9010000000000002e-06 2024-09-13T07:03:28.845 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.845 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.845 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.845 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159514 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.845 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579226+0000", 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00071722699999999997, 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038600549999999998, 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159514 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579226+0000", 2024-09-13T07:03:28.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579226+0000", 2024-09-13T07:03:28.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579225+0000", 2024-09-13T07:03:28.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579228+0000", 2024-09-13T07:03:28.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8030000000000001e-06 2024-09-13T07:03:28.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579234+0000", 2024-09-13T07:03:28.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3110000000000002e-06 2024-09-13T07:03:28.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579240+0000", 2024-09-13T07:03:28.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159626 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.580013+0000", 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9999304, 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003123476, 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.849 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.850 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159626 2024-09-13T07:03:28.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.850 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580013+0000", 2024-09-13T07:03:28.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580013+0000", 2024-09-13T07:03:28.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580012+0000", 2024-09-13T07:03:28.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580014+0000", 2024-09-13T07:03:28.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1449999999999998e-06 2024-09-13T07:03:28.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580020+0000", 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3040000000000004e-06 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580024+0000", 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4939999999999997e-06 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159347 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.852 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577484+0000", 2024-09-13T07:03:28.853 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0024588219999999998, 2024-09-13T07:03:28.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057074270000000002, 2024-09-13T07:03:28.853 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.853 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.853 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.853 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.853 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.853 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159347 2024-09-13T07:03:28.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.853 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577484+0000", 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577484+0000", 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577484+0000", 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577487+0000", 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.258e-06 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577493+0000", 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5400000000000003e-06 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577498+0000", 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4049999999999999e-06 2024-09-13T07:03:28.855 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.856 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.856 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159453 3.16 3.233fd676 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.856 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578202+0000", 2024-09-13T07:03:28.856 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001740632, 2024-09-13T07:03:28.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050399570000000003, 2024-09-13T07:03:28.856 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.856 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159453 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578202+0000", 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578202+0000", 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578202+0000", 2024-09-13T07:03:28.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578205+0000", 2024-09-13T07:03:28.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3100000000000001e-06 2024-09-13T07:03:28.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578211+0000", 2024-09-13T07:03:28.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9889999999999997e-06 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578216+0000", 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0159999999999999e-06 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159516 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579244+0000", 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00069861100000000002, 2024-09-13T07:03:28.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040509420000000001, 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159516 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579244+0000", 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579244+0000", 2024-09-13T07:03:28.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579244+0000", 2024-09-13T07:03:28.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579247+0000", 2024-09-13T07:03:28.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8449999999999999e-06 2024-09-13T07:03:28.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579252+0000", 2024-09-13T07:03:28.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4190000000000001e-06 2024-09-13T07:03:28.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579257+0000", 2024-09-13T07:03:28.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9729999999999997e-06 2024-09-13T07:03:28.862 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.862 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.863 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.863 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159629 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.863 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.580028+0000", 2024-09-13T07:03:28.863 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9999151, 2024-09-13T07:03:28.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033202980000000002, 2024-09-13T07:03:28.863 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.863 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159629 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580028+0000", 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580028+0000", 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580027+0000", 2024-09-13T07:03:28.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580030+0000", 2024-09-13T07:03:28.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5059999999999998e-06 2024-09-13T07:03:28.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580035+0000", 2024-09-13T07:03:28.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4840000000000003e-06 2024-09-13T07:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580040+0000", 2024-09-13T07:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7609999999999998e-06 2024-09-13T07:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159348 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577502+0000", 2024-09-13T07:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0024405030000000001, 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059960120000000002, 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159348 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577502+0000", 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577502+0000", 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577502+0000", 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577505+0000", 2024-09-13T07:03:28.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9050000000000001e-06 2024-09-13T07:03:28.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577511+0000", 2024-09-13T07:03:28.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.711e-06 2024-09-13T07:03:28.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577515+0000", 2024-09-13T07:03:28.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7960000000000003e-06 2024-09-13T07:03:28.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.869 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159454 3.1c 3:3a5225c1:::rbd_data.119485e97dc3.0000000000000062:head [sparse-read 0~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578367+0000", 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001576098, 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00518637, 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159454 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.870 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578367+0000", 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578367+0000", 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578366+0000", 2024-09-13T07:03:28.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578386+0000", 2024-09-13T07:03:28.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9485000000000001e-05 2024-09-13T07:03:28.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578395+0000", 2024-09-13T07:03:28.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:03:28.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578401+0000", 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8129999999999998e-06 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.583219+0000", 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048176080000000001 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.583361+0000", 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014221500000000001 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.873 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.874 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.874 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159518 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.874 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579262+0000", 2024-09-13T07:03:28.874 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.000681181, 2024-09-13T07:03:28.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043500250000000004, 2024-09-13T07:03:28.874 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.874 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159518 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579262+0000", 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579262+0000", 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579261+0000", 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579264+0000", 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1039999999999998e-06 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579270+0000", 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-06 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579276+0000", 2024-09-13T07:03:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5409999999999997e-06 2024-09-13T07:03:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159631 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.580044+0000", 2024-09-13T07:03:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9998989, 2024-09-13T07:03:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035964429999999999, 2024-09-13T07:03:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159631 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580044+0000", 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580044+0000", 2024-09-13T07:03:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580044+0000", 2024-09-13T07:03:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580046+0000", 2024-09-13T07:03:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2350000000000002e-06 2024-09-13T07:03:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580051+0000", 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5099999999999998e-06 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580056+0000", 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6469999999999997e-06 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159349 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.577520+0000", 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0024233900000000001, 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061630859999999999, 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159349 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577520+0000", 2024-09-13T07:03:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577520+0000", 2024-09-13T07:03:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577519+0000", 2024-09-13T07:03:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577522+0000", 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.802e-06 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577528+0000", 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.558e-06 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.577533+0000", 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-13T07:03:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159455 3.16 3.233fd676 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578407+0000", 2024-09-13T07:03:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001535834, 2024-09-13T07:03:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053249769999999998, 2024-09-13T07:03:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159455 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578407+0000", 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578407+0000", 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578407+0000", 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578410+0000", 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1030000000000002e-06 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578416+0000", 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2659999999999998e-06 2024-09-13T07:03:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578422+0000", 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159519 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579422+0000", 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00052130799999999995, 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043643070000000004, 2024-09-13T07:03:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159519 2024-09-13T07:03:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579422+0000", 2024-09-13T07:03:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579422+0000", 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579421+0000", 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579426+0000", 2024-09-13T07:03:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8910000000000003e-06 2024-09-13T07:03:28.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579436+0000", 2024-09-13T07:03:28.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0285999999999999e-05 2024-09-13T07:03:28.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579442+0000", 2024-09-13T07:03:28.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.46e-06 2024-09-13T07:03:28.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.890 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159634 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.580060+0000", 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9998832, 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037766240000000001, 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159634 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580060+0000", 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580060+0000", 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580059+0000", 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580062+0000", 2024-09-13T07:03:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.232e-06 2024-09-13T07:03:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580067+0000", 2024-09-13T07:03:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.604e-06 2024-09-13T07:03:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580072+0000", 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5229999999999997e-06 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159483 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578426+0000", 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001516608, 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054631860000000001, 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159483 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578426+0000", 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578426+0000", 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578426+0000", 2024-09-13T07:03:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578429+0000", 2024-09-13T07:03:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:03:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578435+0000", 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8669999999999999e-06 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578440+0000", 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3689999999999998e-06 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159521 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579447+0000", 2024-09-13T07:03:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00049583899999999998, 2024-09-13T07:03:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004485602, 2024-09-13T07:03:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159521 2024-09-13T07:03:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579447+0000", 2024-09-13T07:03:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579447+0000", 2024-09-13T07:03:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579447+0000", 2024-09-13T07:03:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579450+0000", 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.004e-06 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579455+0000", 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7150000000000003e-06 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579460+0000", 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.977e-06 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159636 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.580075+0000", 2024-09-13T07:03:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9998674, 2024-09-13T07:03:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039046480000000001, 2024-09-13T07:03:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159636 2024-09-13T07:03:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580075+0000", 2024-09-13T07:03:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580075+0000", 2024-09-13T07:03:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580075+0000", 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580077+0000", 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2589999999999999e-06 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580082+0000", 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2270000000000001e-06 2024-09-13T07:03:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580087+0000", 2024-09-13T07:03:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159484 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578444+0000", 2024-09-13T07:03:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001498669, 2024-09-13T07:03:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055729409999999997, 2024-09-13T07:03:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159484 2024-09-13T07:03:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578444+0000", 2024-09-13T07:03:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578444+0000", 2024-09-13T07:03:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578444+0000", 2024-09-13T07:03:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578447+0000", 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.615e-06 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578452+0000", 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.242e-06 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578457+0000", 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7489999999999998e-06 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159522 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579465+0000", 2024-09-13T07:03:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00047848600000000003, 2024-09-13T07:03:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046020380000000001, 2024-09-13T07:03:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159522 2024-09-13T07:03:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579465+0000", 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579465+0000", 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579464+0000", 2024-09-13T07:03:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579467+0000", 2024-09-13T07:03:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7300000000000001e-06 2024-09-13T07:03:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579473+0000", 2024-09-13T07:03:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7919999999999998e-06 2024-09-13T07:03:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579477+0000", 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.724e-06 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159639 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.580092+0000", 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9998512, 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040287960000000003, 2024-09-13T07:03:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159639 2024-09-13T07:03:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580092+0000", 2024-09-13T07:03:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580092+0000", 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580091+0000", 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580093+0000", 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2800000000000002e-06 2024-09-13T07:03:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580099+0000", 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4720000000000003e-06 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580104+0000", 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6009999999999997e-06 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159485 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578461+0000", 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0014823760000000001, 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057210150000000003, 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159485 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578461+0000", 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578461+0000", 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578460+0000", 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578463+0000", 2024-09-13T07:03:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7030000000000002e-06 2024-09-13T07:03:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578469+0000", 2024-09-13T07:03:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8579999999999996e-06 2024-09-13T07:03:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578474+0000", 2024-09-13T07:03:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9590000000000003e-06 2024-09-13T07:03:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159565 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579482+0000", 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00046136400000000002, 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047751260000000002, 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159565 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579482+0000", 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579482+0000", 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579481+0000", 2024-09-13T07:03:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579484+0000", 2024-09-13T07:03:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5940000000000001e-06 2024-09-13T07:03:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579489+0000", 2024-09-13T07:03:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5539999999999997e-06 2024-09-13T07:03:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579494+0000", 2024-09-13T07:03:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3989999999999997e-06 2024-09-13T07:03:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159643 3.5 3.a1267405 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.580107+0000", 2024-09-13T07:03:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.999836, 2024-09-13T07:03:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042025989999999996, 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159643 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580107+0000", 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580107+0000", 2024-09-13T07:03:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580107+0000", 2024-09-13T07:03:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580109+0000", 2024-09-13T07:03:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2369999999999999e-06 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580114+0000", 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3159999999999996e-06 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580119+0000", 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7879999999999997e-06 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159486 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578478+0000", 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0014650799999999999, 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058866550000000002, 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159486 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578478+0000", 2024-09-13T07:03:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578478+0000", 2024-09-13T07:03:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578478+0000", 2024-09-13T07:03:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578480+0000", 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4959999999999999e-06 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578485+0000", 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3550000000000004e-06 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578490+0000", 2024-09-13T07:03:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7720000000000002e-06 2024-09-13T07:03:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159570 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579498+0000", 2024-09-13T07:03:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00044469300000000002, 2024-09-13T07:03:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049162570000000003, 2024-09-13T07:03:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159570 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579498+0000", 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579498+0000", 2024-09-13T07:03:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579498+0000", 2024-09-13T07:03:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579514+0000", 2024-09-13T07:03:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5749000000000001e-05 2024-09-13T07:03:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579521+0000", 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8469999999999998e-06 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579526+0000", 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7610000000000004e-06 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159644 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.931 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.580124+0000", 2024-09-13T07:03:28.932 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9998188, 2024-09-13T07:03:28.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043442489999999997, 2024-09-13T07:03:28.932 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.932 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.932 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159644 2024-09-13T07:03:28.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.932 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580124+0000", 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580124+0000", 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580124+0000", 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580126+0000", 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580131+0000", 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4940000000000003e-06 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580136+0000", 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8670000000000002e-06 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.934 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159487 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578494+0000", 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0014491459999999999, 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060248460000000004, 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159487 2024-09-13T07:03:28.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578494+0000", 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578494+0000", 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578493+0000", 2024-09-13T07:03:28.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578496+0000", 2024-09-13T07:03:28.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3810000000000002e-06 2024-09-13T07:03:28.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578501+0000", 2024-09-13T07:03:28.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:03:28.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578506+0000", 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6550000000000003e-06 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159573 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579531+0000", 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.000412235, 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050496580000000003, 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.938 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159573 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579531+0000", 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579531+0000", 2024-09-13T07:03:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579530+0000", 2024-09-13T07:03:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579533+0000", 2024-09-13T07:03:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5620000000000002e-06 2024-09-13T07:03:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579539+0000", 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7379999999999999e-06 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579543+0000", 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6959999999999996e-06 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159645 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.580140+0000", 2024-09-13T07:03:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9998026, 2024-09-13T07:03:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044907949999999997, 2024-09-13T07:03:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159645 2024-09-13T07:03:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580140+0000", 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580140+0000", 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580140+0000", 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580142+0000", 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3819999999999998e-06 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580156+0000", 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3155e-05 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580161+0000", 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2079999999999999e-06 2024-09-13T07:03:28.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.945 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.945 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.945 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159488 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.945 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578510+0000", 2024-09-13T07:03:28.945 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0014325480000000001, 2024-09-13T07:03:28.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061701200000000003, 2024-09-13T07:03:28.945 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.945 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159488 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578510+0000", 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578510+0000", 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578510+0000", 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578512+0000", 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578518+0000", 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3809999999999998e-06 2024-09-13T07:03:28.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578523+0000", 2024-09-13T07:03:28.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.69e-06 2024-09-13T07:03:28.948 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.948 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.948 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.948 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159576 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.948 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579548+0000", 2024-09-13T07:03:28.948 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.000395206, 2024-09-13T07:03:28.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051884879999999998, 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159576 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579548+0000", 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579548+0000", 2024-09-13T07:03:28.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579547+0000", 2024-09-13T07:03:28.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579550+0000", 2024-09-13T07:03:28.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6680000000000001e-06 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579556+0000", 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5740000000000003e-06 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579561+0000", 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0679999999999996e-06 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159646 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.580165+0000", 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9997778, 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046205559999999996, 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159646 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580165+0000", 2024-09-13T07:03:28.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580165+0000", 2024-09-13T07:03:28.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580165+0000", 2024-09-13T07:03:28.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580168+0000", 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7049999999999999e-06 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580173+0000", 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-06 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580179+0000", 2024-09-13T07:03:28.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3210000000000004e-06 2024-09-13T07:03:28.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.955 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.955 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159489 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.955 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578526+0000", 2024-09-13T07:03:28.955 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001416671, 2024-09-13T07:03:28.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006317273, 2024-09-13T07:03:28.955 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.955 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.955 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.956 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159489 2024-09-13T07:03:28.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.956 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578526+0000", 2024-09-13T07:03:28.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578526+0000", 2024-09-13T07:03:28.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578526+0000", 2024-09-13T07:03:28.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578529+0000", 2024-09-13T07:03:28.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5639999999999999e-06 2024-09-13T07:03:28.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578534+0000", 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2959999999999998e-06 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578538+0000", 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5549999999999996e-06 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.958 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159579 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579565+0000", 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00037819200000000001, 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053281099999999996, 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159579 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579565+0000", 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579565+0000", 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579564+0000", 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579567+0000", 2024-09-13T07:03:28.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7819999999999998e-06 2024-09-13T07:03:28.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579573+0000", 2024-09-13T07:03:28.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7180000000000001e-06 2024-09-13T07:03:28.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579578+0000", 2024-09-13T07:03:28.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.826e-06 2024-09-13T07:03:28.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159647 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.580182+0000", 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9997606, 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047648350000000003, 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.962 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159647 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580182+0000", 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580182+0000", 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580182+0000", 2024-09-13T07:03:28.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580185+0000", 2024-09-13T07:03:28.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5270000000000001e-06 2024-09-13T07:03:28.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580190+0000", 2024-09-13T07:03:28.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3859999999999998e-06 2024-09-13T07:03:28.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580195+0000", 2024-09-13T07:03:28.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7310000000000001e-06 2024-09-13T07:03:28.965 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.965 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.965 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.965 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159490 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.965 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.578542+0000", 2024-09-13T07:03:28.965 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001401097, 2024-09-13T07:03:28.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006455138, 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159490 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578542+0000", 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578542+0000", 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578542+0000", 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578544+0000", 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2919999999999998e-06 2024-09-13T07:03:28.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578549+0000", 2024-09-13T07:03:28.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.293e-06 2024-09-13T07:03:28.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.578554+0000", 2024-09-13T07:03:28.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7679999999999999e-06 2024-09-13T07:03:28.968 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.968 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.968 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159582 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.579582+0000", 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00036116000000000002, 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054683800000000001, 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159582 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579582+0000", 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579582+0000", 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579582+0000", 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579584+0000", 2024-09-13T07:03:28.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-13T07:03:28.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579590+0000", 2024-09-13T07:03:28.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8200000000000002e-06 2024-09-13T07:03:28.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.579595+0000", 2024-09-13T07:03:28.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0849999999999996e-06 2024-09-13T07:03:28.972 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.972 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.972 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.972 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:159648 3.16 3.c0f69bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:28.972 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.580199+0000", 2024-09-13T07:03:28.972 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4294967295.9997444, 2024-09-13T07:03:28.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049037880000000001, 2024-09-13T07:03:28.972 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:28.972 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "queued for pg", 2024-09-13T07:03:28.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159648 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580199+0000", 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580199+0000", 2024-09-13T07:03:28.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:28.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:28.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580198+0000", 2024-09-13T07:03:28.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:28.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:28.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580204+0000", 2024-09-13T07:03:28.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8030000000000002e-06 2024-09-13T07:03:28.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:28.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580210+0000", 2024-09-13T07:03:28.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-13T07:03:28.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:28.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:28.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:28.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.580215+0000", 2024-09-13T07:03:28.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0080000000000002e-06 2024-09-13T07:03:28.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.975 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:28.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:28.975 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-13T07:03:28.975 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 101 2024-09-13T07:03:28.976 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:28.976 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:03:28.978 INFO:teuthology.orchestra.run.smithi118.stdout:5487 write 0xd5c0ab0 thru 0xd5c476b (0x3cbc bytes) 2024-09-13T07:03:28.978 INFO:teuthology.orchestra.run.smithi118.stdout:5488 read 0xa6e58e5 thru 0xa6ea81c (0x4f38 bytes) 2024-09-13T07:03:28.978 INFO:teuthology.orchestra.run.smithi118.stdout:5489 punch from 0x9adda5b to 0x9ae3497, (0x5a3c bytes) 2024-09-13T07:03:28.978 INFO:teuthology.orchestra.run.smithi118.stdout:5490 read 0x4a5dacf thru 0x4a6315f (0x5691 bytes) 2024-09-13T07:03:28.978 INFO:teuthology.orchestra.run.smithi118.stdout:5491 write 0x11578c5 thru 0x11588b7 (0xff3 bytes) 2024-09-13T07:03:28.978 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:28.975+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:28.979 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:03:29.025 INFO:teuthology.orchestra.run.smithi118.stdout:5492 trunc from 0xed936a1 to 0x70e405e 2024-09-13T07:03:29.078 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:29.074+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:29.100 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:29.100 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:29.100 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:29.101 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:29.101 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:29.113 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:03:29.127 INFO:teuthology.orchestra.run.smithi118.stdout:5494 read 0x969b5e thru 0x96fd8b (0x622e bytes) 2024-09-13T07:03:29.132 INFO:teuthology.orchestra.run.smithi118.stdout:5496 trunc from 0x70e405e to 0x8a1091f 2024-09-13T07:03:29.159 INFO:teuthology.orchestra.run.smithi118.stdout:5497 write 0xaea3c61 thru 0xaea70e6 (0x3486 bytes) 2024-09-13T07:03:29.165 INFO:teuthology.orchestra.run.smithi118.stdout:5499 read 0x66fcd48 thru 0x670772c (0xa9e5 bytes) 2024-09-13T07:03:29.167 INFO:teuthology.orchestra.run.smithi118.stdout:5503 read 0x5c3670c thru 0x5c3dc64 (0x7559 bytes) 2024-09-13T07:03:29.167 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:29.167 INFO:teuthology.orchestra.run.smithi118.stdout:5505 write 0x21837d5 thru 0x218d130 (0x995c bytes) 2024-09-13T07:03:29.179 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:29.175+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:29.223 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:29.224 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:29.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:29.224 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:29.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.224 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:03:29.224 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:29.224 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.10430132400001, 2024-09-13T07:03:29.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:03:29.224 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.224 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:29.224 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.224 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.224 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.225 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:03:29.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.225 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:29.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:29.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:03:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:03:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:03:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:03:29.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:03:29.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:03:29.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:03:29.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:29.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129532 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.607898437000003, 2024-09-13T07:03:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89889205500000002, 2024-09-13T07:03:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129532 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:29.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614994+0000", 2024-09-13T07:03:29.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.044e-06 2024-09-13T07:03:29.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615000+0000", 2024-09-13T07:03:29.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9229999999999996e-06 2024-09-13T07:03:29.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615005+0000", 2024-09-13T07:03:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:03:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615060+0000", 2024-09-13T07:03:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4561000000000002e-05 2024-09-13T07:03:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615069+0000", 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1540000000000008e-06 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513869+0000", 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89879977700000002 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513883+0000", 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4361e-05 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.235 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129533 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.607803103999998, 2024-09-13T07:03:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.899270177, 2024-09-13T07:03:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129533 2024-09-13T07:03:29.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615090+0000", 2024-09-13T07:03:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.856e-06 2024-09-13T07:03:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615096+0000", 2024-09-13T07:03:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.703e-06 2024-09-13T07:03:29.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615102+0000", 2024-09-13T07:03:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.378e-06 2024-09-13T07:03:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615140+0000", 2024-09-13T07:03:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7966999999999997e-05 2024-09-13T07:03:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615147+0000", 2024-09-13T07:03:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8800000000000002e-06 2024-09-13T07:03:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514342+0000", 2024-09-13T07:03:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919565300000004 2024-09-13T07:03:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514357+0000", 2024-09-13T07:03:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4355000000000001e-05 2024-09-13T07:03:29.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129534 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.607730205999999, 2024-09-13T07:03:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89923080600000005, 2024-09-13T07:03:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129534 2024-09-13T07:03:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615162+0000", 2024-09-13T07:03:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3569999999999998e-06 2024-09-13T07:03:29.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615169+0000", 2024-09-13T07:03:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.455e-06 2024-09-13T07:03:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615175+0000", 2024-09-13T07:03:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3679999999999998e-06 2024-09-13T07:03:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615212+0000", 2024-09-13T07:03:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6733999999999998e-05 2024-09-13T07:03:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615218+0000", 2024-09-13T07:03:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7569999999999998e-06 2024-09-13T07:03:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514383+0000", 2024-09-13T07:03:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916408000000003 2024-09-13T07:03:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514390+0000", 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5429999999999996e-06 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129535 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.60765679, 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919205499999999, 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.246 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129535 2024-09-13T07:03:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615232+0000", 2024-09-13T07:03:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615236+0000", 2024-09-13T07:03:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:29.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615244+0000", 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7369999999999995e-06 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615250+0000", 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2270000000000001e-06 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:29.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4627000000000002e-05 2024-09-13T07:03:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5089999999999993e-06 2024-09-13T07:03:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514412+0000", 2024-09-13T07:03:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89907890599999996 2024-09-13T07:03:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514425+0000", 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3138e-05 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129536 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.607623312999998, 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89918503500000002, 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129536 2024-09-13T07:03:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:29.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615268+0000", 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8370000000000001e-06 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615288+0000", 2024-09-13T07:03:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9945999999999998e-05 2024-09-13T07:03:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615294+0000", 2024-09-13T07:03:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3340000000000001e-06 2024-09-13T07:03:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615374+0000", 2024-09-13T07:03:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9988999999999996e-05 2024-09-13T07:03:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615381+0000", 2024-09-13T07:03:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:03:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514444+0000", 2024-09-13T07:03:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89906347499999995 2024-09-13T07:03:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514451+0000", 2024-09-13T07:03:29.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7630000000000003e-06 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129537 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.607574761000002, 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916833399999996, 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.256 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129537 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615314+0000", 2024-09-13T07:03:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615317+0000", 2024-09-13T07:03:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1410000000000001e-06 2024-09-13T07:03:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3589999999999996e-06 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615329+0000", 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615408+0000", 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8678999999999997e-05 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615414+0000", 2024-09-13T07:03:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:03:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514471+0000", 2024-09-13T07:03:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905719200000001 2024-09-13T07:03:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514483+0000", 2024-09-13T07:03:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1691999999999999e-05 2024-09-13T07:03:29.260 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129538 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.607556221999999, 2024-09-13T07:03:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89917603999999995, 2024-09-13T07:03:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129538 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615335+0000", 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4090000000000001e-06 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615346+0000", 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0866000000000001e-05 2024-09-13T07:03:29.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615424+0000", 2024-09-13T07:03:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7628999999999998e-05 2024-09-13T07:03:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615459+0000", 2024-09-13T07:03:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5046999999999998e-05 2024-09-13T07:03:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615466+0000", 2024-09-13T07:03:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7610000000000002e-06 2024-09-13T07:03:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514503+0000", 2024-09-13T07:03:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89903750299999996 2024-09-13T07:03:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514509+0000", 2024-09-13T07:03:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2140000000000001e-06 2024-09-13T07:03:29.265 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129539 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.607411470999999, 2024-09-13T07:03:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905735799999997, 2024-09-13T07:03:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.266 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129539 2024-09-13T07:03:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615477+0000", 2024-09-13T07:03:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615481+0000", 2024-09-13T07:03:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6490000000000001e-06 2024-09-13T07:03:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615488+0000", 2024-09-13T07:03:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4359999999999999e-06 2024-09-13T07:03:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615493+0000", 2024-09-13T07:03:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3120000000000001e-06 2024-09-13T07:03:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615529+0000", 2024-09-13T07:03:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6468999999999999e-05 2024-09-13T07:03:29.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615536+0000", 2024-09-13T07:03:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9789999999999996e-06 2024-09-13T07:03:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514529+0000", 2024-09-13T07:03:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89899276400000006 2024-09-13T07:03:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514535+0000", 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2920000000000001e-06 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129540 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.607340477000001, 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89901438700000003, 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.271 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129540 2024-09-13T07:03:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:29.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615548+0000", 2024-09-13T07:03:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615552+0000", 2024-09-13T07:03:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8349999999999997e-06 2024-09-13T07:03:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615559+0000", 2024-09-13T07:03:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-13T07:03:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615564+0000", 2024-09-13T07:03:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2360000000000004e-06 2024-09-13T07:03:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615600+0000", 2024-09-13T07:03:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6637000000000002e-05 2024-09-13T07:03:29.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615607+0000", 2024-09-13T07:03:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.669e-06 2024-09-13T07:03:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514557+0000", 2024-09-13T07:03:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89894939399999996 2024-09-13T07:03:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514563+0000", 2024-09-13T07:03:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8560000000000001e-06 2024-09-13T07:03:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129541 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.607267864999997, 2024-09-13T07:03:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89896824600000003, 2024-09-13T07:03:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129541 2024-09-13T07:03:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615621+0000", 2024-09-13T07:03:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615625+0000", 2024-09-13T07:03:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-13T07:03:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615631+0000", 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3890000000000002e-06 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615636+0000", 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3489999999999999e-06 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615671+0000", 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4570000000000003e-05 2024-09-13T07:03:29.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615678+0000", 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8510000000000001e-06 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514583+0000", 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89890530599999996 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514590+0000", 2024-09-13T07:03:29.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7469999999999999e-06 2024-09-13T07:03:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129542 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.607194374999999, 2024-09-13T07:03:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89892092400000001, 2024-09-13T07:03:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.281 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129542 2024-09-13T07:03:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615698+0000", 2024-09-13T07:03:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.664e-06 2024-09-13T07:03:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615705+0000", 2024-09-13T07:03:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615711+0000", 2024-09-13T07:03:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6470000000000003e-06 2024-09-13T07:03:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615752+0000", 2024-09-13T07:03:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0389999999999998e-05 2024-09-13T07:03:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615759+0000", 2024-09-13T07:03:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9230000000000002e-06 2024-09-13T07:03:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514609+0000", 2024-09-13T07:03:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89884984899999998 2024-09-13T07:03:29.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514616+0000", 2024-09-13T07:03:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7299999999999999e-06 2024-09-13T07:03:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129543 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.607115501999999, 2024-09-13T07:03:29.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89886872600000001, 2024-09-13T07:03:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129543 2024-09-13T07:03:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615773+0000", 2024-09-13T07:03:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615777+0000", 2024-09-13T07:03:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1860000000000002e-06 2024-09-13T07:03:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615785+0000", 2024-09-13T07:03:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2710000000000003e-06 2024-09-13T07:03:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615790+0000", 2024-09-13T07:03:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3619999999999997e-06 2024-09-13T07:03:29.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615828+0000", 2024-09-13T07:03:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8059999999999998e-05 2024-09-13T07:03:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615836+0000", 2024-09-13T07:03:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8809999999999996e-06 2024-09-13T07:03:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514636+0000", 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89880009599999999 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514643+0000", 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129544 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.607039980000003, 2024-09-13T07:03:29.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89881970799999999, 2024-09-13T07:03:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129544 2024-09-13T07:03:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615853+0000", 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8600000000000003e-06 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615860+0000", 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1709999999999996e-06 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615867+0000", 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7020000000000004e-06 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615905+0000", 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8265e-05 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615913+0000", 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514663+0000", 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89874952500000005 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514669+0000", 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129545 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.606961806999998, 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89876792599999999, 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129545 2024-09-13T07:03:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615927+0000", 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615931+0000", 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8010000000000003e-06 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615939+0000", 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6310000000000003e-06 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615947+0000", 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7090000000000002e-06 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615985+0000", 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8389000000000001e-05 2024-09-13T07:03:29.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615993+0000", 2024-09-13T07:03:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2209999999999997e-06 2024-09-13T07:03:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514689+0000", 2024-09-13T07:03:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89869535199999995 2024-09-13T07:03:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514696+0000", 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8179999999999998e-06 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129546 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.606881719, 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89872448199999999, 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129546 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616007+0000", 2024-09-13T07:03:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616011+0000", 2024-09-13T07:03:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1659999999999996e-06 2024-09-13T07:03:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616018+0000", 2024-09-13T07:03:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0829999999999998e-06 2024-09-13T07:03:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616024+0000", 2024-09-13T07:03:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8170000000000004e-06 2024-09-13T07:03:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616063+0000", 2024-09-13T07:03:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8708000000000003e-05 2024-09-13T07:03:29.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616070+0000", 2024-09-13T07:03:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6299999999999998e-06 2024-09-13T07:03:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514721+0000", 2024-09-13T07:03:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89865028800000002 2024-09-13T07:03:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514732+0000", 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1452e-05 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129547 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.606806390999999, 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89868529799999997, 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129547 2024-09-13T07:03:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:29.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616086+0000", 2024-09-13T07:03:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.394e-06 2024-09-13T07:03:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616093+0000", 2024-09-13T07:03:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1130000000000003e-06 2024-09-13T07:03:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616100+0000", 2024-09-13T07:03:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1520000000000003e-06 2024-09-13T07:03:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616138+0000", 2024-09-13T07:03:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8084000000000003e-05 2024-09-13T07:03:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616146+0000", 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3010000000000001e-06 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89861259299999996 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514768+0000", 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0175e-05 2024-09-13T07:03:29.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129548 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.606722331999997, 2024-09-13T07:03:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89862847899999998, 2024-09-13T07:03:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129548 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616166+0000", 2024-09-13T07:03:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616170+0000", 2024-09-13T07:03:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8619999999999996e-06 2024-09-13T07:03:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616177+0000", 2024-09-13T07:03:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0450000000000003e-06 2024-09-13T07:03:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616183+0000", 2024-09-13T07:03:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.139e-06 2024-09-13T07:03:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616221+0000", 2024-09-13T07:03:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7320000000000002e-05 2024-09-13T07:03:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616228+0000", 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2980000000000003e-06 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514789+0000", 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89856044599999996 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514796+0000", 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0809999999999996e-06 2024-09-13T07:03:29.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129549 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.606648868999997, 2024-09-13T07:03:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89859381900000002, 2024-09-13T07:03:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.316 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129549 2024-09-13T07:03:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616240+0000", 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616244+0000", 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4149999999999999e-06 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616251+0000", 2024-09-13T07:03:29.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-13T07:03:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616256+0000", 2024-09-13T07:03:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3299999999999998e-06 2024-09-13T07:03:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616302+0000", 2024-09-13T07:03:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5646999999999999e-05 2024-09-13T07:03:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616309+0000", 2024-09-13T07:03:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5530000000000004e-06 2024-09-13T07:03:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514825+0000", 2024-09-13T07:03:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89851528400000003 2024-09-13T07:03:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514834+0000", 2024-09-13T07:03:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8349999999999999e-06 2024-09-13T07:03:29.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129550 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.606564489999997, 2024-09-13T07:03:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89854296199999995, 2024-09-13T07:03:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:29.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129550 2024-09-13T07:03:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:29.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616324+0000", 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616328+0000", 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7479999999999999e-06 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616335+0000", 2024-09-13T07:03:29.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9280000000000004e-06 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616342+0000", 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2690000000000002e-06 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616380+0000", 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:29.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616387+0000", 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.075e-06 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514861+0000", 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89847450799999995 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514868+0000", 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6050000000000003e-06 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:29.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:29.326 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:29.327 INFO:teuthology.orchestra.run.smithi118.stdout:5506 write 0x18ce7be thru 0x18dcf37 (0xe77a bytes) 2024-09-13T07:03:29.327 INFO:teuthology.orchestra.run.smithi118.stdout:5507 writesame 0x95ce151 thru 0x95d07f1 data_size 0x1d(0x26a1 bytes) 2024-09-13T07:03:29.327 INFO:teuthology.orchestra.run.smithi118.stdout:5509 punch from 0x54f1a94 to 0x54f25d5, (0xb41 bytes) 2024-09-13T07:03:29.327 INFO:teuthology.orchestra.run.smithi118.stdout:5510 write 0x1f20aac thru 0x1f23d5e (0x32b3 bytes) 2024-09-13T07:03:29.328 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:29.323+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:29.328 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:29.351 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:03:29.428 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:29.426+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:29.443 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:29.443 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:29.452 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:29.457 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:29.457 INFO:teuthology.orchestra.run.smithi118.stdout:5512 write 0x5b365f thru 0x5bee96 (0xb838 bytes) 2024-09-13T07:03:29.460 INFO:teuthology.orchestra.run.smithi118.stdout:5513 punch from 0x455f11f to 0x45638ee, (0x47cf bytes) 2024-09-13T07:03:29.460 INFO:teuthology.orchestra.run.smithi118.stdout:5514 read 0x709eb29 thru 0x70a8fd6 (0xa4ae bytes) 2024-09-13T07:03:29.461 INFO:teuthology.orchestra.run.smithi118.stdout:5515 writesame 0x95b9b82 thru 0x95c2a94 data_size 0x1d(0x8f13 bytes) 2024-09-13T07:03:29.466 INFO:teuthology.orchestra.run.smithi118.stdout:5517 punch from 0xa60dc51 to 0xa611284, (0x3633 bytes) 2024-09-13T07:03:29.466 INFO:teuthology.orchestra.run.smithi118.stdout:5519 read 0x3d5d6b9 thru 0x3d65038 (0x7980 bytes) 2024-09-13T07:03:29.467 INFO:teuthology.orchestra.run.smithi118.stdout:5520 write 0x2f9727e thru 0x2f9e439 (0x71bc bytes) 2024-09-13T07:03:29.484 INFO:teuthology.orchestra.run.smithi118.stdout:5521 trunc from 0xaea70e7 to 0x3147513 2024-09-13T07:03:29.486 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:29.516 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:29.528 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:29.523+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:29.542 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:29.560 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:29.582 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:29.610 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:29.610 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:29.610 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:29.610 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:29.610 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:29.610 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:29.621 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:29.629 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:29.626+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:29.631 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:29.639 INFO:teuthology.orchestra.run.smithi118.stdout:5522 writesame 0x3bce956 thru 0x3bcec97 data_size 0x3(0x342 bytes) 2024-09-13T07:03:29.655 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:29.657 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:29.657 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:29.657 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:29.658 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:29.658 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:29.658 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:29.658 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:29.658 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:29.659 INFO:teuthology.orchestra.run.smithi118.stdout:5523 write 0x726d4f6 thru 0x7276558 (0x9063 bytes) 2024-09-13T07:03:29.678 INFO:teuthology.orchestra.run.smithi118.stdout:5524 write 0x8126509 thru 0x8134c0e (0xe706 bytes) 2024-09-13T07:03:29.687 INFO:teuthology.orchestra.run.smithi118.stdout:5525 trunc from 0x8134c0f to 0x964907f 2024-09-13T07:03:29.718 INFO:teuthology.orchestra.run.smithi118.stdout:5526 trunc from 0x964907f to 0x1201d0c 2024-09-13T07:03:29.727 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:29.727 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:29.730 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:29.727+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:29.738 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:29.755 INFO:teuthology.orchestra.run.smithi118.stdout:5527 trunc from 0x1201d0c to 0x5bff469 2024-09-13T07:03:29.766 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:29.786 INFO:teuthology.orchestra.run.smithi118.stdout:5529 write 0xaacac72 thru 0xaad3a49 (0x8dd8 bytes) 2024-09-13T07:03:29.792 INFO:teuthology.orchestra.run.smithi118.stdout:5530 read 0x7b9455f thru 0x7b9cac7 (0x8569 bytes) 2024-09-13T07:03:29.792 INFO:teuthology.orchestra.run.smithi118.stdout:5531 read 0x77aac19 thru 0x77b9830 (0xec18 bytes) 2024-09-13T07:03:29.792 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:29.792 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:29.793 INFO:teuthology.orchestra.run.smithi118.stdout:5535 punch from 0x784b6e9 to 0x7859d1b, (0xe632 bytes) 2024-09-13T07:03:29.813 INFO:teuthology.orchestra.run.smithi118.stdout:5536 writesame 0xe4b861e thru 0xe4c18bd data_size 0x66(0x92a0 bytes) 2024-09-13T07:03:29.831 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:29.827+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:29.833 INFO:teuthology.orchestra.run.smithi118.stdout:5540 trunc from 0xe4c18be to 0x2e58d2f 2024-09-13T07:03:29.856 INFO:teuthology.orchestra.run.smithi118.stdout:5541 writesame 0x1c0c046 thru 0x1c15c52 data_size 0xd(0x9c0d bytes) 2024-09-13T07:03:29.873 INFO:teuthology.orchestra.run.smithi118.stdout:5542 trunc from 0x2e58d2f to 0x9c2943b 2024-09-13T07:03:29.886 INFO:teuthology.orchestra.run.smithi118.stdout:5543 read 0x8e02643 thru 0x8e04f1f (0x28dd bytes) 2024-09-13T07:03:29.894 INFO:teuthology.orchestra.run.smithi118.stdout:5544 trunc from 0x9c2943b to 0x606e0dd 2024-09-13T07:03:29.896 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:29.896 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:29.906 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:29.906 INFO:teuthology.orchestra.run.smithi118.stdout:5545 punch from 0x42ca68d to 0x42cb496, (0xe09 bytes) 2024-09-13T07:03:29.907 INFO:teuthology.orchestra.run.smithi118.stdout:5547 write 0x4a9b9ca thru 0x4aa880a (0xce41 bytes) 2024-09-13T07:03:29.933 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:29.927+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:29.933 INFO:teuthology.orchestra.run.smithi118.stdout:5549 write 0x9d10b2b thru 0x9d186cf (0x7ba5 bytes) 2024-09-13T07:03:29.939 INFO:teuthology.orchestra.run.smithi118.stdout:5551 writesame 0x856053a thru 0x856a429 data_size 0x8(0x9ef0 bytes) 2024-09-13T07:03:29.950 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:29.955 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:03:29.962 INFO:teuthology.orchestra.run.smithi118.stdout:5554 write 0xad950d7 thru 0xad9aa39 (0x5963 bytes) 2024-09-13T07:03:29.964 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:29.964 INFO:teuthology.orchestra.run.smithi118.stdout:5556 punch from 0xa5da4f3 to 0xa5e59c2, (0xb4cf bytes) 2024-09-13T07:03:29.964 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:29.964 INFO:teuthology.orchestra.run.smithi118.stdout:5558 write 0x9efb500 thru 0x9efb88a (0x38b bytes) 2024-09-13T07:03:29.969 INFO:teuthology.orchestra.run.smithi118.stdout:5559 writesame 0x4aadf8b thru 0x4abdece data_size 0x44(0xff44 bytes) 2024-09-13T07:03:29.972 INFO:teuthology.orchestra.run.smithi118.stdout:5560 write 0x47d6dc2 thru 0x47e051d (0x975c bytes) 2024-09-13T07:03:29.979 INFO:teuthology.orchestra.run.smithi118.stdout:5563 read 0x75b25df thru 0x75b544a (0x2e6c bytes) 2024-09-13T07:03:29.980 INFO:teuthology.orchestra.run.smithi118.stdout:5564 writesame 0x253d653 thru 0x2543c0f data_size 0x1(0x65bd bytes) 2024-09-13T07:03:30.011 INFO:teuthology.orchestra.run.smithi118.stdout:5565 write 0xd3b9be3 thru 0xd3c8d90 (0xf1ae bytes) 2024-09-13T07:03:30.020 INFO:teuthology.orchestra.run.smithi118.stdout:5566 punch from 0xb57323d to 0xb57b11d, (0x7ee0 bytes) 2024-09-13T07:03:30.025 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:30.025 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:30.033 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:30.030+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:30.034 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:30.035 INFO:teuthology.orchestra.run.smithi118.stdout:5567 trunc from 0xd3c8d91 to 0x5970d8d 2024-09-13T07:03:30.057 INFO:teuthology.orchestra.run.smithi118.stdout:5569 read 0xa9b5cf thru 0xaa3022 (0x7a54 bytes) 2024-09-13T07:03:30.058 INFO:teuthology.orchestra.run.smithi118.stdout:5570 punch from 0x3d44f80 to 0x3d4a013, (0x5093 bytes) 2024-09-13T07:03:30.080 INFO:teuthology.orchestra.run.smithi118.stdout:5571 writesame 0xae066a3 thru 0xae08cb2 data_size 0x30(0x2610 bytes) 2024-09-13T07:03:30.081 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:30.093 INFO:teuthology.orchestra.run.smithi118.stdout:5573 trunc from 0xae08cb3 to 0x2bc9d74 2024-09-13T07:03:30.118 INFO:teuthology.orchestra.run.smithi118.stdout:5574 trunc from 0x2bc9d74 to 0x34470fd 2024-09-13T07:03:30.129 INFO:teuthology.orchestra.run.smithi118.stdout:5575 read 0x174aad7 thru 0x174c72c (0x1c56 bytes) 2024-09-13T07:03:30.134 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:30.131+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:30.164 INFO:teuthology.orchestra.run.smithi118.stdout:5577 write 0xdffc9a4 thru 0xe002829 (0x5e86 bytes) 2024-09-13T07:03:30.173 INFO:teuthology.orchestra.run.smithi118.stdout:5578 read 0x603a4cb thru 0x603f352 (0x4e88 bytes) 2024-09-13T07:03:30.194 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:30.194 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:30.197 INFO:teuthology.orchestra.run.smithi118.stdout:5579 trunc from 0xe00282a to 0xad5ef0 2024-09-13T07:03:30.205 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:30.235 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:30.234+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:30.237 INFO:teuthology.orchestra.run.smithi118.stdout:5580 trunc from 0xad5ef0 to 0x7d5c0e5 2024-09-13T07:03:30.247 INFO:teuthology.orchestra.run.smithi118.stdout:5581 read 0x6bfd8de thru 0x6c0128b (0x39ae bytes) 2024-09-13T07:03:30.260 INFO:teuthology.orchestra.run.smithi118.stdout:5582 writesame 0x812f9b9 thru 0x813af26 data_size 0x3(0xb56e bytes) 2024-09-13T07:03:30.274 INFO:teuthology.orchestra.run.smithi118.stdout:5584 write 0xa46cf70 thru 0xa471b92 (0x4c23 bytes) 2024-09-13T07:03:30.281 INFO:teuthology.orchestra.run.smithi118.stdout:5585 write 0x48ffa89 thru 0x4902dbe (0x3336 bytes) 2024-09-13T07:03:30.287 INFO:teuthology.orchestra.run.smithi118.stdout:5589 clone 49 order 24 su 2097152 sc 3 2024-09-13T07:03:30.318 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:30.318 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:30.328 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:30.336 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:30.334+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:30.437 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:30.438+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:30.478 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:30.478 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:30.488 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:30.538 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:30.535+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:30.596 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:30.596 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:30.605 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:30.649 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:30.647+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:30.741 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:30.742+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:30.758 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:30.758 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:30.767 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:30.841 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:30.839+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:30.871 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:30.872 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:30.881 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:30.941 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:30.938+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:31.035 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:31.035 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:31.043 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:31.039+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:31.045 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:31.144 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:31.139+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:31.152 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:31.152 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:31.162 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:31.188 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone48 from 0xa471b93 (overlap 0xad5ef0) to 0x45592c 2024-09-13T07:03:31.244 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:31.239+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:31.271 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:31.271 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:31.286 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:31.345 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:31.342+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:31.438 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:31.438 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:31.446 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:31.443+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:31.449 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:31.508 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-09-13T07:03:31.547 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:31.543+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:31.555 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:31.555 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:31.568 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:31.651 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:31.646+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:31.669 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:03:31.704 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:31.704 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:31.715 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:31.749 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:31.746+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:31.811 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:31.833 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:31.850 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:31.846+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:31.852 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:31.870 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:31.875 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.875 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.753744405999996, 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:31.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:03:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:03:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:03:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:03:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:03:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:03:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:03:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:03:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:03:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:03:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:03:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:03:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.753723929000003, 2024-09-13T07:03:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:03:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.880 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.884 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.747027325999994, 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:31.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:03:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:03:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:03:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:03:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:03:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:03:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.888 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.747015257000001, 2024-09-13T07:03:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:03:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.889 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:03:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:03:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.747002813999998, 2024-09-13T07:03:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:03:31.893 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:03:31.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:03:31.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:03:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:03:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:03:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:03:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:03:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.746985215999999, 2024-09-13T07:03:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:03:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.898 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:03:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:03:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:03:31.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:03:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:03:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:03:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:03:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:03:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.902 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.746974512999998, 2024-09-13T07:03:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:03:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:03:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:31.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:03:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:03:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:03:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:03:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:03:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:03:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.746930579999997, 2024-09-13T07:03:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:03:31.907 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:03:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:03:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:03:31.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:03:31.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.911 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.746910794000001, 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:03:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:03:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:03:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:03:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:03:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:03:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:03:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:03:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:03:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:03:31.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.746895043999999, 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.916 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:03:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:31.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:31.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:31.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.746883792000006, 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:03:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:03:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:03:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:03:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:03:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:03:31.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:03:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:03:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:03:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:03:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.746873192999999, 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.925 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:03:31.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:03:31.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.746863700000006, 2024-09-13T07:03:31.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:03:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:03:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:03:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:03:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.746854155999998, 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.937 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.746821553000004, 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:03:31.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:03:31.941 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.746808529999996, 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.942 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:31.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:03:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:03:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:03:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:03:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:03:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:03:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:03:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:03:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:03:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:03:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:03:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:03:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.946 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.743457544999998, 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:03:31.950 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.951 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.951 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.951 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.951 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:31.951 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.743437267999994, 2024-09-13T07:03:31.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:03:31.951 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.951 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.951 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.951 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.951 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.952 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:03:31.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.952 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:31.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:31.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:03:31.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:31.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:03:31.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:03:31.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:31.955 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.743417066999996, 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:03:31.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:03:31.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.959 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:166663 3.b e135/117 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head v 135'409, mlcod=132'407)", 2024-09-13T07:03:31.960 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:03:31.960 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 2.4608281019999998, 2024-09-13T07:03:31.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045108144000000003, 2024-09-13T07:03:31.960 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:31.960 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:31.960 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:31.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:31.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:03:31.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:31.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:03:31.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:31.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:31.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414172+0000", 2024-09-13T07:03:31.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:31.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:31.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427200+0000", 2024-09-13T07:03:31.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013028098 2024-09-13T07:03:31.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427229+0000", 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9178000000000001e-05 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427239+0000", 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6870000000000006e-06 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427393+0000", 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015382899999999999 2024-09-13T07:03:31.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:31.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427451+0000", 2024-09-13T07:03:31.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8301000000000003e-05 2024-09-13T07:03:31.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:31.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459232+0000", 2024-09-13T07:03:31.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031780695999999997 2024-09-13T07:03:31.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:31.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:31.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:31.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459282+0000", 2024-09-13T07:03:31.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0701000000000001e-05 2024-09-13T07:03:31.964 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.964 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.964 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.964 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:31.964 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:31.964 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:31.966 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:31.966 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:31.967 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:31.967 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:31.967 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:31.967 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:31.967 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:31.967 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:31.967 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:31.967 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:31.967 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:31.967 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:31.967 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:31.968 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:31.963+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:31.968 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:03:31.994 INFO:teuthology.orchestra.run.smithi118.stdout:5593 read 0x33ad7fe thru 0x33b70ae (0x98b1 bytes) 2024-09-13T07:03:31.995 INFO:teuthology.orchestra.run.smithi118.stdout:5594 write 0x6ec3599 thru 0x6eceb43 (0xb5ab bytes) 2024-09-13T07:03:32.029 INFO:teuthology.orchestra.run.smithi118.stdout:5595 write 0xb73f5a0 thru 0xb743958 (0x43b9 bytes) 2024-09-13T07:03:32.036 INFO:teuthology.orchestra.run.smithi118.stdout:5596 write 0x86303e9 thru 0x8633b20 (0x3738 bytes) 2024-09-13T07:03:32.039 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 7, 2, 0] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:03:32.039 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:03:32.040 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-13T07:03:32.040 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-13T07:03:32.040 INFO:teuthology.orchestra.run.smithi118.stdout:5597 writesame 0xae089b0 thru 0xae0ea9c data_size 0x9(0x60ed bytes) 2024-09-13T07:03:32.049 INFO:teuthology.orchestra.run.smithi118.stdout:5598 read 0xaec46e6 thru 0xaec786d (0x3188 bytes) 2024-09-13T07:03:32.050 INFO:teuthology.orchestra.run.smithi118.stdout:5599 write 0x1926d29 thru 0x1931f38 (0xb210 bytes) 2024-09-13T07:03:32.066 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:32.063+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:32.085 INFO:teuthology.orchestra.run.smithi118.stdout:5602 write 0x6d43031 thru 0x6d4dbe1 (0xabb1 bytes) 2024-09-13T07:03:32.092 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:32.092 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:32.092 INFO:teuthology.orchestra.run.smithi118.stdout:5606 read 0x6ea0fda thru 0x6eaeea7 (0xdece bytes) 2024-09-13T07:03:32.093 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:32.093 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:32.093 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:32.093 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:32.095 INFO:teuthology.orchestra.run.smithi118.stdout:5607 trunc from 0xb743959 to 0xce1e37c 2024-09-13T07:03:32.104 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:32.105 INFO:teuthology.orchestra.run.smithi118.stdout:5608 read 0x7cb857 thru 0x7d105e (0x5808 bytes) 2024-09-13T07:03:32.105 INFO:teuthology.orchestra.run.smithi118.stdout:5610 read 0x10ccea0 thru 0x10d1c5c (0x4dbd bytes) 2024-09-13T07:03:32.105 INFO:teuthology.orchestra.run.smithi118.stdout:5611 punch from 0x72eb485 to 0x72f3f37, (0x8ab2 bytes) 2024-09-13T07:03:32.116 INFO:teuthology.orchestra.run.smithi118.stdout:5612 trunc from 0xce1e37c to 0x6784866 2024-09-13T07:03:32.164 INFO:teuthology.orchestra.run.smithi118.stdout:5613 write 0xd25eb70 thru 0xd261ab0 (0x2f41 bytes) 2024-09-13T07:03:32.167 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:32.163+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:32.171 INFO:teuthology.orchestra.run.smithi118.stdout:5614 write 0x3ae3a8d thru 0x3ae48d0 (0xe44 bytes) 2024-09-13T07:03:32.198 INFO:teuthology.orchestra.run.smithi118.stdout:5617 trunc from 0xd261ab1 to 0x30b222 2024-09-13T07:03:32.220 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:32.220 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:32.220 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:32.220 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:32.221 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:32.231 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:32.259 INFO:teuthology.orchestra.run.smithi118.stdout:5618 trunc from 0x30b222 to 0xe0b839e 2024-09-13T07:03:32.267 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:32.266+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:32.276 INFO:teuthology.orchestra.run.smithi118.stdout:5619 trunc from 0xe0b839e to 0xae559ae 2024-09-13T07:03:32.287 INFO:teuthology.orchestra.run.smithi118.stdout:5620 read 0x17a738e thru 0x17b64e7 (0xf15a bytes) 2024-09-13T07:03:32.287 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:32.287 INFO:teuthology.orchestra.run.smithi118.stdout:5622 punch from 0x21151d6 to 0x211e875, (0x969f bytes) 2024-09-13T07:03:32.287 INFO:teuthology.orchestra.run.smithi118.stdout:5623 write 0x1b61936 thru 0x1b63296 (0x1961 bytes) 2024-09-13T07:03:32.290 INFO:teuthology.orchestra.run.smithi118.stdout:5625 punch from 0x3f9f081 to 0x3facbef, (0xdb6e bytes) 2024-09-13T07:03:32.290 INFO:teuthology.orchestra.run.smithi118.stdout:5626 writesame 0x32e3537 thru 0x32e4ca4 data_size 0x1(0x176e bytes) 2024-09-13T07:03:32.300 INFO:teuthology.orchestra.run.smithi118.stdout:5627 read 0x9d7f539 thru 0x9d87870 (0x8338 bytes) 2024-09-13T07:03:32.300 INFO:teuthology.orchestra.run.smithi118.stdout:5629 write 0x4f2606f thru 0x4f2ce2d (0x6dbf bytes) 2024-09-13T07:03:32.306 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:32.307 INFO:teuthology.orchestra.run.smithi118.stdout:5631 read 0x9482a07 thru 0x9491550 (0xeb4a bytes) 2024-09-13T07:03:32.320 INFO:teuthology.orchestra.run.smithi118.stdout:5633 writesame 0xb3df4d3 thru 0xb3e6db2 data_size 0x10(0x78e0 bytes) 2024-09-13T07:03:32.320 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:32.320 INFO:teuthology.orchestra.run.smithi118.stdout:5635 write 0x235d136 thru 0x2360a49 (0x3914 bytes) 2024-09-13T07:03:32.326 INFO:teuthology.orchestra.run.smithi118.stdout:5636 punch from 0xaff841e to 0xb005e67, (0xda49 bytes) 2024-09-13T07:03:32.326 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:32.326 INFO:teuthology.orchestra.run.smithi118.stdout:5638 read 0xb19d3ea thru 0xb19deff (0xb16 bytes) 2024-09-13T07:03:32.326 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:32.326 INFO:teuthology.orchestra.run.smithi118.stdout:5640 punch from 0x67437b5 to 0x674ce05, (0x9650 bytes) 2024-09-13T07:03:32.326 INFO:teuthology.orchestra.run.smithi118.stdout:5641 read 0x9d4979e thru 0x9d540b7 (0xa91a bytes) 2024-09-13T07:03:32.326 INFO:teuthology.orchestra.run.smithi118.stdout:5642 punch from 0x88019ed to 0x8803177, (0x178a bytes) 2024-09-13T07:03:32.326 INFO:teuthology.orchestra.run.smithi118.stdout:5646 read 0x4ce5612 thru 0x4cf4643 (0xf032 bytes) 2024-09-13T07:03:32.326 INFO:teuthology.orchestra.run.smithi118.stdout:5647 read 0xa77546e thru 0xa780026 (0xabb9 bytes) 2024-09-13T07:03:32.326 INFO:teuthology.orchestra.run.smithi118.stdout:5648 read 0x4e8fefd thru 0x4e903d1 (0x4d5 bytes) 2024-09-13T07:03:32.339 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:32.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736909201000003, 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:03:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:03:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:03:32.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:03:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:03:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:03:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:03:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:03:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:03:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:03:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:03:32.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:03:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:03:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736878099999998, 2024-09-13T07:03:32.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:03:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:03:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:32.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736855786, 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.350 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:32.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:03:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:03:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:03:32.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:03:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:03:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:03:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:03:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:03:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:03:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:03:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:03:32.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:03:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:03:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:03:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:03:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736832755000002, 2024-09-13T07:03:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:03:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:03:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.356 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:03:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:03:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:03:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:03:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736808205000003, 2024-09-13T07:03:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:03:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:03:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.361 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:03:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:03:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:03:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:03:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:03:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:03:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:03:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:03:32.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:03:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:03:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:03:32.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:03:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:03:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:03:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.367 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736772496, 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.368 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:03:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:03:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:03:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:03:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:03:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:03:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:03:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:03:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:03:32.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736747817000001, 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:03:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:03:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:03:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:03:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:03:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:03:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:03:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:03:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:03:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:03:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736714542999998, 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:03:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:03:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:03:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:03:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:03:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736687754999998, 2024-09-13T07:03:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:03:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:03:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:03:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:03:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:03:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:03:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:03:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:03:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:03:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:03:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:03:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:03:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:03:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:03:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:03:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:03:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:03:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736663200999999, 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:03:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:03:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736633431999998, 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:03:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:03:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:03:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:03:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:03:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:03:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:03:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:03:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:03:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:03:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:03:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:03:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:03:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:03:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:03:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736573589000002, 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:03:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:03:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:03:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:03:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:03:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736506738999999, 2024-09-13T07:03:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:03:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:03:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:03:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:03:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:03:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:03:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:03:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:03:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:03:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:03:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:03:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:03:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:03:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:03:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:03:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:03:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:03:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736474237000003, 2024-09-13T07:03:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:03:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:03:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:03:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:03:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:03:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:03:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:03:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:03:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:03:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:03:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736439615999998, 2024-09-13T07:03:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:03:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:03:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:03:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:03:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:03:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:03:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:03:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:03:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736389385000003, 2024-09-13T07:03:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:03:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:03:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:03:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:03:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:03:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:03:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:03:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736341199000002, 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:03:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:03:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:03:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:03:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:03:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736314561, 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.736267726000001, 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:03:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.732913308999997, 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:03:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:03:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:03:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:03:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:03:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:03:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:03:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:03:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:03:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.444 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:32.445 INFO:teuthology.orchestra.run.smithi118.stdout:5649 write 0xc51e234 thru 0xc52b5c6 (0xd393 bytes) 2024-09-13T07:03:32.445 INFO:teuthology.orchestra.run.smithi118.stdout:5650 write 0x89cb865 thru 0x89d5ce4 (0xa480 bytes) 2024-09-13T07:03:32.446 INFO:teuthology.orchestra.run.smithi118.stdout:5651 writesame 0x1411423 thru 0x141948b data_size 0x1(0x8069 bytes) 2024-09-13T07:03:32.446 INFO:tasks.ceph.mon.a.smithi118.stderr:2024-09-13T07:03:32.371+0000 7fd1ec319700 -1 mon.a@0(leader).osd e137 definitely_dead 0 2024-09-13T07:03:32.446 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:03:32.447 INFO:teuthology.orchestra.run.smithi118.stdout:5652 write 0x3128ea7 thru 0x312df8a (0x50e4 bytes) 2024-09-13T07:03:32.447 INFO:teuthology.orchestra.run.smithi118.stdout:5653 write 0xdc200c7 thru 0xdc2023e (0x178 bytes) 2024-09-13T07:03:32.447 INFO:teuthology.orchestra.run.smithi118.stdout:5654 write 0x361b609 thru 0x36207b9 (0x51b1 bytes) 2024-09-13T07:03:32.447 INFO:teuthology.orchestra.run.smithi118.stdout:5655 writesame 0xbb64f89 thru 0xbb6e15b data_size 0x1(0x91d3 bytes) 2024-09-13T07:03:32.448 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:32.443+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:32.451 INFO:teuthology.orchestra.run.smithi118.stdout:5656 write 0x3e2bea5 thru 0x3e349fe (0x8b5a bytes) 2024-09-13T07:03:32.454 INFO:teuthology.orchestra.run.smithi118.stdout:5657 write 0x918bb9f thru 0x9195c5a (0xa0bc bytes) 2024-09-13T07:03:32.457 INFO:teuthology.orchestra.run.smithi118.stdout:5658 punch from 0x783da47 to 0x7843938, (0x5ef1 bytes) 2024-09-13T07:03:32.457 INFO:teuthology.orchestra.run.smithi118.stdout:5659 read 0x3603abf thru 0x36041b2 (0x6f4 bytes) 2024-09-13T07:03:32.466 INFO:teuthology.orchestra.run.smithi118.stdout:5660 trunc from 0xdc2023f to 0xafd5fb2 2024-09-13T07:03:32.490 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:32.490 INFO:teuthology.orchestra.run.smithi118.stdout:5662 trunc from 0xafd5fb2 to 0xad617ac 2024-09-13T07:03:32.505 INFO:teuthology.orchestra.run.smithi118.stdout:5663 punch from 0x8b6fff6 to 0x8b74b0f, (0x4b19 bytes) 2024-09-13T07:03:32.505 INFO:teuthology.orchestra.run.smithi118.stdout:5667 read 0x1058a10 thru 0x1068934 (0xff25 bytes) 2024-09-13T07:03:32.506 INFO:teuthology.orchestra.run.smithi118.stdout:5668 write 0x1e5d7fb thru 0x1e63b3b (0x6341 bytes) 2024-09-13T07:03:32.531 INFO:teuthology.orchestra.run.smithi118.stdout:5669 writesame 0xdca9226 thru 0xdcb240f data_size 0x13(0x91ea bytes) 2024-09-13T07:03:32.542 INFO:teuthology.orchestra.run.smithi118.stdout:5670 write 0xe2f7834 thru 0xe2f8fce (0x179b bytes) 2024-09-13T07:03:32.546 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:32.543+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:32.550 INFO:teuthology.orchestra.run.smithi118.stdout:5672 writesame 0x3a88c00 thru 0x3a91e13 data_size 0x2(0x9214 bytes) 2024-09-13T07:03:32.563 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:32.564 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:32.564 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:32.564 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:32.571 INFO:teuthology.orchestra.run.smithi118.stdout:5673 write 0xd3eef60 thru 0xd3f7114 (0x81b5 bytes) 2024-09-13T07:03:32.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:03:32.588 INFO:teuthology.orchestra.run.smithi118.stdout:5674 trunc from 0xe2f8fcf to 0xab66658 2024-09-13T07:03:32.611 INFO:teuthology.orchestra.run.smithi118.stdout:5676 write 0x7368ac9 thru 0x7376ebf (0xe3f7 bytes) 2024-09-13T07:03:32.618 INFO:teuthology.orchestra.run.smithi118.stdout:5677 writesame 0x78f8ffe thru 0x78fc141 data_size 0x6(0x3144 bytes) 2024-09-13T07:03:32.623 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:32.632 INFO:teuthology.orchestra.run.smithi118.stdout:5680 trunc from 0xab66658 to 0x6f16569 2024-09-13T07:03:32.647 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:32.643+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:32.681 INFO:teuthology.orchestra.run.smithi118.stdout:5681 write 0xc51eebd thru 0xc528e86 (0x9fca bytes) 2024-09-13T07:03:32.686 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:32.686 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:32.686 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:32.686 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:32.686 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:32.687 INFO:teuthology.orchestra.run.smithi118.stdout:5683 punch from 0x93461d7 to 0x9346e44, (0xc6d bytes) 2024-09-13T07:03:32.688 INFO:teuthology.orchestra.run.smithi118.stdout:5684 write 0x9e1039b thru 0x9e10d23 (0x989 bytes) 2024-09-13T07:03:32.697 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:03:32.711 INFO:teuthology.orchestra.run.smithi118.stdout:5685 trunc from 0xc528e87 to 0x3b774e0 2024-09-13T07:03:32.748 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:32.745+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:32.765 INFO:teuthology.orchestra.run.smithi118.stdout:5686 write 0xbffe03a thru 0xc00d60a (0xf5d1 bytes) 2024-09-13T07:03:32.774 INFO:teuthology.orchestra.run.smithi118.stdout:5687 read 0x82886ad thru 0x828a61a (0x1f6e bytes) 2024-09-13T07:03:32.775 INFO:teuthology.orchestra.run.smithi118.stdout:5688 trunc from 0xc00d60b to 0xba99f02 2024-09-13T07:03:32.797 INFO:teuthology.orchestra.run.smithi118.stdout:5689 writesame 0xa5ffe thru 0xb420a data_size 0x7(0xe20d bytes) 2024-09-13T07:03:32.809 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:03:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.689129662, 2024-09-13T07:03:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:03:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:03:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:03:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:03:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:03:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129532 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.192726774999997, 2024-09-13T07:03:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89889205500000002, 2024-09-13T07:03:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129532 2024-09-13T07:03:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614994+0000", 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.044e-06 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615000+0000", 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9229999999999996e-06 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615005+0000", 2024-09-13T07:03:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615060+0000", 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4561000000000002e-05 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615069+0000", 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1540000000000008e-06 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513869+0000", 2024-09-13T07:03:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89879977700000002 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513883+0000", 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4361e-05 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129533 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.192631442, 2024-09-13T07:03:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.899270177, 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129533 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615090+0000", 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.856e-06 2024-09-13T07:03:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615096+0000", 2024-09-13T07:03:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.703e-06 2024-09-13T07:03:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615102+0000", 2024-09-13T07:03:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.378e-06 2024-09-13T07:03:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615140+0000", 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7966999999999997e-05 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615147+0000", 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8800000000000002e-06 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514342+0000", 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919565300000004 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514357+0000", 2024-09-13T07:03:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4355000000000001e-05 2024-09-13T07:03:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129534 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.192558544000001, 2024-09-13T07:03:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89923080600000005, 2024-09-13T07:03:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129534 2024-09-13T07:03:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615162+0000", 2024-09-13T07:03:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3569999999999998e-06 2024-09-13T07:03:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615169+0000", 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.455e-06 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615175+0000", 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3679999999999998e-06 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615212+0000", 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6733999999999998e-05 2024-09-13T07:03:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615218+0000", 2024-09-13T07:03:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7569999999999998e-06 2024-09-13T07:03:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514383+0000", 2024-09-13T07:03:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916408000000003 2024-09-13T07:03:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514390+0000", 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5429999999999996e-06 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129535 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.192485128000001, 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919205499999999, 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129535 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615232+0000", 2024-09-13T07:03:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615236+0000", 2024-09-13T07:03:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615244+0000", 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7369999999999995e-06 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615250+0000", 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2270000000000001e-06 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4627000000000002e-05 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5089999999999993e-06 2024-09-13T07:03:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514412+0000", 2024-09-13T07:03:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89907890599999996 2024-09-13T07:03:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514425+0000", 2024-09-13T07:03:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3138e-05 2024-09-13T07:03:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129536 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.192451650999999, 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89918503500000002, 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129536 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615268+0000", 2024-09-13T07:03:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8370000000000001e-06 2024-09-13T07:03:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615288+0000", 2024-09-13T07:03:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9945999999999998e-05 2024-09-13T07:03:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615294+0000", 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3340000000000001e-06 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615374+0000", 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9988999999999996e-05 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615381+0000", 2024-09-13T07:03:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514444+0000", 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89906347499999995 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514451+0000", 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7630000000000003e-06 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129537 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.192403099000003, 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916833399999996, 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129537 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615314+0000", 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615317+0000", 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1410000000000001e-06 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3589999999999996e-06 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615329+0000", 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615408+0000", 2024-09-13T07:03:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8678999999999997e-05 2024-09-13T07:03:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615414+0000", 2024-09-13T07:03:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:03:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514471+0000", 2024-09-13T07:03:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905719200000001 2024-09-13T07:03:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514483+0000", 2024-09-13T07:03:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1691999999999999e-05 2024-09-13T07:03:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129538 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.192384560000001, 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89917603999999995, 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129538 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615335+0000", 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4090000000000001e-06 2024-09-13T07:03:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615346+0000", 2024-09-13T07:03:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0866000000000001e-05 2024-09-13T07:03:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615424+0000", 2024-09-13T07:03:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7628999999999998e-05 2024-09-13T07:03:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615459+0000", 2024-09-13T07:03:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5046999999999998e-05 2024-09-13T07:03:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615466+0000", 2024-09-13T07:03:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7610000000000002e-06 2024-09-13T07:03:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514503+0000", 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89903750299999996 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514509+0000", 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2140000000000001e-06 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129539 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.192239809, 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905735799999997, 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129539 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615477+0000", 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615481+0000", 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6490000000000001e-06 2024-09-13T07:03:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615488+0000", 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4359999999999999e-06 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615493+0000", 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3120000000000001e-06 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615529+0000", 2024-09-13T07:03:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6468999999999999e-05 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615536+0000", 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9789999999999996e-06 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514529+0000", 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89899276400000006 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514535+0000", 2024-09-13T07:03:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2920000000000001e-06 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129540 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.192168815000002, 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89901438700000003, 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129540 2024-09-13T07:03:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615548+0000", 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615552+0000", 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8349999999999997e-06 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615559+0000", 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-13T07:03:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615564+0000", 2024-09-13T07:03:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2360000000000004e-06 2024-09-13T07:03:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615600+0000", 2024-09-13T07:03:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6637000000000002e-05 2024-09-13T07:03:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615607+0000", 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.669e-06 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514557+0000", 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89894939399999996 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514563+0000", 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8560000000000001e-06 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129541 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.192096202999998, 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89896824600000003, 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129541 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615621+0000", 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615625+0000", 2024-09-13T07:03:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-13T07:03:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615631+0000", 2024-09-13T07:03:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3890000000000002e-06 2024-09-13T07:03:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615636+0000", 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3489999999999999e-06 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615671+0000", 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4570000000000003e-05 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615678+0000", 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8510000000000001e-06 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514583+0000", 2024-09-13T07:03:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89890530599999996 2024-09-13T07:03:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514590+0000", 2024-09-13T07:03:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7469999999999999e-06 2024-09-13T07:03:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129542 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.192022713, 2024-09-13T07:03:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89892092400000001, 2024-09-13T07:03:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129542 2024-09-13T07:03:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615698+0000", 2024-09-13T07:03:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.664e-06 2024-09-13T07:03:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615705+0000", 2024-09-13T07:03:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615711+0000", 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6470000000000003e-06 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615752+0000", 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0389999999999998e-05 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615759+0000", 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9230000000000002e-06 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514609+0000", 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89884984899999998 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514616+0000", 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7299999999999999e-06 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129543 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.19194384, 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89886872600000001, 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129543 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615773+0000", 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615777+0000", 2024-09-13T07:03:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1860000000000002e-06 2024-09-13T07:03:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615785+0000", 2024-09-13T07:03:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2710000000000003e-06 2024-09-13T07:03:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615790+0000", 2024-09-13T07:03:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3619999999999997e-06 2024-09-13T07:03:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615828+0000", 2024-09-13T07:03:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8059999999999998e-05 2024-09-13T07:03:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615836+0000", 2024-09-13T07:03:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8809999999999996e-06 2024-09-13T07:03:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514636+0000", 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89880009599999999 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514643+0000", 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129544 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.191868317999997, 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89881970799999999, 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129544 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615853+0000", 2024-09-13T07:03:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8600000000000003e-06 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615860+0000", 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1709999999999996e-06 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615867+0000", 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7020000000000004e-06 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615905+0000", 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8265e-05 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615913+0000", 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514663+0000", 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89874952500000005 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514669+0000", 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129545 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.191790144999999, 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89876792599999999, 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129545 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615927+0000", 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615931+0000", 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8010000000000003e-06 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615939+0000", 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6310000000000003e-06 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615947+0000", 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7090000000000002e-06 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615985+0000", 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8389000000000001e-05 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615993+0000", 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2209999999999997e-06 2024-09-13T07:03:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514689+0000", 2024-09-13T07:03:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89869535199999995 2024-09-13T07:03:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514696+0000", 2024-09-13T07:03:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8179999999999998e-06 2024-09-13T07:03:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129546 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.191710057000002, 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89872448199999999, 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129546 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616007+0000", 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616011+0000", 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1659999999999996e-06 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616018+0000", 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0829999999999998e-06 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616024+0000", 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8170000000000004e-06 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616063+0000", 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8708000000000003e-05 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616070+0000", 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6299999999999998e-06 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514721+0000", 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89865028800000002 2024-09-13T07:03:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514732+0000", 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1452e-05 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129547 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.191634729, 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89868529799999997, 2024-09-13T07:03:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129547 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616086+0000", 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.394e-06 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616093+0000", 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1130000000000003e-06 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616100+0000", 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1520000000000003e-06 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616138+0000", 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8084000000000003e-05 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616146+0000", 2024-09-13T07:03:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3010000000000001e-06 2024-09-13T07:03:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89861259299999996 2024-09-13T07:03:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514768+0000", 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0175e-05 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129548 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.191550669999998, 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89862847899999998, 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129548 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616166+0000", 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616170+0000", 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8619999999999996e-06 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616177+0000", 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0450000000000003e-06 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616183+0000", 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.139e-06 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616221+0000", 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7320000000000002e-05 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616228+0000", 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2980000000000003e-06 2024-09-13T07:03:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514789+0000", 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89856044599999996 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514796+0000", 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0809999999999996e-06 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129549 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.191477206999998, 2024-09-13T07:03:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89859381900000002, 2024-09-13T07:03:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129549 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616240+0000", 2024-09-13T07:03:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616244+0000", 2024-09-13T07:03:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4149999999999999e-06 2024-09-13T07:03:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616251+0000", 2024-09-13T07:03:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-13T07:03:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616256+0000", 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3299999999999998e-06 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616302+0000", 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5646999999999999e-05 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616309+0000", 2024-09-13T07:03:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5530000000000004e-06 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514825+0000", 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89851528400000003 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514834+0000", 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8349999999999999e-06 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129550 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.191392827999998, 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89854296199999995, 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129550 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616324+0000", 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616328+0000", 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7479999999999999e-06 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616335+0000", 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9280000000000004e-06 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616342+0000", 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2690000000000002e-06 2024-09-13T07:03:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616380+0000", 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616387+0000", 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.075e-06 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514861+0000", 2024-09-13T07:03:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89847450799999995 2024-09-13T07:03:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514868+0000", 2024-09-13T07:03:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6050000000000003e-06 2024-09-13T07:03:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:32.909 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:32.911 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:32.912 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:32.909+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:32.912 INFO:teuthology.orchestra.run.smithi118.stdout:5690 trunc from 0xba99f02 to 0xbbe9585 2024-09-13T07:03:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:5691 writesame 0xc8a9fd0 thru 0xc8ad4e3 data_size 0x2b(0x3514 bytes) 2024-09-13T07:03:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:5693 write 0xa09b446 thru 0xa09f280 (0x3e3b bytes) 2024-09-13T07:03:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:5695 write 0x1de0e2d thru 0x1de2d07 (0x1edb bytes) 2024-09-13T07:03:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:5696 punch from 0x94ce482 to 0x94d1f69, (0x3ae7 bytes) 2024-09-13T07:03:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:5697 punch from 0x9ee42e4 to 0x9ef19e2, (0xd6fe bytes) 2024-09-13T07:03:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:5699 trunc from 0xc8ad4e4 to 0xded0aca 2024-09-13T07:03:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:5701 write 0xb32f4a5 thru 0xb33a059 (0xabb5 bytes) 2024-09-13T07:03:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:5703 trunc from 0xded0aca to 0x82df618 2024-09-13T07:03:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:5705 read 0x59c26bf thru 0x59cedda (0xc71c bytes) 2024-09-13T07:03:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:32.913 INFO:teuthology.orchestra.run.smithi118.stdout:5708 trunc from 0x82df618 to 0x1b1e0fe 2024-09-13T07:03:32.939 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:32.939 INFO:teuthology.orchestra.run.smithi118.stdout:5710 punch from 0x18a797f to 0x18b5663, (0xdce4 bytes) 2024-09-13T07:03:32.962 INFO:teuthology.orchestra.run.smithi118.stdout:5711 write 0x6359c0e thru 0x635f104 (0x54f7 bytes) 2024-09-13T07:03:32.971 INFO:teuthology.orchestra.run.smithi118.stderr:marked in osd.1. 2024-09-13T07:03:32.972 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:03:32.973 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:32.973 INFO:teuthology.orchestra.run.smithi118.stdout:5714 write 0x3a486be thru 0x3a5860f (0xff52 bytes) 2024-09-13T07:03:32.983 INFO:teuthology.orchestra.run.smithi118.stdout:5715 punch from 0x1ec707d to 0x1ed3814, (0xc797 bytes) 2024-09-13T07:03:32.990 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-13T07:03:32.994 INFO:teuthology.orchestra.run.smithi118.stdout:5717 write 0x6b13c86 thru 0x6b18106 (0x4481 bytes) 2024-09-13T07:03:33.012 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:33.009+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:33.027 INFO:teuthology.orchestra.run.smithi118.stdout:5718 write 0xd13aa76 thru 0xd1460d9 (0xb664 bytes) 2024-09-13T07:03:33.034 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.034 INFO:teuthology.orchestra.run.smithi118.stdout:5720 write 0x8f2af56 thru 0x8f350d2 (0xa17d bytes) 2024-09-13T07:03:33.038 INFO:teuthology.orchestra.run.smithi118.stdout:5721 write 0x948f087 thru 0x9495f92 (0x6f0c bytes) 2024-09-13T07:03:33.045 INFO:teuthology.orchestra.run.smithi118.stdout:5722 read 0x5aeee4d thru 0x5af0ca6 (0x1e5a bytes) 2024-09-13T07:03:33.045 INFO:teuthology.orchestra.run.smithi118.stdout:5726 write 0x3c3b1fc thru 0x3c462eb (0xb0f0 bytes) 2024-09-13T07:03:33.050 INFO:teuthology.orchestra.run.smithi118.stdout:5727 write 0x54cfe3a thru 0x54d06a5 (0x86c bytes) 2024-09-13T07:03:33.053 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.053 INFO:teuthology.orchestra.run.smithi118.stdout:5729 write 0x5d65d02 thru 0x5d6d988 (0x7c87 bytes) 2024-09-13T07:03:33.059 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.059 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.059 INFO:teuthology.orchestra.run.smithi118.stdout:5736 read 0x3fc03b3 thru 0x3fc79be (0x760c bytes) 2024-09-13T07:03:33.060 INFO:teuthology.orchestra.run.smithi118.stdout:5737 read 0xc65a4f7 thru 0xc661c85 (0x778f bytes) 2024-09-13T07:03:33.060 INFO:teuthology.orchestra.run.smithi118.stdout:5738 write 0x85849ee thru 0x859147d (0xca90 bytes) 2024-09-13T07:03:33.066 INFO:teuthology.orchestra.run.smithi118.stdout:5741 read 0x3be2bc thru 0x3beffa (0xd3f bytes) 2024-09-13T07:03:33.068 INFO:teuthology.orchestra.run.smithi118.stdout:5742 punch from 0x95514a5 to 0x955e162, (0xccbd bytes) 2024-09-13T07:03:33.068 INFO:teuthology.orchestra.run.smithi118.stdout:5743 write 0x8e72fd4 thru 0x8e7afe6 (0x8013 bytes) 2024-09-13T07:03:33.071 INFO:teuthology.orchestra.run.smithi118.stdout:5746 punch from 0x329ebbc to 0x32ada2e, (0xee72 bytes) 2024-09-13T07:03:33.071 INFO:teuthology.orchestra.run.smithi118.stdout:5749 write 0x15624a thru 0x15db4b (0x7902 bytes) 2024-09-13T07:03:33.075 INFO:teuthology.orchestra.run.smithi118.stdout:5750 punch from 0xa43d8a3 to 0xa440807, (0x2f64 bytes) 2024-09-13T07:03:33.075 INFO:teuthology.orchestra.run.smithi118.stdout:5751 read 0x989c280 thru 0x98a2429 (0x61aa bytes) 2024-09-13T07:03:33.075 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:33.075 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:33.075 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:33.075 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:33.086 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:33.092 INFO:teuthology.orchestra.run.smithi118.stdout:5752 write 0xe147091 thru 0xe1521bb (0xb12b bytes) 2024-09-13T07:03:33.097 INFO:teuthology.orchestra.run.smithi118.stdout:5754 read 0xaefbb60 thru 0xaefd350 (0x17f1 bytes) 2024-09-13T07:03:33.097 INFO:teuthology.orchestra.run.smithi118.stdout:5755 writesame 0xaa43c1b thru 0xaa45edb data_size 0x29(0x22c1 bytes) 2024-09-13T07:03:33.104 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.107 INFO:teuthology.orchestra.run.smithi118.stdout:5757 trunc from 0xe1521bc to 0xce809a0 2024-09-13T07:03:33.122 INFO:teuthology.orchestra.run.smithi118.stdout:5758 read 0x7110a97 thru 0x7116533 (0x5a9d bytes) 2024-09-13T07:03:33.123 INFO:teuthology.orchestra.run.smithi118.stdout:5760 punch from 0x6783da7 to 0x67891ab, (0x5404 bytes) 2024-09-13T07:03:33.123 INFO:teuthology.orchestra.run.smithi118.stdout:5761 punch from 0x5c21250 to 0x5c2fcd7, (0xea87 bytes) 2024-09-13T07:03:33.123 INFO:teuthology.orchestra.run.smithi118.stdout:5762 punch from 0x8829ff5 to 0x882fc62, (0x5c6d bytes) 2024-09-13T07:03:33.123 INFO:teuthology.orchestra.run.smithi118.stdout:5763 punch from 0x5c67bc8 to 0x5c70206, (0x863e bytes) 2024-09-13T07:03:33.123 INFO:teuthology.orchestra.run.smithi118.stdout:5765 read 0xc1a81ca thru 0xc1b3baf (0xb9e6 bytes) 2024-09-13T07:03:33.123 INFO:teuthology.orchestra.run.smithi118.stdout:5766 read 0xa163e87 thru 0xa1678f5 (0x3a6f bytes) 2024-09-13T07:03:33.124 INFO:teuthology.orchestra.run.smithi118.stdout:5767 write 0x877b3ab thru 0x8789a4d (0xe6a3 bytes) 2024-09-13T07:03:33.128 INFO:teuthology.orchestra.run.smithi118.stdout:5770 write 0x1eaeb93 thru 0x1eb6883 (0x7cf1 bytes) 2024-09-13T07:03:33.130 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:33.129+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:33.131 INFO:teuthology.orchestra.run.smithi118.stdout:5771 write 0x36fb6a0 thru 0x3709672 (0xdfd3 bytes) 2024-09-13T07:03:33.151 INFO:teuthology.orchestra.run.smithi118.stdout:5772 writesame 0xed3f845 thru 0xed4095a data_size 0x1b(0x1116 bytes) 2024-09-13T07:03:33.154 INFO:teuthology.orchestra.run.smithi118.stdout:5774 writesame 0x417072d thru 0x4177594 data_size 0x4(0x6e68 bytes) 2024-09-13T07:03:33.163 INFO:teuthology.orchestra.run.smithi118.stdout:5775 read 0x8f69e46 thru 0x8f744f5 (0xa6b0 bytes) 2024-09-13T07:03:33.164 INFO:teuthology.orchestra.run.smithi118.stdout:5776 punch from 0xe21e033 to 0xe221645, (0x3612 bytes) 2024-09-13T07:03:33.164 INFO:teuthology.orchestra.run.smithi118.stdout:5777 read 0xaea6046 thru 0xaeb3c27 (0xdbe2 bytes) 2024-09-13T07:03:33.164 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.187 INFO:teuthology.orchestra.run.smithi118.stdout:5780 trunc from 0xed4095b to 0x4333d56 2024-09-13T07:03:33.188 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:33.188 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:33.188 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:33.188 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:33.188 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:33.200 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:33.213 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:33.209+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:33.229 INFO:teuthology.orchestra.run.smithi118.stdout:5781 write 0x7a2d181 thru 0x7a3228a (0x510a bytes) 2024-09-13T07:03:33.263 INFO:teuthology.orchestra.run.smithi118.stdout:5782 write 0xe9fb30d thru 0xea044db (0x91cf bytes) 2024-09-13T07:03:33.273 INFO:teuthology.orchestra.run.smithi118.stdout:5785 write 0x82042e thru 0x82eed5 (0xeaa8 bytes) 2024-09-13T07:03:33.276 INFO:teuthology.orchestra.run.smithi118.stdout:5786 read 0x6b4ebc0 thru 0x6b4f7e8 (0xc29 bytes) 2024-09-13T07:03:33.276 INFO:teuthology.orchestra.run.smithi118.stdout:5787 writesame 0x859a5f1 thru 0x859f967 data_size 0x1(0x5377 bytes) 2024-09-13T07:03:33.300 INFO:teuthology.orchestra.run.smithi118.stdout:5789 write 0x842ba30 thru 0x842e625 (0x2bf6 bytes) 2024-09-13T07:03:33.303 INFO:teuthology.orchestra.run.smithi118.stdout:5790 write 0x1dee7ad thru 0x1df1d4b (0x359f bytes) 2024-09-13T07:03:33.306 INFO:teuthology.orchestra.run.smithi118.stdout:5791 punch from 0x1b72130 to 0x1b77ccf, (0x5b9f bytes) 2024-09-13T07:03:33.307 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 127.245934152, 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:03:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:03:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93979 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.688174838999998, 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042253801000000001, 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93979 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616680+0000", 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616683+0000", 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2820000000000001e-06 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616691+0000", 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0779999999999996e-06 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616698+0000", 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0119999999999999e-06 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658799+0000", 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042100763999999999 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658869+0000", 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0065000000000001e-05 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658934+0000", 2024-09-13T07:03:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5071000000000001e-05 2024-09-13T07:03:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.688150273999995, 2024-09-13T07:03:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:03:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:03:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.688127307000002, 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:03:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:03:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:03:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:03:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:03:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:03:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.688110128999995, 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:03:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.688096263999995, 2024-09-13T07:03:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:03:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:03:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:03:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:03:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:03:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:03:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:03:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:03:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:03:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:03:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.688083118999998, 2024-09-13T07:03:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:03:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:03:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.688069912000003, 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.688055844999994, 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:03:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:03:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:03:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:03:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:03:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:03:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.688035830999993, 2024-09-13T07:03:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:03:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:03:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:03:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:03:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:03:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:03:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.688016081000001, 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:03:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.687991275000002, 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.687967009999994, 2024-09-13T07:03:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:03:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:03:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:03:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:03:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.687925277000005, 2024-09-13T07:03:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:03:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:03:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.687888987999997, 2024-09-13T07:03:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:03:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:03:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:03:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:03:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:03:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:03:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:03:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:03:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.244785655999998, 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:03:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:03:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:03:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:03:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:03:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:03:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.244768774000001, 2024-09-13T07:03:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:03:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:03:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:03:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.243718217000001, 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:03:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:03:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.242043668000001, 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:03:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:03:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.721461278, 2024-09-13T07:03:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:03:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:03:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:03:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:03:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:03:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.397 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:33.398 INFO:teuthology.orchestra.run.smithi118.stdout:5794 trunc from 0xea044dc to 0x40a4d26 2024-09-13T07:03:33.399 INFO:teuthology.orchestra.run.smithi118.stdout:5796 writesame 0xf909ef thru 0xf94794 data_size 0xd(0x3da6 bytes) 2024-09-13T07:03:33.399 INFO:teuthology.orchestra.run.smithi118.stdout:5797 write 0x6202d93 thru 0x620ec8f (0xbefd bytes) 2024-09-13T07:03:33.399 INFO:teuthology.orchestra.run.smithi118.stdout:5799 trunc from 0x620ec90 to 0xbc05754 2024-09-13T07:03:33.399 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:33.395+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:33.399 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:33.405 INFO:teuthology.orchestra.run.smithi118.stdout:5800 writesame 0x322b082 thru 0x3232a5a data_size 0x1(0x79d9 bytes) 2024-09-13T07:03:33.439 INFO:teuthology.orchestra.run.smithi118.stdout:5802 trunc from 0xbc05754 to 0xd244bbb 2024-09-13T07:03:33.454 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.454 INFO:teuthology.orchestra.run.smithi118.stdout:5805 write 0x496f1b0 thru 0x496f812 (0x663 bytes) 2024-09-13T07:03:33.457 INFO:teuthology.orchestra.run.smithi118.stdout:5806 writesame 0x9cd4fb3 thru 0x9ce3d40 data_size 0xd(0xed8e bytes) 2024-09-13T07:03:33.467 INFO:teuthology.orchestra.run.smithi118.stdout:5807 punch from 0x80860d8 to 0x809190d, (0xb835 bytes) 2024-09-13T07:03:33.467 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.467 INFO:teuthology.orchestra.run.smithi118.stdout:5812 write 0x41c8e77 thru 0x41d6cc6 (0xde50 bytes) 2024-09-13T07:03:33.471 INFO:teuthology.orchestra.run.smithi118.stdout:5814 read 0x233c62 thru 0x238286 (0x4625 bytes) 2024-09-13T07:03:33.476 INFO:teuthology.orchestra.run.smithi118.stdout:5815 trunc from 0xd244bbb to 0xbd645bc 2024-09-13T07:03:33.488 INFO:teuthology.orchestra.run.smithi118.stdout:5816 write 0x542bbbe thru 0x542f874 (0x3cb7 bytes) 2024-09-13T07:03:33.491 INFO:teuthology.orchestra.run.smithi118.stdout:5818 writesame 0x77ad562 thru 0x77ae08f data_size 0x9(0xb2e bytes) 2024-09-13T07:03:33.499 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:33.497+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:33.499 INFO:teuthology.orchestra.run.smithi118.stdout:5820 read 0xaf760a6 thru 0xaf81803 (0xb75e bytes) 2024-09-13T07:03:33.500 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.500 INFO:teuthology.orchestra.run.smithi118.stdout:5822 writesame 0x2df0e7a thru 0x2dfda33 data_size 0x59(0xcbba bytes) 2024-09-13T07:03:33.505 INFO:teuthology.orchestra.run.smithi118.stdout:5823 read 0x1b1eb40 thru 0x1b25b76 (0x7037 bytes) 2024-09-13T07:03:33.506 INFO:teuthology.orchestra.run.smithi118.stdout:5824 write 0x296be4e thru 0x2977468 (0xb61b bytes) 2024-09-13T07:03:33.509 INFO:teuthology.orchestra.run.smithi118.stdout:5825 read 0xd6577a thru 0xd72a50 (0xd2d7 bytes) 2024-09-13T07:03:33.510 INFO:teuthology.orchestra.run.smithi118.stdout:5826 read 0x150365 thru 0x15a7a2 (0xa43e bytes) 2024-09-13T07:03:33.521 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:33.521 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:33.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.521 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:169808 3.f 3:f5c6cf0f:::rbd_header.1194e181a50b:head [notify cookie 140335508196848 in=22b out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e138)", 2024-09-13T07:03:33.521 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:33.522397+0000", 2024-09-13T07:03:33.521 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00017663800000000001, 2024-09-13T07:03:33.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021251700000000001, 2024-09-13T07:03:33.521 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 169808 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:33.522397+0000", 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:33.522397+0000", 2024-09-13T07:03:33.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:33.522395+0000", 2024-09-13T07:03:33.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:33.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:33.522405+0000", 2024-09-13T07:03:33.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4630000000000007e-06 2024-09-13T07:03:33.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:33.522425+0000", 2024-09-13T07:03:33.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0341999999999999e-05 2024-09-13T07:03:33.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:33.522441+0000", 2024-09-13T07:03:33.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5500000000000001e-05 2024-09-13T07:03:33.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:33.522494+0000", 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3156999999999999e-05 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:33.522544+0000", 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9926999999999998e-05 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:33.522614+0000", 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0337000000000001e-05 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.526 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.526 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-13T07:03:33.526 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-13T07:03:33.526 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:33.529 INFO:teuthology.orchestra.run.smithi118.stdout:5828 writesame 0xcc1f07e thru 0xcc2b046 data_size 0x1(0xbfc9 bytes) 2024-09-13T07:03:33.529 INFO:teuthology.orchestra.run.smithi118.stdout:5829 write 0x8b4d6f2 thru 0x8b56c30 (0x953f bytes) 2024-09-13T07:03:33.532 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:33.537 INFO:teuthology.orchestra.run.smithi118.stdout:5830 punch from 0x77d1f8b to 0x77d4309, (0x237e bytes) 2024-09-13T07:03:33.537 INFO:teuthology.orchestra.run.smithi118.stdout:5831 write 0x662afea thru 0x6633cb9 (0x8cd0 bytes) 2024-09-13T07:03:33.540 INFO:teuthology.orchestra.run.smithi118.stdout:5833 write 0x8e12e94 thru 0x8e20925 (0xda92 bytes) 2024-09-13T07:03:33.542 INFO:teuthology.orchestra.run.smithi118.stdout:5835 read 0x4f15fc0 thru 0x4f21fd3 (0xc014 bytes) 2024-09-13T07:03:33.543 INFO:teuthology.orchestra.run.smithi118.stdout:5836 read 0x48bfc thru 0x55597 (0xc99c bytes) 2024-09-13T07:03:33.545 INFO:teuthology.orchestra.run.smithi118.stdout:5838 writesame 0xbebd3d2 thru 0xbec4365 data_size 0x4a(0x6f94 bytes) 2024-09-13T07:03:33.569 INFO:teuthology.orchestra.run.smithi118.stdout:5839 trunc from 0xcc2b047 to 0x4d2f548 2024-09-13T07:03:33.592 INFO:teuthology.orchestra.run.smithi118.stdout:5840 read 0x3fa7131 thru 0x3faf208 (0x80d8 bytes) 2024-09-13T07:03:33.600 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:33.597+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:33.610 INFO:teuthology.orchestra.run.smithi118.stdout:5842 write 0x7b01aa0 thru 0x7b066ff (0x4c60 bytes) 2024-09-13T07:03:33.617 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.617 INFO:teuthology.orchestra.run.smithi118.stdout:5844 read 0x56b285a thru 0x56b4c4e (0x23f5 bytes) 2024-09-13T07:03:33.631 INFO:teuthology.orchestra.run.smithi118.stdout:5845 writesame 0x82bcfb6 thru 0x82caffd data_size 0x4(0xe048 bytes) 2024-09-13T07:03:33.636 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:33.636 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:33.636 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:33.636 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:33.636 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:33.642 INFO:teuthology.orchestra.run.smithi118.stdout:5846 trunc from 0x82caffe to 0x41e6825 2024-09-13T07:03:33.646 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:33.664 INFO:teuthology.orchestra.run.smithi118.stdout:5847 read 0x3768eeb thru 0x37754d7 (0xc5ed bytes) 2024-09-13T07:03:33.700 INFO:teuthology.orchestra.run.smithi118.stdout:5848 write 0xe812d40 thru 0xe81a4fd (0x77be bytes) 2024-09-13T07:03:33.700 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:33.697+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:33.707 INFO:teuthology.orchestra.run.smithi118.stdout:5849 read 0xe5d3be0 thru 0xe5e1893 (0xdcb4 bytes) 2024-09-13T07:03:33.707 INFO:teuthology.orchestra.run.smithi118.stdout:5851 write 0x3915a52 thru 0x3923cac (0xe25b bytes) 2024-09-13T07:03:33.710 INFO:teuthology.orchestra.run.smithi118.stdout:5852 read 0x997aa11 thru 0x9983f14 (0x9504 bytes) 2024-09-13T07:03:33.733 INFO:teuthology.orchestra.run.smithi118.stdout:5853 trunc from 0xe81a4fe to 0x3dcd53d 2024-09-13T07:03:33.750 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:03:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 127.69597524700001, 2024-09-13T07:03:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:03:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:03:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:03:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:03:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:03:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:03:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:03:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:03:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:03:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:03:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 95.538094013000006, 2024-09-13T07:03:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:03:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:03:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:03:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:03:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:03:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:03:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 95.512324301999996, 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:03:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:03:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:03:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:03:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:03:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:03:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:03:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:03:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:03:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:03:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:03:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:03:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:03:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:03:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 95.491173136, 2024-09-13T07:03:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:03:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:03:33.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:33.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:03:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:03:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:03:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:03:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:03:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:03:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:03:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:03:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:03:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.374128373000005, 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:03:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:03:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:03:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:03:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:03:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:03:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 67.089131128000005, 2024-09-13T07:03:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:03:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:03:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:03:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:03:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:03:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:03:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.837421990000003, 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.837403956000003, 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.788 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:33.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:03:33.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:03:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:03:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:03:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:03:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:03:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:03:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:03:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:03:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:03:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.837387876000001, 2024-09-13T07:03:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:03:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:03:33.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.837372199999997, 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:03:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.837349277999998, 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:33.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:03:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:33.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:33.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:03:33.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:03:33.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:03:33.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:03:33.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:03:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:03:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:03:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:03:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:03:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:03:33.805 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.837333180999998, 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:03:33.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:03:33.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:03:33.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:03:33.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:03:33.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:03:33.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:03:33.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:03:33.809 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.809 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107067 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.836773164, 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04441055, 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107067 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.810 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912832+0000", 2024-09-13T07:03:33.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9994574 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913379+0000", 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00054719999999999997 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913386+0000", 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4549999999999998e-06 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913393+0000", 2024-09-13T07:03:33.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6259999999999997e-06 2024-09-13T07:03:33.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957776+0000", 2024-09-13T07:03:33.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044383143999999999 2024-09-13T07:03:33.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957785+0000", 2024-09-13T07:03:33.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9220000000000003e-06 2024-09-13T07:03:33.813 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107068 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.836748733, 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044410502999999997, 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.814 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107068 2024-09-13T07:03:33.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.815 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:33.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:33.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:33.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913401+0000", 2024-09-13T07:03:33.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:03:33.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913407+0000", 2024-09-13T07:03:33.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7520000000000002e-06 2024-09-13T07:03:33.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913412+0000", 2024-09-13T07:03:33.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8550000000000001e-06 2024-09-13T07:03:33.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957802+0000", 2024-09-13T07:03:33.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044389606999999998 2024-09-13T07:03:33.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957809+0000", 2024-09-13T07:03:33.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9160000000000001e-06 2024-09-13T07:03:33.818 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.818 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.818 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.818 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:33.818 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:33.818 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.836731569000001, 2024-09-13T07:03:33.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:03:33.819 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.819 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:33.819 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.819 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.819 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.819 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:03:33.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.819 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:33.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:33.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:33.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:03:33.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:03:33.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:03:33.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:03:33.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.823 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.823 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:33.823 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:33.823 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.836688793, 2024-09-13T07:03:33.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:03:33.823 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.823 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:33.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:03:33.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:03:33.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:03:33.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:03:33.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:03:33.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:03:33.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:03:33.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:33.827 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.679671022999997, 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:33.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:03:33.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:03:33.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:03:33.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:03:33.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:03:33.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:03:33.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:03:33.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:03:33.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:03:33.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:03:33.831 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.831 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.831 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.827294092999999, 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:33.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:03:33.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:03:33.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:03:33.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:03:33.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:03:33.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:03:33.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:03:33.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:03:33.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.836 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.837 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.837 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:164179 3.14 e135/67 3:2fc485bd:::rbd_data.11942e4f5fc7.0000000000000002:head v 135'401, mlcod=132'398)", 2024-09-13T07:03:33.837 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:33.837 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8043536390000003, 2024-09-13T07:03:33.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063492174999999998, 2024-09-13T07:03:33.837 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.837 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:33.837 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:33.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:33.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945792+0000", 2024-09-13T07:03:33.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:33.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954811+0000", 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090187940000000001 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954840+0000", 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8659000000000001e-05 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954852+0000", 2024-09-13T07:03:33.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1966e-05 2024-09-13T07:03:33.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954938+0000", 2024-09-13T07:03:33.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6539000000000006e-05 2024-09-13T07:03:33.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954975+0000", 2024-09-13T07:03:33.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6303999999999997e-05 2024-09-13T07:03:33.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009206+0000", 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.054231044999999999 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009286+0000", 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700999999999999e-05 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:165953 3.d 3:b3a20b45:::rbd_data.11942e4f5fc7.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8796117~39260] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:33.841 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:33.842 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.5279060810000002, 2024-09-13T07:03:33.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.095290105, 2024-09-13T07:03:33.842 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:33.842 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:33.842 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:33.842 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:33.842 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:33.842 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 165953 2024-09-13T07:03:33.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.842 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:33.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222240+0000", 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258929+0000", 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.036688401000000002 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258971+0000", 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2212000000000002e-05 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258989+0000", 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7674000000000001e-05 2024-09-13T07:03:33.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259072+0000", 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3656e-05 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259297+0000", 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022450700000000001 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.260463+0000", 2024-09-13T07:03:33.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011665340000000001 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.316214+0000", 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.055750268999999998 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317335+0000", 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011211719999999999 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:33.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317451+0000", 2024-09-13T07:03:33.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011616 2024-09-13T07:03:33.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:33.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:33.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:33.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317532+0000", 2024-09-13T07:03:33.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0860999999999997e-05 2024-09-13T07:03:33.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.847 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:33.847 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:33.848 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:33.850 INFO:teuthology.orchestra.run.smithi118.stdout:5855 writesame 0x2894055 thru 0x289bf46 data_size 0x1(0x7ef2 bytes) 2024-09-13T07:03:33.850 INFO:teuthology.orchestra.run.smithi118.stdout:5856 read 0x15aa8d9 thru 0x15b2b46 (0x826e bytes) 2024-09-13T07:03:33.850 INFO:teuthology.orchestra.run.smithi118.stdout:5857 trunc from 0x3dcd53d to 0xa0eab52 2024-09-13T07:03:33.850 INFO:teuthology.orchestra.run.smithi118.stdout:5858 write 0xca9de76 thru 0xcaa14a4 (0x362f bytes) 2024-09-13T07:03:33.850 INFO:teuthology.orchestra.run.smithi118.stdout:5859 read 0xc34274b thru 0xc34fc77 (0xd52d bytes) 2024-09-13T07:03:33.850 INFO:teuthology.orchestra.run.smithi118.stdout:5860 trunc from 0xcaa14a5 to 0x8119bc5 2024-09-13T07:03:33.850 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:33.845+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:33.851 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:33.883 INFO:teuthology.orchestra.run.smithi118.stdout:5861 write 0xb85ec99 thru 0xb860797 (0x1aff bytes) 2024-09-13T07:03:33.890 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.890 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.890 INFO:teuthology.orchestra.run.smithi118.stdout:5865 read 0xb6df932 thru 0xb6e660e (0x6cdd bytes) 2024-09-13T07:03:33.909 INFO:teuthology.orchestra.run.smithi118.stdout:5866 writesame 0xec9bdf3 thru 0xeca8ddb data_size 0x5(0xcfe9 bytes) 2024-09-13T07:03:33.909 INFO:teuthology.orchestra.run.smithi118.stdout:5868 punch from 0x410bba2 to 0x4119a25, (0xde83 bytes) 2024-09-13T07:03:33.909 INFO:teuthology.orchestra.run.smithi118.stdout:5869 write 0x980751a thru 0x98106a5 (0x918c bytes) 2024-09-13T07:03:33.933 INFO:teuthology.orchestra.run.smithi118.stdout:5870 trunc from 0xeca8ddc to 0x3742b1f 2024-09-13T07:03:33.950 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:33.945+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:33.958 INFO:teuthology.orchestra.run.smithi118.stdout:5872 punch from 0x2cef9eb to 0x2cfcc2f, (0xd244 bytes) 2024-09-13T07:03:33.958 INFO:teuthology.orchestra.run.smithi118.stdout:5875 read 0x27a8c5c thru 0x27b8961 (0xfd06 bytes) 2024-09-13T07:03:33.959 INFO:teuthology.orchestra.run.smithi118.stdout:5876 read 0x30615ac thru 0x3064c35 (0x368a bytes) 2024-09-13T07:03:33.964 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:33.964 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:33.964 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:33.964 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:33.972 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:03:33.979 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:33.979 INFO:teuthology.orchestra.run.smithi118.stdout:5878 write 0x7faa7bd thru 0x7fb4b68 (0xa3ac bytes) 2024-09-13T07:03:33.986 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.986 INFO:teuthology.orchestra.run.smithi118.stdout:5880 read 0x333236d thru 0x3335639 (0x32cd bytes) 2024-09-13T07:03:33.986 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:33.987 INFO:teuthology.orchestra.run.smithi118.stdout:5882 read 0x1cd1603 thru 0x1cd9758 (0x8156 bytes) 2024-09-13T07:03:33.987 INFO:teuthology.orchestra.run.smithi118.stdout:5883 write 0x782014 thru 0x78a13a (0x8127 bytes) 2024-09-13T07:03:33.991 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:34.000 INFO:teuthology.orchestra.run.smithi118.stdout:5886 trunc from 0x7fb4b69 to 0x2c5fc11 2024-09-13T07:03:34.025 INFO:teuthology.orchestra.run.smithi118.stdout:5888 read 0x19e4342 thru 0x19ed208 (0x8ec7 bytes) 2024-09-13T07:03:34.049 INFO:teuthology.orchestra.run.smithi118.stdout:5889 write 0x84deb7a thru 0x84e25ef (0x3a76 bytes) 2024-09-13T07:03:34.051 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:34.049+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:34.058 INFO:teuthology.orchestra.run.smithi118.stdout:5890 trunc from 0x84e25f0 to 0x9ba4aa6 2024-09-13T07:03:34.069 INFO:teuthology.orchestra.run.smithi118.stdout:5891 punch from 0x72a931e to 0x72b7545, (0xe227 bytes) 2024-09-13T07:03:34.084 INFO:teuthology.orchestra.run.smithi118.stdout:5892 write 0xc297212 thru 0xc29fe92 (0x8c81 bytes) 2024-09-13T07:03:34.090 INFO:teuthology.orchestra.run.smithi118.stdout:5893 write 0x113eee3 thru 0x1145649 (0x6767 bytes) 2024-09-13T07:03:34.093 INFO:teuthology.orchestra.run.smithi118.stdout:5894 write 0x1aab5ac thru 0x1ab8c69 (0xd6be bytes) 2024-09-13T07:03:34.096 INFO:teuthology.orchestra.run.smithi118.stdout:5895 read 0x614ac1f thru 0x6159ea2 (0xf284 bytes) 2024-09-13T07:03:34.096 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:34.096 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:34.096 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:34.097 INFO:teuthology.orchestra.run.smithi118.stdout:5898 punch from 0xa504d99 to 0xa50b9cb, (0x6c32 bytes) 2024-09-13T07:03:34.097 INFO:teuthology.orchestra.run.smithi118.stdout:5900 write 0x521ebe7 thru 0x521f786 (0xba0 bytes) 2024-09-13T07:03:34.102 INFO:teuthology.orchestra.run.smithi118.stdout:5902 read 0x4c1a1d9 thru 0x4c29f29 (0xfd51 bytes) 2024-09-13T07:03:34.103 INFO:teuthology.orchestra.run.smithi118.stdout:5903 read 0x51fe204 thru 0x52006d4 (0x24d1 bytes) 2024-09-13T07:03:34.104 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:34.104 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:34.104 INFO:teuthology.orchestra.run.smithi118.stdout:5906 write 0x497b89f thru 0x49866fc (0xae5e bytes) 2024-09-13T07:03:34.104 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:34.106 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:34.110 INFO:teuthology.orchestra.run.smithi118.stdout:5907 read 0xb3d2072 thru 0xb3d8982 (0x6911 bytes) 2024-09-13T07:03:34.123 INFO:teuthology.orchestra.run.smithi118.stdout:5908 writesame 0xce42165 thru 0xce5186a data_size 0x7(0xf706 bytes) 2024-09-13T07:03:34.126 INFO:teuthology.orchestra.run.smithi118.stdout:5909 read 0xbc44ebb thru 0xbc4f3a1 (0xa4e7 bytes) 2024-09-13T07:03:34.126 INFO:teuthology.orchestra.run.smithi118.stdout:5910 read 0x8bcb4f1 thru 0x8bd8b33 (0xd643 bytes) 2024-09-13T07:03:34.126 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:34.126 INFO:teuthology.orchestra.run.smithi118.stdout:5911 read 0x8cfef8a thru 0x8d09799 (0xa810 bytes) 2024-09-13T07:03:34.126 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:34.126 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:34.127 INFO:teuthology.orchestra.run.smithi118.stdout:5918 clone 50 order 20 su 65536 sc 7 2024-09-13T07:03:34.146 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:34.152 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:34.149+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:34.164 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:34.185 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:34.208 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:34.214 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:34.214 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:34.224 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:34.231 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:34.255 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:34.255 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:34.251+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:34.257 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:34.257 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:34.257 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:34.257 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:34.257 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:34.257 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:34.258 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:34.258 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:34.335 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:34.335 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:34.345 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:34.353 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:34.351+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:34.453 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:34.453 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:34.455 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:34.453+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:34.462 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:34.556 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:34.551+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:34.572 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:34.572 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:34.582 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:34.657 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:34.655+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:34.735 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:34.735 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:34.745 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:34.757 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:34.753+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:34.854 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:34.854 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:34.859 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:34.857+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:34.864 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:34.960 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:34.957+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:34.973 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:34.973 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:34.984 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:34.990 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:03:35.060 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:35.055+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:35.094 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:35.094 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:35.104 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:35.161 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:35.157+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:35.205 INFO:teuthology.orchestra.run.smithi118.stdout:truncating image image_client.0-clone49 from 0xce5186b (overlap 0x30b222) to 0x1d37ae 2024-09-13T07:03:35.207 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:35.207 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:35.218 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:35.262 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:35.259+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:35.319 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:35.319 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:35.329 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:35.377 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:35.375+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:35.447 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:35.447 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:35.458 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:35.464 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:35.461+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:35.548 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #48, image image_client.0-clone48 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent49 2024-09-13T07:03:35.564 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:35.564 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:35.564 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:35.559+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:35.572 INFO:teuthology.orchestra.run.smithi118.stdout:5919 write 0xe1ed6d thru 0xe217e3 (0x2a77 bytes) 2024-09-13T07:03:35.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:35.593 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:35.593 INFO:teuthology.orchestra.run.smithi118.stdout:5921 writesame 0x7d6052d thru 0x7d6c17e data_size 0xf(0xbc52 bytes) 2024-09-13T07:03:35.602 INFO:teuthology.orchestra.run.smithi118.stdout:5922 write 0x69bf85b thru 0x69cd591 (0xdd37 bytes) 2024-09-13T07:03:35.613 INFO:teuthology.orchestra.run.smithi118.stdout:5924 punch from 0x1395a9 to 0x146b37, (0xd58e bytes) 2024-09-13T07:03:35.613 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:35.614 INFO:teuthology.orchestra.run.smithi118.stdout:5927 read 0x6ff5e8e thru 0x7002bd7 (0xcd4a bytes) 2024-09-13T07:03:35.614 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:35.614 INFO:teuthology.orchestra.run.smithi118.stdout:5930 read 0x3c81b37 thru 0x3c88b75 (0x703f bytes) 2024-09-13T07:03:35.614 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:35.615 INFO:teuthology.orchestra.run.smithi118.stdout:5934 punch from 0x69474a9 to 0x694db5e, (0x66b5 bytes) 2024-09-13T07:03:35.627 INFO:teuthology.orchestra.run.smithi118.stdout:5935 write 0xce46a25 thru 0xce54b0d (0xe0e9 bytes) 2024-09-13T07:03:35.635 INFO:teuthology.orchestra.run.smithi118.stdout:5936 read 0x5ac0331 thru 0x5ac4703 (0x43d3 bytes) 2024-09-13T07:03:35.636 INFO:teuthology.orchestra.run.smithi118.stdout:5938 write 0x1839802 thru 0x184031b (0x6b1a bytes) 2024-09-13T07:03:35.655 INFO:teuthology.orchestra.run.smithi118.stdout:5939 trunc from 0xce54b0e to 0xa838326 2024-09-13T07:03:35.665 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:35.663+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:35.685 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:35.685 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:35.685 INFO:teuthology.orchestra.run.smithi118.stdout:5942 writesame 0x698ca1b thru 0x698d9a6 data_size 0xa(0xf8c bytes) 2024-09-13T07:03:35.693 INFO:teuthology.orchestra.run.smithi118.stdout:5944 punch from 0x3966cb5 to 0x396c5b8, (0x5903 bytes) 2024-09-13T07:03:35.693 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:35.693 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:35.696 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:03:35.709 INFO:teuthology.orchestra.run.smithi118.stdout:5948 writesame 0xbe1e336 thru 0xbe29ead data_size 0x1c(0xbb78 bytes) 2024-09-13T07:03:35.722 INFO:teuthology.orchestra.run.smithi118.stdout:5949 trunc from 0xbe29eae to 0x651933c 2024-09-13T07:03:35.760 INFO:teuthology.orchestra.run.smithi118.stdout:5951 punch from 0xa0c90e to 0xa12ad8, (0x61ca bytes) 2024-09-13T07:03:35.765 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:35.761+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:35.773 INFO:teuthology.orchestra.run.smithi118.stdout:5952 write 0x7087c80 thru 0x7089769 (0x1aea bytes) 2024-09-13T07:03:35.778 INFO:teuthology.orchestra.run.smithi118.stdout:5953 read 0x285d246 thru 0x286d1b7 (0xff72 bytes) 2024-09-13T07:03:35.780 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:35.780 INFO:teuthology.orchestra.run.smithi118.stdout:5955 read 0x3a076a6 thru 0x3a09719 (0x2074 bytes) 2024-09-13T07:03:35.781 INFO:teuthology.orchestra.run.smithi118.stdout:5956 write 0x1ba3ad7 thru 0x1bb21e9 (0xe713 bytes) 2024-09-13T07:03:35.792 INFO:teuthology.orchestra.run.smithi118.stdout:5960 punch from 0x4fc97a8 to 0x4fd7bb7, (0xe40f bytes) 2024-09-13T07:03:35.792 INFO:teuthology.orchestra.run.smithi118.stdout:5961 punch from 0x2ff5097 to 0x2ffe71c, (0x9685 bytes) 2024-09-13T07:03:35.802 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:35.802 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:35.811 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:03:35.812 INFO:teuthology.orchestra.run.smithi118.stdout:5962 write 0x9bd1d07 thru 0x9bde5a8 (0xc8a2 bytes) 2024-09-13T07:03:35.818 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:35.818 INFO:teuthology.orchestra.run.smithi118.stdout:5965 read 0x542254 thru 0x54cf50 (0xacfd bytes) 2024-09-13T07:03:35.819 INFO:teuthology.orchestra.run.smithi118.stdout:5966 punch from 0x1a15ef8 to 0x1a1a7cc, (0x48d4 bytes) 2024-09-13T07:03:35.819 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:35.819 INFO:teuthology.orchestra.run.smithi118.stdout:5969 write 0x9402454 thru 0x940d895 (0xb442 bytes) 2024-09-13T07:03:35.825 INFO:teuthology.orchestra.run.smithi118.stdout:5970 write 0x4f36114 thru 0x4f39fc2 (0x3eaf bytes) 2024-09-13T07:03:35.834 INFO:teuthology.orchestra.run.smithi118.stdout:5973 read 0xf918a4 thru 0xf942af (0x2a0c bytes) 2024-09-13T07:03:35.840 INFO:teuthology.orchestra.run.smithi118.stdout:5975 trunc from 0x9bde5a9 to 0x7fba944 2024-09-13T07:03:35.859 INFO:teuthology.orchestra.run.smithi118.stdout:5976 read 0x4825452 thru 0x4833b11 (0xe6c0 bytes) 2024-09-13T07:03:35.867 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:35.865+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:35.874 INFO:teuthology.orchestra.run.smithi118.stdout:5978 writesame 0x8422c9d thru 0x842414d data_size 0x1(0x14b1 bytes) 2024-09-13T07:03:35.888 INFO:teuthology.orchestra.run.smithi118.stdout:5979 trunc from 0x842414e to 0xdf654c7 2024-09-13T07:03:35.902 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:35.902 INFO:teuthology.orchestra.run.smithi118.stdout:5981 punch from 0xaaddd15 to 0xaae10f5, (0x33e0 bytes) 2024-09-13T07:03:35.902 INFO:teuthology.orchestra.run.smithi118.stdout:5982 read 0x538cc88 thru 0x5391a09 (0x4d82 bytes) 2024-09-13T07:03:35.904 INFO:teuthology.orchestra.run.smithi118.stdout:5984 read 0xadccb99 thru 0xadd5ff1 (0x9459 bytes) 2024-09-13T07:03:35.904 INFO:teuthology.orchestra.run.smithi118.stdout:5985 punch from 0xadaeab3 to 0xadb95a0, (0xaaed bytes) 2024-09-13T07:03:35.904 INFO:teuthology.orchestra.run.smithi118.stdout:5986 read 0xcf4dfcd thru 0xcf5db4f (0xfb83 bytes) 2024-09-13T07:03:35.904 INFO:teuthology.orchestra.run.smithi118.stdout:5987 read 0x9f98e13 thru 0x9f9f873 (0x6a61 bytes) 2024-09-13T07:03:35.904 INFO:teuthology.orchestra.run.smithi118.stdout:5989 punch from 0x209e35b to 0x20a7fec, (0x9c91 bytes) 2024-09-13T07:03:35.904 INFO:teuthology.orchestra.run.smithi118.stdout:5991 read 0x2045df9 thru 0x204d011 (0x7219 bytes) 2024-09-13T07:03:35.905 INFO:teuthology.orchestra.run.smithi118.stdout:5992 punch from 0xd3e24d3 to 0xd3f2015, (0xfb42 bytes) 2024-09-13T07:03:35.905 INFO:teuthology.orchestra.run.smithi118.stdout:5993 write 0x2ba7de thru 0x2c64eb (0xbd0e bytes) 2024-09-13T07:03:35.919 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:35.919 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:35.922 INFO:teuthology.orchestra.run.smithi118.stdout:5994 writesame 0x89a07a7 thru 0x89ab9cd data_size 0x1(0xb227 bytes) 2024-09-13T07:03:35.929 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:03:35.968 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:35.963+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:35.984 INFO:teuthology.orchestra.run.smithi118.stdout:5996 trunc from 0xdf654c7 to 0x292e098 2024-09-13T07:03:35.991 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:03:36.022 INFO:teuthology.orchestra.run.smithi118.stdout:compare and write disabled 2024-09-13T07:03:36.022 INFO:teuthology.orchestra.run.smithi118.stdout:6000 read 0x61cf8a thru 0x61efcc (0x2043 bytes) 2024-09-13T07:03:36.039 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:36.039 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:36.049 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:36.068 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:36.063+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:36.152 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:36.152 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:36.161 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:36.168 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:36.165+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:36.182 INFO:teuthology.orchestra.run.smithi118.stdout:checking clone #49, image image_client.0-clone49 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent50 2024-09-13T07:03:36.267 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:36.267 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:36.269 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:03:36.269 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:36.267+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:36.277 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:36.370 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:36.367+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:36.421 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:36.435 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:36.435 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:36.441 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:36.446 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:36.464 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:36.470 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:36.467+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:36.493 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:36.513 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:36.532 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:36.555 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:36.566 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:36.566 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:36.571 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:36.569+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:36.573 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:36.578 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:36.580 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:36.580 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:36.580 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:36.580 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:36.580 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:36.580 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:36.580 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:36.580 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:36.672 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:36.669+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:36.718 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:36.718 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:36.728 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:36.773 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:36.769+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:36.838 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:36.838 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:36.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:36.839 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:36.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.839 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:03:36.839 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:36.839 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 130.782189192, 2024-09-13T07:03:36.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:03:36.839 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.839 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:36.839 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:36.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:36.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:03:36.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:36.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:03:36.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:03:36.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:03:36.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:03:36.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:03:36.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:03:36.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:03:36.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:03:36.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:36.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:03:36.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:03:36.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:36.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:03:36.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:03:36.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:03:36.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.624307958000003, 2024-09-13T07:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:03:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:36.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:03:36.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:36.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:03:36.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:36.851 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.598538246999993, 2024-09-13T07:03:36.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:03:36.851 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.851 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:36.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.851 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:03:36.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.851 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:36.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:36.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:03:36.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:03:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:03:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:03:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:03:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:03:36.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:03:36.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:03:36.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:03:36.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:36.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:03:36.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:03:36.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:36.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:03:36.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:03:36.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:36.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:03:36.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:03:36.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:36.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:03:36.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:03:36.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:36.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:03:36.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:03:36.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:03:36.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:03:36.857 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.857 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.857 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.857 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:36.857 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:36.857 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.577387080999998, 2024-09-13T07:03:36.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:03:36.857 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.857 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:36.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:03:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:03:36.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:36.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:03:36.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:03:36.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:03:36.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:03:36.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:03:36.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:03:36.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:03:36.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:03:36.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:36.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:03:36.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:03:36.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:36.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:03:36.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:03:36.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:36.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:03:36.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:03:36.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:36.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:03:36.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:03:36.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:36.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:03:36.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:03:36.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:03:36.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:03:36.863 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.864 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.864 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.864 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:03:36.864 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:36.864 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.460342318000002, 2024-09-13T07:03:36.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:03:36.864 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.864 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:36.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.865 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:03:36.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.865 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:36.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:36.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:36.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:36.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:03:36.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:03:36.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:03:36.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:03:36.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:03:36.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:03:36.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:03:36.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:03:36.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:36.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:03:36.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:36.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:03:36.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:03:36.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:36.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:03:36.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:03:36.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:03:36.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:03:36.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.873 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.873 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:36.873 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:36.874 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.175345073000003, 2024-09-13T07:03:36.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:03:36.874 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.874 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:36.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:03:36.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.874 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:36.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:36.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:03:36.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:36.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:03:36.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:03:36.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:03:36.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:03:36.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:03:36.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:03:36.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:03:36.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:03:36.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:36.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:03:36.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:03:36.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:03:36.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:03:36.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.880 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.880 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:36.880 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:36.880 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.923635934999993, 2024-09-13T07:03:36.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:03:36.880 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.880 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:36.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.881 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.881 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.881 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:03:36.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.881 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:36.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:36.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:03:36.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:36.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:03:36.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:03:36.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:03:36.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:03:36.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:03:36.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:03:36.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:03:36.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:03:36.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:36.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:03:36.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:03:36.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:03:36.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:03:36.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.887 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.887 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:36.887 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:36.887 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.923617901, 2024-09-13T07:03:36.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:03:36.888 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.888 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:36.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.889 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:03:36.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.889 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:03:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:36.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:03:36.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:03:36.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:03:36.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:03:36.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:03:36.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:03:36.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.893 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.897 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:36.897 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:36.898 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.923601821000005, 2024-09-13T07:03:36.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:03:36.898 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.898 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:36.898 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.899 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:03:36.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.900 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:36.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:36.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:03:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:03:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:03:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:03:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:03:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:03:36.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:36.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:03:36.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:03:36.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:03:36.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:03:36.904 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.904 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.904 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.904 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:36.905 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:36.905 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.923586145000002, 2024-09-13T07:03:36.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:03:36.905 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:03:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:03:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:03:36.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:03:36.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:03:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:03:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:03:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:03:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:03:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:03:36.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:03:36.910 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.910 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.910 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.911 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:36.911 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:36.911 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.923563223000002, 2024-09-13T07:03:36.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:03:36.911 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.911 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:36.911 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.912 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:03:36.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.913 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:36.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:36.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:36.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:36.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:03:36.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:03:36.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:03:36.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:03:36.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:03:36.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:03:36.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:03:36.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:03:36.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:03:36.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:03:36.917 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.917 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.917 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.917 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:36.917 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:36.918 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.923547126000003, 2024-09-13T07:03:36.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:03:36.918 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.918 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:36.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.918 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:03:36.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.919 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:36.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:36.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:03:36.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:36.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:03:36.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:03:36.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:03:36.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:03:36.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:03:36.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:03:36.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:03:36.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:03:36.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:03:36.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:03:36.924 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.924 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.924 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.925 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107067 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:36.925 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:36.925 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.922987109000005, 2024-09-13T07:03:36.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04441055, 2024-09-13T07:03:36.925 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.925 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:36.925 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.925 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.925 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.925 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107067 2024-09-13T07:03:36.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.926 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:36.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:36.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912832+0000", 2024-09-13T07:03:36.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9994574 2024-09-13T07:03:36.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913379+0000", 2024-09-13T07:03:36.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00054719999999999997 2024-09-13T07:03:36.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913386+0000", 2024-09-13T07:03:36.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4549999999999998e-06 2024-09-13T07:03:36.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913393+0000", 2024-09-13T07:03:36.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6259999999999997e-06 2024-09-13T07:03:36.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957776+0000", 2024-09-13T07:03:36.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044383143999999999 2024-09-13T07:03:36.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957785+0000", 2024-09-13T07:03:36.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9220000000000003e-06 2024-09-13T07:03:36.929 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.929 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.929 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107068 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.922962678000005, 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044410502999999997, 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107068 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:36.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:36.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:36.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:36.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913401+0000", 2024-09-13T07:03:36.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:03:36.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913407+0000", 2024-09-13T07:03:36.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7520000000000002e-06 2024-09-13T07:03:36.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913412+0000", 2024-09-13T07:03:36.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8550000000000001e-06 2024-09-13T07:03:36.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957802+0000", 2024-09-13T07:03:36.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044389606999999998 2024-09-13T07:03:36.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957809+0000", 2024-09-13T07:03:36.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9160000000000001e-06 2024-09-13T07:03:36.935 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.922945514000006, 2024-09-13T07:03:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:03:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:03:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:36.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:36.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:03:36.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:03:36.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:03:36.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:03:36.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:03:36.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:03:36.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:03:36.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:03:36.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:03:36.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:03:36.941 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.941 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.941 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.941 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:36.941 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:36.941 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.922902738000005, 2024-09-13T07:03:36.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:03:36.941 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.941 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:36.941 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.942 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.942 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.942 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:03:36.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.942 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:36.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:36.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:03:36.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:36.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:03:36.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:03:36.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:03:36.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:03:36.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:03:36.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:03:36.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:03:36.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:03:36.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:03:36.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:03:36.945 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.945 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.945 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.946 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:36.946 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:36.946 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.765884968000002, 2024-09-13T07:03:36.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:03:36.946 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.946 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:36.946 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.946 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.946 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.946 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:03:36.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.947 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:36.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:36.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:03:36.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:36.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:03:36.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:03:36.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:03:36.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:03:36.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:03:36.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:03:36.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:03:36.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:03:36.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:03:36.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:03:36.950 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.950 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.950 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.950 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.913508038, 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:36.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:36.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:03:36.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:36.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:03:36.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:03:36.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:03:36.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:03:36.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:03:36.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:03:36.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:03:36.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:03:36.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.955 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.955 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:164179 3.14 e135/67 3:2fc485bd:::rbd_data.11942e4f5fc7.0000000000000002:head v 135'401, mlcod=132'398)", 2024-09-13T07:03:36.956 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:36.956 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8905675840000002, 2024-09-13T07:03:36.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063492174999999998, 2024-09-13T07:03:36.956 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.956 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:36.956 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:36.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:36.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945792+0000", 2024-09-13T07:03:36.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:36.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954811+0000", 2024-09-13T07:03:36.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090187940000000001 2024-09-13T07:03:36.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954840+0000", 2024-09-13T07:03:36.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8659000000000001e-05 2024-09-13T07:03:36.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954852+0000", 2024-09-13T07:03:36.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1966e-05 2024-09-13T07:03:36.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954938+0000", 2024-09-13T07:03:36.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6539000000000006e-05 2024-09-13T07:03:36.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:36.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954975+0000", 2024-09-13T07:03:36.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6303999999999997e-05 2024-09-13T07:03:36.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:36.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009206+0000", 2024-09-13T07:03:36.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.054231044999999999 2024-09-13T07:03:36.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009286+0000", 2024-09-13T07:03:36.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700999999999999e-05 2024-09-13T07:03:36.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.961 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.961 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:165953 3.d 3:b3a20b45:::rbd_data.11942e4f5fc7.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8796117~39260] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:36.961 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:36.962 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.6141200260000002, 2024-09-13T07:03:36.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.095290105, 2024-09-13T07:03:36.962 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:36.962 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:36.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:36.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:36.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:36.962 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 165953 2024-09-13T07:03:36.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.962 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:36.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:36.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:36.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:36.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:36.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:36.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:36.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222240+0000", 2024-09-13T07:03:36.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:36.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:36.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258929+0000", 2024-09-13T07:03:36.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.036688401000000002 2024-09-13T07:03:36.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:36.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258971+0000", 2024-09-13T07:03:36.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2212000000000002e-05 2024-09-13T07:03:36.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:36.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258989+0000", 2024-09-13T07:03:36.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7674000000000001e-05 2024-09-13T07:03:36.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:36.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259072+0000", 2024-09-13T07:03:36.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3656e-05 2024-09-13T07:03:36.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:36.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259297+0000", 2024-09-13T07:03:36.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022450700000000001 2024-09-13T07:03:36.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:36.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.260463+0000", 2024-09-13T07:03:36.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011665340000000001 2024-09-13T07:03:36.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.316214+0000", 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.055750268999999998 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317335+0000", 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011211719999999999 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317451+0000", 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011616 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:36.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:36.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:36.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317532+0000", 2024-09-13T07:03:36.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0860999999999997e-05 2024-09-13T07:03:36.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.969 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.970 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:36.970 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:36.970 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:36.971 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:36.972 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:36.967+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:37.072 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:37.069+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:37.081 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:37.081 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:37.081 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:37.081 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:37.091 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:37.177 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:37.175+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:37.193 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:37.193 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:37.193 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:37.193 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:37.193 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:37.203 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:37.278 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:37.275+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:37.316 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:37.316 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:37.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:37.316 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:37.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.316 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137474 3.6 3:65c31370:::rbd_data.1194af11889b.000000000000013b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 186185~11715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-13T07:03:37.316 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:37.316 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.572473049000003, 2024-09-13T07:03:37.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0054357140000001, 2024-09-13T07:03:37.317 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.317 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:37.317 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.317 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.317 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.317 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137474 2024-09-13T07:03:37.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.317 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:37.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:37.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743934+0000", 2024-09-13T07:03:37.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:37.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743949+0000", 2024-09-13T07:03:37.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5705999999999998e-05 2024-09-13T07:03:37.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743960+0000", 2024-09-13T07:03:37.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0084e-05 2024-09-13T07:03:37.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743971+0000", 2024-09-13T07:03:37.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0942e-05 2024-09-13T07:03:37.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744045+0000", 2024-09-13T07:03:37.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4626e-05 2024-09-13T07:03:37.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:37.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744062+0000", 2024-09-13T07:03:37.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7339000000000001e-05 2024-09-13T07:03:37.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746366+0000", 2024-09-13T07:03:37.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.002303457 2024-09-13T07:03:37.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746520+0000", 2024-09-13T07:03:37.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015385900000000001 2024-09-13T07:03:37.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746734+0000", 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000214253 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749128+0000", 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002394129 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749267+0000", 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013878299999999999 2024-09-13T07:03:37.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:37.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749328+0000", 2024-09-13T07:03:37.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0918000000000002e-05 2024-09-13T07:03:37.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749371+0000", 2024-09-13T07:03:37.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3188000000000001e-05 2024-09-13T07:03:37.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.323 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.324 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163305 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2387968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.324 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:03:37.324 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6382545680000007, 2024-09-13T07:03:37.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044745869000000001, 2024-09-13T07:03:37.324 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.324 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.324 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163305 2024-09-13T07:03:37.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.325 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:03:37.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:03:37.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678153+0000", 2024-09-13T07:03:37.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678156+0000", 2024-09-13T07:03:37.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6350000000000002e-06 2024-09-13T07:03:37.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678161+0000", 2024-09-13T07:03:37.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678166+0000", 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.498e-06 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722751+0000", 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044585134999999998 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722836+0000", 2024-09-13T07:03:37.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4672999999999994e-05 2024-09-13T07:03:37.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722900+0000", 2024-09-13T07:03:37.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4042999999999997e-05 2024-09-13T07:03:37.328 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.328 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.328 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.328 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163307 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.328 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:03:37.329 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6382341040000004, 2024-09-13T07:03:37.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045060944999999998, 2024-09-13T07:03:37.329 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.329 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.329 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.329 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.329 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.329 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163307 2024-09-13T07:03:37.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.329 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:03:37.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:03:37.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:03:37.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678177+0000", 2024-09-13T07:03:37.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:03:37.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678183+0000", 2024-09-13T07:03:37.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0599999999999996e-06 2024-09-13T07:03:37.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678188+0000", 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722941+0000", 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044752844999999999 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723073+0000", 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013128599999999999 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723235+0000", 2024-09-13T07:03:37.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000162671 2024-09-13T07:03:37.333 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.333 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.333 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.333 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163310 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 544768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.333 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:03:37.333 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6382151520000008, 2024-09-13T07:03:37.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045290534, 2024-09-13T07:03:37.333 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.334 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.334 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.334 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.334 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.334 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163310 2024-09-13T07:03:37.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.334 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:03:37.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:03:37.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:03:37.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678195+0000", 2024-09-13T07:03:37.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.785e-06 2024-09-13T07:03:37.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678201+0000", 2024-09-13T07:03:37.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:03:37.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678205+0000", 2024-09-13T07:03:37.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6999999999999999e-06 2024-09-13T07:03:37.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723299+0000", 2024-09-13T07:03:37.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045093964 2024-09-13T07:03:37.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723384+0000", 2024-09-13T07:03:37.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4562999999999997e-05 2024-09-13T07:03:37.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723484+0000", 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100045 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163313 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1093632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6381991669999998, 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045562975999999998, 2024-09-13T07:03:37.338 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.339 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.339 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.339 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.339 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.339 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163313 2024-09-13T07:03:37.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.339 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:03:37.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:03:37.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:03:37.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678212+0000", 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8650000000000001e-06 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678217+0000", 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2660000000000001e-06 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678221+0000", 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-06 2024-09-13T07:03:37.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723552+0000", 2024-09-13T07:03:37.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045330489000000002 2024-09-13T07:03:37.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723660+0000", 2024-09-13T07:03:37.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010770800000000001 2024-09-13T07:03:37.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723772+0000", 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112608 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163317 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1642496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6381703590000001, 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045770165000000002, 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.344 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.344 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163317 2024-09-13T07:03:37.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.344 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:03:37.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:03:37.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678237+0000", 2024-09-13T07:03:37.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678241+0000", 2024-09-13T07:03:37.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2669999999999998e-06 2024-09-13T07:03:37.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678247+0000", 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7309999999999996e-06 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678252+0000", 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723830+0000", 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045577234000000001 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723936+0000", 2024-09-13T07:03:37.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106277 2024-09-13T07:03:37.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724008+0000", 2024-09-13T07:03:37.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2422999999999994e-05 2024-09-13T07:03:37.347 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.347 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.347 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.348 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163320 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.348 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:03:37.348 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6381499349999995, 2024-09-13T07:03:37.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045968883000000002, 2024-09-13T07:03:37.348 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.348 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.348 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163320 2024-09-13T07:03:37.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.348 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:03:37.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:03:37.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678261+0000", 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8090000000000001e-06 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678267+0000", 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8860000000000001e-06 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678271+0000", 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4549999999999997e-06 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724061+0000", 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045789656999999997 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724151+0000", 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9734999999999994e-05 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724227+0000", 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6848999999999996e-05 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163324 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6378656019999998, 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045988788000000003, 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.352 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.353 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.353 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.353 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163324 2024-09-13T07:03:37.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.353 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:03:37.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:03:37.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678542+0000", 2024-09-13T07:03:37.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:37.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678547+0000", 2024-09-13T07:03:37.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6829999999999999e-06 2024-09-13T07:03:37.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678556+0000", 2024-09-13T07:03:37.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1929999999999999e-06 2024-09-13T07:03:37.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678562+0000", 2024-09-13T07:03:37.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.083e-06 2024-09-13T07:03:37.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724267+0000", 2024-09-13T07:03:37.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045705427 2024-09-13T07:03:37.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724417+0000", 2024-09-13T07:03:37.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014984699999999999 2024-09-13T07:03:37.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724532+0000", 2024-09-13T07:03:37.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011430900000000001 2024-09-13T07:03:37.357 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.357 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.357 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.357 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163327 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 733184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.357 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:03:37.357 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6378405679999997, 2024-09-13T07:03:37.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046185902000000001, 2024-09-13T07:03:37.357 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.358 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.358 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.358 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.358 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.358 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163327 2024-09-13T07:03:37.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.358 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:03:37.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:03:37.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678567+0000", 2024-09-13T07:03:37.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678571+0000", 2024-09-13T07:03:37.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3799999999999998e-06 2024-09-13T07:03:37.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678576+0000", 2024-09-13T07:03:37.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7760000000000003e-06 2024-09-13T07:03:37.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678581+0000", 2024-09-13T07:03:37.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8640000000000004e-06 2024-09-13T07:03:37.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724561+0000", 2024-09-13T07:03:37.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045979178000000002 2024-09-13T07:03:37.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724649+0000", 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8677000000000005e-05 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724754+0000", 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104527 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163331 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:03:37.362 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6378224140000004, 2024-09-13T07:03:37.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046361773000000002, 2024-09-13T07:03:37.363 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.363 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.363 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.363 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.363 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.363 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163331 2024-09-13T07:03:37.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.364 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:03:37.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:03:37.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678585+0000", 2024-09-13T07:03:37.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678588+0000", 2024-09-13T07:03:37.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0469999999999998e-06 2024-09-13T07:03:37.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678595+0000", 2024-09-13T07:03:37.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0399999999999998e-06 2024-09-13T07:03:37.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678599+0000", 2024-09-13T07:03:37.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:03:37.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724780+0000", 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046181168000000002 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724871+0000", 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0562000000000005e-05 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724948+0000", 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6985999999999998e-05 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.367 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.368 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163334 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1662976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.368 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:03:37.368 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6378052109999999, 2024-09-13T07:03:37.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046544502000000001, 2024-09-13T07:03:37.368 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.368 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.368 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.368 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.368 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.368 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163334 2024-09-13T07:03:37.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.369 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:03:37.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:03:37.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:03:37.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678605+0000", 2024-09-13T07:03:37.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5380000000000001e-06 2024-09-13T07:03:37.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678611+0000", 2024-09-13T07:03:37.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7389999999999996e-06 2024-09-13T07:03:37.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678615+0000", 2024-09-13T07:03:37.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-13T07:03:37.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724987+0000", 2024-09-13T07:03:37.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046371969999999998 2024-09-13T07:03:37.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725075+0000", 2024-09-13T07:03:37.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8091999999999996e-05 2024-09-13T07:03:37.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725148+0000", 2024-09-13T07:03:37.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2206999999999995e-05 2024-09-13T07:03:37.372 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.373 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.373 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.373 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163338 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.373 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:03:37.373 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6377884809999994, 2024-09-13T07:03:37.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046738452999999999, 2024-09-13T07:03:37.373 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.373 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.373 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.373 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.373 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163338 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678619+0000", 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678622+0000", 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.649e-06 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678628+0000", 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5029999999999997e-06 2024-09-13T07:03:37.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678632+0000", 2024-09-13T07:03:37.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4020000000000004e-06 2024-09-13T07:03:37.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725182+0000", 2024-09-13T07:03:37.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046550389999999997 2024-09-13T07:03:37.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725280+0000", 2024-09-13T07:03:37.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7260000000000001e-05 2024-09-13T07:03:37.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725358+0000", 2024-09-13T07:03:37.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8720000000000005e-05 2024-09-13T07:03:37.380 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.380 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.380 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.381 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163339 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2363392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.381 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:03:37.381 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6377729530000007, 2024-09-13T07:03:37.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046920835000000001, 2024-09-13T07:03:37.381 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.381 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.381 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.381 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.381 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.381 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163339 2024-09-13T07:03:37.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.382 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:03:37.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:03:37.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:03:37.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678637+0000", 2024-09-13T07:03:37.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2479999999999999e-06 2024-09-13T07:03:37.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678642+0000", 2024-09-13T07:03:37.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0030000000000002e-06 2024-09-13T07:03:37.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678646+0000", 2024-09-13T07:03:37.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9870000000000001e-06 2024-09-13T07:03:37.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725397+0000", 2024-09-13T07:03:37.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046750127000000002 2024-09-13T07:03:37.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725482+0000", 2024-09-13T07:03:37.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5928999999999997e-05 2024-09-13T07:03:37.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725556+0000", 2024-09-13T07:03:37.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3823000000000001e-05 2024-09-13T07:03:37.385 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.386 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.386 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.386 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163342 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.386 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:03:37.386 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6377583139999992, 2024-09-13T07:03:37.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047212991000000003, 2024-09-13T07:03:37.386 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.386 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.387 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163342 2024-09-13T07:03:37.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.387 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:03:37.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:03:37.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:03:37.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678652+0000", 2024-09-13T07:03:37.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.486e-06 2024-09-13T07:03:37.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678657+0000", 2024-09-13T07:03:37.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6909999999999996e-06 2024-09-13T07:03:37.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678660+0000", 2024-09-13T07:03:37.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8009999999999999e-06 2024-09-13T07:03:37.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725597+0000", 2024-09-13T07:03:37.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046937372999999998 2024-09-13T07:03:37.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725745+0000", 2024-09-13T07:03:37.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000148314 2024-09-13T07:03:37.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725863+0000", 2024-09-13T07:03:37.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011768 2024-09-13T07:03:37.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.391 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.391 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.392 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163345 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.392 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:03:37.392 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6368967510000001, 2024-09-13T07:03:37.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046569193000000002, 2024-09-13T07:03:37.392 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.392 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.392 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.392 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163345 2024-09-13T07:03:37.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.392 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:03:37.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:03:37.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678662+0000", 2024-09-13T07:03:37.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9991498 2024-09-13T07:03:37.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679516+0000", 2024-09-13T07:03:37.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000854789 2024-09-13T07:03:37.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679530+0000", 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3492999999999999e-05 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679537+0000", 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.323e-06 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725892+0000", 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046354244000000003 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725975+0000", 2024-09-13T07:03:37.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3356000000000006e-05 2024-09-13T07:03:37.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726081+0000", 2024-09-13T07:03:37.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105992 2024-09-13T07:03:37.396 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.396 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.396 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163349 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1298432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6368646879999993, 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046744247000000003, 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163349 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:03:37.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:03:37.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679543+0000", 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679546+0000", 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679551+0000", 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679554+0000", 2024-09-13T07:03:37.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8629999999999999e-06 2024-09-13T07:03:37.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726117+0000", 2024-09-13T07:03:37.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046563081999999999 2024-09-13T07:03:37.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726213+0000", 2024-09-13T07:03:37.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6207999999999998e-05 2024-09-13T07:03:37.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726288+0000", 2024-09-13T07:03:37.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4741999999999997e-05 2024-09-13T07:03:37.401 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.401 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.401 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.402 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163350 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1449984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.402 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:03:37.402 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6368520619999991, 2024-09-13T07:03:37.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046935635000000003, 2024-09-13T07:03:37.402 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.402 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.402 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.402 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.402 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.402 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163350 2024-09-13T07:03:37.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.402 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:03:37.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:03:37.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:03:37.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679558+0000", 2024-09-13T07:03:37.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.928e-06 2024-09-13T07:03:37.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679561+0000", 2024-09-13T07:03:37.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7659999999999999e-06 2024-09-13T07:03:37.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679564+0000", 2024-09-13T07:03:37.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4960000000000001e-06 2024-09-13T07:03:37.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726327+0000", 2024-09-13T07:03:37.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046762321000000003 2024-09-13T07:03:37.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726415+0000", 2024-09-13T07:03:37.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8487999999999997e-05 2024-09-13T07:03:37.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726492+0000", 2024-09-13T07:03:37.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6995000000000004e-05 2024-09-13T07:03:37.406 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.406 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.406 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.406 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163353 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.406 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:03:37.406 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6368419529999994, 2024-09-13T07:03:37.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047130755000000003, 2024-09-13T07:03:37.407 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.407 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.407 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.407 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.407 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.407 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163353 2024-09-13T07:03:37.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.407 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:03:37.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:03:37.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:03:37.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:37.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679568+0000", 2024-09-13T07:03:37.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5969999999999999e-06 2024-09-13T07:03:37.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679571+0000", 2024-09-13T07:03:37.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:03:37.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679573+0000", 2024-09-13T07:03:37.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:03:37.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726523+0000", 2024-09-13T07:03:37.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046949882999999998 2024-09-13T07:03:37.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726616+0000", 2024-09-13T07:03:37.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3236e-05 2024-09-13T07:03:37.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726697+0000", 2024-09-13T07:03:37.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0834000000000006e-05 2024-09-13T07:03:37.411 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.411 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.411 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.411 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163356 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2379776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.411 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:03:37.411 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.6368322709999994, 2024-09-13T07:03:37.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047304662999999997, 2024-09-13T07:03:37.412 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.412 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.412 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.412 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.412 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.412 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163356 2024-09-13T07:03:37.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.412 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:03:37.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:03:37.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:03:37.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679578+0000", 2024-09-13T07:03:37.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:03:37.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679582+0000", 2024-09-13T07:03:37.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:03:37.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679589+0000", 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1149999999999996e-06 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726723+0000", 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047133768999999999 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726809+0000", 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5402000000000007e-05 2024-09-13T07:03:37.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726881+0000", 2024-09-13T07:03:37.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2152999999999998e-05 2024-09-13T07:03:37.416 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.416 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.416 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.416 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:166663 3.b 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8129196~12979] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:37.416 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 7.9184577569999997, 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.061564718999999997, 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 166663 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:03:37.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:03:37.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397947+0000", 2024-09-13T07:03:37.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999967 2024-09-13T07:03:37.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412144+0000", 2024-09-13T07:03:37.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014196972 2024-09-13T07:03:37.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412172+0000", 2024-09-13T07:03:37.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7781999999999999e-05 2024-09-13T07:03:37.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412182+0000", 2024-09-13T07:03:37.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6069999999999997e-06 2024-09-13T07:03:37.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412239+0000", 2024-09-13T07:03:37.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.732e-05 2024-09-13T07:03:37.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412365+0000", 2024-09-13T07:03:37.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125356 2024-09-13T07:03:37.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:37.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.413178+0000", 2024-09-13T07:03:37.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081355299999999995 2024-09-13T07:03:37.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:37.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.443176+0000", 2024-09-13T07:03:37.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029997903999999999 2024-09-13T07:03:37.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459446+0000", 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016269921999999999 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459484+0000", 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8488000000000001e-05 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459515+0000", 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0981e-05 2024-09-13T07:03:37.422 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.423 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.423 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.423 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.423 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.423 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:37.424 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:37.425 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:37.419+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:37.525 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:37.521+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:37.582 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:37.582 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:37.582 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:37.593 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:37.626 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:37.625+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:37.700 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:37.701 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:37.701 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:37.701 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:37.701 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:37.710 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:37.728 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:37.725+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:37.823 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:37.823 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:37.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:37.823 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:37.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.823 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:03:37.824 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:37.824 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.328778607, 2024-09-13T07:03:37.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:03:37.824 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.824 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.824 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:03:37.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.824 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:37.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:37.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:03:37.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:37.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:03:37.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:03:37.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:03:37.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:03:37.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:03:37.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:03:37.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:03:37.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:03:37.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:37.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:03:37.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:03:37.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:03:37.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:03:37.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:03:37.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:03:37.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:03:37.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:03:37.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.829 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.829 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.829 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:37.829 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.842295043000007, 2024-09-13T07:03:37.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:03:37.829 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.829 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:37.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:37.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:03:37.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:03:37.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:03:37.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:03:37.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:03:37.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:37.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:03:37.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:03:37.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:03:37.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:03:37.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:03:37.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:03:37.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.833 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.834 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.834 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:37.834 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.842225114000001, 2024-09-13T07:03:37.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:03:37.834 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.834 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.834 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:03:37.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:03:37.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:03:37.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:03:37.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:03:37.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:03:37.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:03:37.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:37.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:03:37.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:03:37.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:03:37.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:03:37.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:03:37.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:03:37.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.838 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.838 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.839 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:37.839 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.842209241999996, 2024-09-13T07:03:37.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:03:37.839 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.839 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.839 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.839 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.839 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.839 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:03:37.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.839 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:03:37.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:03:37.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:03:37.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:03:37.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:03:37.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:03:37.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:03:37.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:03:37.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:03:37.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:03:37.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:03:37.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:03:37.843 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.843 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.843 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.843 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.843 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:37.843 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.842138785000003, 2024-09-13T07:03:37.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:03:37.844 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.844 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:37.844 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.844 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.844 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.844 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:03:37.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.844 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:37.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:37.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:37.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:03:37.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:03:37.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:37.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:03:37.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:03:37.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:03:37.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:03:37.848 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.848 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.848 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.848 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.848 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:37.848 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838786747, 2024-09-13T07:03:37.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:03:37.848 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.848 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.848 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.848 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.849 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.849 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:03:37.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.849 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:37.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:37.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:03:37.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:03:37.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:03:37.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:03:37.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:03:37.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:03:37.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:03:37.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:03:37.852 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.852 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.852 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.852 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.852 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:37.852 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838770445999998, 2024-09-13T07:03:37.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:03:37.852 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.853 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.853 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.853 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.853 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.853 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:03:37.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.853 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:37.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:03:37.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:03:37.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838750661000006, 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:03:37.856 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.857 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.857 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.857 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.857 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.857 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:03:37.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.857 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:37.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:37.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:37.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:03:37.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:03:37.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:03:37.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:03:37.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:03:37.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:03:37.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:03:37.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:03:37.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:03:37.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:03:37.860 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.860 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.860 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838734388000006, 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:03:37.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:03:37.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:03:37.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:03:37.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:03:37.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838716563000006, 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:03:37.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:03:37.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:03:37.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:03:37.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:03:37.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:03:37.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:03:37.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:03:37.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:03:37.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:03:37.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.868 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838700470000006, 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:37.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:37.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:03:37.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:03:37.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:37.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:03:37.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:03:37.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:03:37.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:03:37.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:03:37.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:03:37.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:03:37.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:03:37.873 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.873 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.873 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.873 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.873 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:37.873 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838679283999994, 2024-09-13T07:03:37.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:03:37.873 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.873 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.873 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.874 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:03:37.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.874 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:37.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:37.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:03:37.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:03:37.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:03:37.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:03:37.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:03:37.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:03:37.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:03:37.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:03:37.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:03:37.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:03:37.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:03:37.877 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.877 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.877 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.877 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838663251, 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:37.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:03:37.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:03:37.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:03:37.881 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.881 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.881 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.881 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.881 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:37.881 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838614011999994, 2024-09-13T07:03:37.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:37.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:03:37.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:03:37.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:03:37.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:03:37.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:03:37.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:03:37.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:37.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:03:37.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:03:37.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:03:37.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:03:37.885 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.885 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.885 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.885 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838595952000006, 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:37.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:03:37.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:03:37.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:03:37.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:03:37.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:03:37.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:03:37.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:03:37.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:03:37.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:03:37.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:03:37.889 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.889 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.889 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838579437000007, 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:37.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:37.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:03:37.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:03:37.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:03:37.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:03:37.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:03:37.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:03:37.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:03:37.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:03:37.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:03:37.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.893 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.893 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:03:37.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:03:37.893 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.893 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838561072000005, 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:03:37.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.895 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:37.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:37.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:37.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:03:37.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:03:37.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:03:37.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:03:37.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:03:37.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:03:37.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:03:37.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:03:37.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.897 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:03:37.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:03:37.898 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.898 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.898 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.898 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.898 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:37.898 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838544974000001, 2024-09-13T07:03:37.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:37.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:37.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:03:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:03:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:03:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:03:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:03:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:03:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.902 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838529038999994, 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.903 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.906 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 99.838512542000004, 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:37.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:37.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:03:37.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:03:37.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:37.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:03:37.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:03:37.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.909 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:37.909 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:03:37.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:03:37.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:37.910 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:37.912 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:37.912 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:37.909+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:37.991 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 7, 2, 0, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:03:37.992 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:03:37.992 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-13T07:03:37.992 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T07:03:37.992 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T07:03:37.992 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-13T07:03:38.004 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:03:38.012 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:38.009+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:38.029 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:38.030 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:38.030 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:38.030 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:38.041 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:38.113 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:38.111+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:38.119 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:38.120 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T07:03:38.120 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:38.159 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:38.159 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:38.159 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:38.159 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:38.159 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:38.170 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:38.214 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:38.209+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:38.277 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:38.277 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:38.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:38.277 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:38.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.278 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:03:38.278 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:38.278 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.22017499, 2024-09-13T07:03:38.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:03:38.278 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.278 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:38.278 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:38.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:38.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:03:38.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:38.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:03:38.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:03:38.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:03:38.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:03:38.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:03:38.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:03:38.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:03:38.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.401958633, 2024-09-13T07:03:38.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:03:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:03:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:38.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:03:38.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:38.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:03:38.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:03:38.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:03:38.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:03:38.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:03:38.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:03:38.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:03:38.286 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.287 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.287 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.287 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:38.287 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:38.287 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.401909244999999, 2024-09-13T07:03:38.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:03:38.287 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.287 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.287 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.287 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.287 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:03:38.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:03:38.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.401886537999999, 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:03:38.291 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.292 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.292 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:03:38.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.292 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:38.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:38.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:03:38.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:03:38.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:03:38.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:03:38.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:03:38.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:03:38.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:03:38.296 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.296 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.296 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.296 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:38.296 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:38.296 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.401842344999999, 2024-09-13T07:03:38.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:03:38.296 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.296 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:03:38.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:03:38.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:03:38.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:03:38.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 48.370863256, 2024-09-13T07:03:38.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:03:38.301 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.301 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:38.301 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:38.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:38.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:38.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:03:38.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:03:38.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:03:38.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:03:38.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:38.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:03:38.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:03:38.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:03:38.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:03:38.304 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.305 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.305 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.305 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137457 3.18 3:1a596dcc:::rbd_data.1194af11889b.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T07:03:38.305 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:38.305 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.625241445999997, 2024-09-13T07:03:38.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033734403000000003, 2024-09-13T07:03:38.305 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.305 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:38.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.306 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137457 2024-09-13T07:03:38.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.306 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:38.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:38.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651523+0000", 2024-09-13T07:03:38.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:38.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651531+0000", 2024-09-13T07:03:38.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.328e-06 2024-09-13T07:03:38.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651545+0000", 2024-09-13T07:03:38.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4270000000000001e-05 2024-09-13T07:03:38.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651555+0000", 2024-09-13T07:03:38.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5210000000000008e-06 2024-09-13T07:03:38.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651626+0000", 2024-09-13T07:03:38.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1915999999999994e-05 2024-09-13T07:03:38.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651761+0000", 2024-09-13T07:03:38.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000134838 2024-09-13T07:03:38.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:38.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.658577+0000", 2024-09-13T07:03:38.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068151820000000004 2024-09-13T07:03:38.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:38.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685181+0000", 2024-09-13T07:03:38.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026604200000000001 2024-09-13T07:03:38.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:38.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685193+0000", 2024-09-13T07:03:38.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2128e-05 2024-09-13T07:03:38.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:38.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685226+0000", 2024-09-13T07:03:38.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3504999999999999e-05 2024-09-13T07:03:38.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685259+0000", 2024-09-13T07:03:38.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2866e-05 2024-09-13T07:03:38.311 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.311 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.311 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.311 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138565 3.0 3:002d0425:::rbd_data.11949f62878b.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:38.311 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:38.311 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.611870954, 2024-09-13T07:03:38.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033394489999999999, 2024-09-13T07:03:38.312 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.312 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:38.312 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.312 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.312 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.312 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138565 2024-09-13T07:03:38.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.312 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:38.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:38.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664893+0000", 2024-09-13T07:03:38.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:38.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664906+0000", 2024-09-13T07:03:38.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2996000000000001e-05 2024-09-13T07:03:38.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664936+0000", 2024-09-13T07:03:38.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9609999999999999e-05 2024-09-13T07:03:38.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664949+0000", 2024-09-13T07:03:38.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383e-05 2024-09-13T07:03:38.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665046+0000", 2024-09-13T07:03:38.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6459000000000006e-05 2024-09-13T07:03:38.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665259+0000", 2024-09-13T07:03:38.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000213141 2024-09-13T07:03:38.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:38.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.672522+0000", 2024-09-13T07:03:38.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072632110000000003 2024-09-13T07:03:38.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:38.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.697982+0000", 2024-09-13T07:03:38.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025459699999999998 2024-09-13T07:03:38.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:38.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698018+0000", 2024-09-13T07:03:38.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6375999999999999e-05 2024-09-13T07:03:38.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:38.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698166+0000", 2024-09-13T07:03:38.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014818700000000001 2024-09-13T07:03:38.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698290+0000", 2024-09-13T07:03:38.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012333999999999999 2024-09-13T07:03:38.317 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.317 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.317 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138587 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.563405286999998, 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019224502000000001, 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138587 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.318 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:38.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:38.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713360+0000", 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713368+0000", 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713383+0000", 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5176e-05 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713390+0000", 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8669999999999996e-06 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713449+0000", 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8938999999999999e-05 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713549+0000", 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010009 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:38.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.716087+0000", 2024-09-13T07:03:38.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025383820000000001 2024-09-13T07:03:38.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:38.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732351+0000", 2024-09-13T07:03:38.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016263564000000001 2024-09-13T07:03:38.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:38.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732450+0000", 2024-09-13T07:03:38.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9529000000000002e-05 2024-09-13T07:03:38.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:38.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732532+0000", 2024-09-13T07:03:38.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1526000000000005e-05 2024-09-13T07:03:38.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732585+0000", 2024-09-13T07:03:38.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3819000000000001e-05 2024-09-13T07:03:38.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.323 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138786 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 29.129190032, 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018847452000000001, 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138786 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.324 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:38.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:38.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147575+0000", 2024-09-13T07:03:38.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:38.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147583+0000", 2024-09-13T07:03:38.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:03:38.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147601+0000", 2024-09-13T07:03:38.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7961e-05 2024-09-13T07:03:38.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147610+0000", 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147661+0000", 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0447e-05 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147754+0000", 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2974999999999997e-05 2024-09-13T07:03:38.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.150580+0000", 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002826566 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166135+0000", 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015554772 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166225+0000", 2024-09-13T07:03:38.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0258000000000002e-05 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166334+0000", 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010816500000000001 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166424+0000", 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0116000000000003e-05 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:139127 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 28.502139067999998, 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019295931999999998, 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139127 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774626+0000", 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774635+0000", 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5419999999999994e-06 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774654+0000", 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8359000000000002e-05 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774663+0000", 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774718+0000", 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5315000000000001e-05 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774805+0000", 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7284e-05 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.779319+0000", 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045138310000000003 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:38.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793772+0000", 2024-09-13T07:03:38.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014452692 2024-09-13T07:03:38.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:38.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793821+0000", 2024-09-13T07:03:38.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9563e-05 2024-09-13T07:03:38.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:38.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793878+0000", 2024-09-13T07:03:38.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6953999999999997e-05 2024-09-13T07:03:38.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793923+0000", 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4781999999999999e-05 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:164691 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2181791939999993, 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015985102000000001, 2024-09-13T07:03:38.335 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.336 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.336 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.336 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.336 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.336 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 164691 2024-09-13T07:03:38.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.336 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:03:38.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:03:38.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:03:38.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058589+0000", 2024-09-13T07:03:38.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2510000000000001e-06 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058594+0000", 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9089999999999999e-06 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058598+0000", 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2069999999999997e-06 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064175+0000", 2024-09-13T07:03:38.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055769909999999999 2024-09-13T07:03:38.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064210+0000", 2024-09-13T07:03:38.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4966999999999999e-05 2024-09-13T07:03:38.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074572+0000", 2024-09-13T07:03:38.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010362219000000001 2024-09-13T07:03:38.339 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165201 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.215651265, 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013054266, 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165201 2024-09-13T07:03:38.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.341 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:03:38.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:03:38.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061117+0000", 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.12e-06 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061125+0000", 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8539999999999997e-06 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061129+0000", 2024-09-13T07:03:38.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:03:38.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074092+0000", 2024-09-13T07:03:38.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012963166999999999 2024-09-13T07:03:38.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074136+0000", 2024-09-13T07:03:38.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3578000000000002e-05 2024-09-13T07:03:38.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074169+0000", 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3581000000000003e-05 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165202 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7831552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2156335899999995, 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013116245, 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.344 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165202 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061132+0000", 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061135+0000", 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3410000000000001e-06 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061142+0000", 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7079999999999997e-06 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061147+0000", 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8930000000000004e-06 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074180+0000", 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013032346 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074217+0000", 2024-09-13T07:03:38.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:03:38.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074249+0000", 2024-09-13T07:03:38.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1711000000000002e-05 2024-09-13T07:03:38.348 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.348 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.348 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.348 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165203 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:38.348 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.215615476, 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013180211000000001, 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165203 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061150+0000", 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061153+0000", 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.108e-06 2024-09-13T07:03:38.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061158+0000", 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0649999999999998e-06 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061162+0000", 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0080000000000004e-06 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074262+0000", 2024-09-13T07:03:38.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013100557000000001 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074299+0000", 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6981999999999999e-05 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074331+0000", 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1801000000000003e-05 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.353 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165204 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8093696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:38.353 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:03:38.353 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2156007649999996, 2024-09-13T07:03:38.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0132388, 2024-09-13T07:03:38.353 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.353 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.353 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.353 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.353 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.353 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165204 2024-09-13T07:03:38.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.353 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061167+0000", 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1409999999999999e-06 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061175+0000", 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5519999999999999e-06 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061179+0000", 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.172e-06 2024-09-13T07:03:38.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074342+0000", 2024-09-13T07:03:38.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013162821999999999 2024-09-13T07:03:38.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074373+0000", 2024-09-13T07:03:38.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1103999999999999e-05 2024-09-13T07:03:38.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074404+0000", 2024-09-13T07:03:38.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1359999999999998e-05 2024-09-13T07:03:38.357 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.357 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.357 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.357 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165205 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:38.357 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:03:38.357 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2155838059999997, 2024-09-13T07:03:38.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013300635, 2024-09-13T07:03:38.357 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.357 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.357 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165205 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061184+0000", 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0779999999999998e-06 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061192+0000", 2024-09-13T07:03:38.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7209999999999994e-06 2024-09-13T07:03:38.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061196+0000", 2024-09-13T07:03:38.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4469999999999999e-06 2024-09-13T07:03:38.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074417+0000", 2024-09-13T07:03:38.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013220750999999999 2024-09-13T07:03:38.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074452+0000", 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4814999999999998e-05 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074483+0000", 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1118000000000003e-05 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.361 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165206 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:38.362 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:03:38.362 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.2155656849999996, 2024-09-13T07:03:38.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013364454, 2024-09-13T07:03:38.362 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.362 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.362 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.362 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.362 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.362 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165206 2024-09-13T07:03:38.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.362 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061200+0000", 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061203+0000", 2024-09-13T07:03:38.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.559e-06 2024-09-13T07:03:38.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061210+0000", 2024-09-13T07:03:38.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-13T07:03:38.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061214+0000", 2024-09-13T07:03:38.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0999999999999997e-06 2024-09-13T07:03:38.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074494+0000", 2024-09-13T07:03:38.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013279094 2024-09-13T07:03:38.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074530+0000", 2024-09-13T07:03:38.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6538999999999997e-05 2024-09-13T07:03:38.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074565+0000", 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4972999999999999e-05 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169377 3.0 3:07b65a69:::rbd_data.1194e181a50b.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.91989929, 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.032744151999999999, 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.366 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169377 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:03:38.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356864+0000", 2024-09-13T07:03:38.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:38.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356880+0000", 2024-09-13T07:03:38.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6022999999999999e-05 2024-09-13T07:03:38.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356916+0000", 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5921999999999999e-05 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356931+0000", 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4311000000000001e-05 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357011+0000", 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9992000000000003e-05 2024-09-13T07:03:38.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357143+0000", 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000131822 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.361578+0000", 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00443525 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389466+0000", 2024-09-13T07:03:38.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027888342999999999 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389529+0000", 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2460999999999997e-05 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389574+0000", 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5405999999999999e-05 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389611+0000", 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7122000000000001e-05 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169731 3.0 3:031f8e93:::rbd_data.1194e181a50b.0000000000000006:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.9954734920000003, 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.021389756999999999, 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.372 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169731 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281291+0000", 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281302+0000", 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0651e-05 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281324+0000", 2024-09-13T07:03:38.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1713999999999999e-05 2024-09-13T07:03:38.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281340+0000", 2024-09-13T07:03:38.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:03:38.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281450+0000", 2024-09-13T07:03:38.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110086 2024-09-13T07:03:38.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281593+0000", 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014306 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.286356+0000", 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047626320000000002 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302404+0000", 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016048605000000001 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:38.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302476+0000", 2024-09-13T07:03:38.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1483000000000007e-05 2024-09-13T07:03:38.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:38.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302578+0000", 2024-09-13T07:03:38.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101883 2024-09-13T07:03:38.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302683+0000", 2024-09-13T07:03:38.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104681 2024-09-13T07:03:38.378 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.378 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.378 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.378 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.378 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.378 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:38.379 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:38.380 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:38.377+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:38.481 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:38.479+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:38.489 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:38.489 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:38.489 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:38.489 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:38.499 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:38.582 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:38.579+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:38.592 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:03:38.607 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:38.607 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:38.607 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:38.617 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:38.682 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:38.679+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:38.731 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:38.753 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:38.770 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:38.771 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:38.772 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:38.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:38.772 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:38.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.772 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.772 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:38.772 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.651156809, 2024-09-13T07:03:38.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:03:38.772 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.772 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.773 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.773 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:03:38.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.773 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:38.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:03:38.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:03:38.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:03:38.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:03:38.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:03:38.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:03:38.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:03:38.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:03:38.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:03:38.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:03:38.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.776 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.777 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.777 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:38.777 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.651136331999993, 2024-09-13T07:03:38.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:03:38.777 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.777 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.777 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:03:38.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.778 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:38.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:38.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:03:38.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:03:38.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:03:38.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:03:38.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:03:38.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:03:38.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:03:38.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:03:38.781 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.781 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.781 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.781 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.781 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:38.781 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644439728999998, 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:38.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:38.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:38.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:03:38.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:03:38.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:03:38.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:03:38.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:03:38.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:03:38.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:03:38.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.786 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.786 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.786 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.786 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:38.786 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644427660000005, 2024-09-13T07:03:38.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:03:38.786 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.786 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.787 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.787 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:03:38.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.787 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:38.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:38.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:38.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:38.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:03:38.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:03:38.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:03:38.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:03:38.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:03:38.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:03:38.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:03:38.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:03:38.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:03:38.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:03:38.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:03:38.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:03:38.790 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.790 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.790 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644415217000002, 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:38.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:38.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:38.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644397619000003, 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.795 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.796 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.796 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:03:38.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.796 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:38.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:03:38.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:03:38.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:03:38.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:03:38.799 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.799 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.799 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.800 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.800 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:38.800 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644386916000002, 2024-09-13T07:03:38.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:03:38.800 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.800 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.800 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.800 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.800 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.800 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:03:38.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.800 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:38.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:38.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:38.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:03:38.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:03:38.804 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.804 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.804 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.804 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.804 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:38.804 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644342983000001, 2024-09-13T07:03:38.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:03:38.804 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:03:38.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:03:38.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:03:38.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:03:38.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:03:38.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644323197000006, 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:38.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:38.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:38.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:03:38.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:03:38.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:03:38.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:03:38.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:03:38.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:03:38.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:03:38.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:03:38.813 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.813 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.813 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.813 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.813 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:38.813 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644307447000003, 2024-09-13T07:03:38.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:03:38.814 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.814 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.814 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:03:38.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.814 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:38.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:38.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:38.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:38.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:03:38.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:03:38.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:03:38.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:03:38.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:03:38.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:03:38.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:03:38.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:03:38.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:03:38.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:03:38.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:03:38.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:03:38.818 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.818 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.818 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.818 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.818 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:38.818 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644296194999995, 2024-09-13T07:03:38.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:03:38.818 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:38.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:03:38.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:38.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:03:38.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:03:38.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:03:38.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:03:38.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:03:38.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:03:38.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:03:38.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:03:38.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:03:38.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:03:38.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:03:38.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:03:38.823 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.823 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.823 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.823 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.823 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:38.823 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644285596000003, 2024-09-13T07:03:38.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:03:38.824 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.824 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.824 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:03:38.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.824 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:38.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:38.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:38.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:38.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:03:38.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:03:38.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:03:38.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:03:38.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:03:38.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:03:38.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:03:38.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:03:38.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:03:38.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:03:38.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:03:38.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:03:38.828 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.828 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.828 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.828 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.828 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:38.828 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644276102999996, 2024-09-13T07:03:38.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:03:38.828 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.828 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.828 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.829 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:03:38.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.829 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:38.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:03:38.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:03:38.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:03:38.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:03:38.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:03:38.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:03:38.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:03:38.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:03:38.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:03:38.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:03:38.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.832 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.833 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.833 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:38.833 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644266559000002, 2024-09-13T07:03:38.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:03:38.833 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.833 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.833 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:03:38.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:03:38.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:03:38.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:03:38.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:03:38.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:03:38.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:03:38.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644233955999994, 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:03:38.837 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:03:38.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:03:38.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:03:38.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:03:38.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:03:38.841 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.841 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.841 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.644220933, 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:38.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:03:38.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:03:38.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:03:38.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:03:38.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:03:38.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:03:38.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.640869948000002, 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.846 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:38.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:03:38.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:03:38.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:03:38.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:03:38.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:03:38.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:03:38.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:03:38.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.850 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.851 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.851 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:38.851 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.640849670999998, 2024-09-13T07:03:38.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:03:38.851 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.851 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.851 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:03:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:03:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:03:38.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:03:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:03:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:03:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:03:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:03:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:03:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.855 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 76.64082947, 2024-09-13T07:03:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:03:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:38.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:03:38.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:03:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:03:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:166663 3.b e135/117 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head v 135'409, mlcod=132'407)", 2024-09-13T07:03:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:03:38.860 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.3582405049999995, 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045108144000000003, 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:38.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414172+0000", 2024-09-13T07:03:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427200+0000", 2024-09-13T07:03:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013028098 2024-09-13T07:03:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:38.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427229+0000", 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9178000000000001e-05 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427239+0000", 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6870000000000006e-06 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427393+0000", 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015382899999999999 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427451+0000", 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8301000000000003e-05 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459232+0000", 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031780695999999997 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459282+0000", 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0701000000000001e-05 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.864 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:38.865 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:38.865 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:38.866 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:38.867 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:38.867 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:38.867 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:38.867 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:38.868 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:38.865+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:38.880 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:38.883 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:38.883 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:38.883 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:38.883 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:38.883 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:38.883 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:38.883 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:38.883 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:38.967 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:38.963+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:38.986 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:38.986 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:38.997 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:39.021 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:03:39.068 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:39.065+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:39.120 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:39.120 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:39.130 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:39.169 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:39.167+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:39.244 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:39.244 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:39.254 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:03:39.269 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:39.267+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:39.360 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:39.361 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:39.370 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:03:39.371 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:39.367+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:39.471 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:39.469+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:39.477 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:39.477 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:39.487 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:03:39.572 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:39.569+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:39.591 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:39.591 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:39.601 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:39.672 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:39.667+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:39.752 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:39.752 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:39.761 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:39.773 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:39.769+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:39.862 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:39.862 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:39.871 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:39.915 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:39.913+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:39.975 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:39.975+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:40.034 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:40.034 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:40.044 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:40.076 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:40.071+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:40.151 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:40.151 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:40.161 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:40.177 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:40.173+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:40.268 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:40.268 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:40.278 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:40.278 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:40.273+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:40.379 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:40.375+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:40.433 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:40.433 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:40.443 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:40.480 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:40.477+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:40.562 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:40.562 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:40.572 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:40.580 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:40.575+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:40.681 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:40.677+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:40.683 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:40.683 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:40.693 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:40.782 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:40.781+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:40.801 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:40.801 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:40.811 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:40.883 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:40.879+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:40.895 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:03:40.924 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:40.925 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:40.938 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:40.985 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:40.983+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:41.037 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:03:41.058 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:41.080 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:41.084 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:41.081+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:41.099 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:41.105 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:41.105 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:41.105 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:41.115 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:41.120 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:41.143 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:41.180 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5: {} 2024-09-13T07:03:41.186 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:41.181+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:41.215 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:41.242 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:41.242 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:41.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:41.242 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.748283986, 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:41.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:41.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:03:41.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:41.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:03:41.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:03:41.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:03:41.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:03:41.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:03:41.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.26180042199999, 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.248 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.249 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.249 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:03:41.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.249 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:41.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:41.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:41.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:03:41.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:03:41.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:03:41.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:03:41.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:03:41.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:03:41.252 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.252 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.252 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.261730493, 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:41.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:41.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:03:41.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:41.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:03:41.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:03:41.257 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.257 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.257 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.257 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.257 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:41.257 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.261714621, 2024-09-13T07:03:41.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:03:41.257 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.257 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.257 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.257 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.258 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.258 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:03:41.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.258 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:41.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:41.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:03:41.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:03:41.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:03:41.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:03:41.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:03:41.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:03:41.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:03:41.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:03:41.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:03:41.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:03:41.261 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.261 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.261 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.262 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.262 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:41.262 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.261644164, 2024-09-13T07:03:41.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:03:41.262 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.262 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.262 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.262 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.262 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.262 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:41.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:03:41.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:03:41.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:03:41.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:03:41.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:03:41.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:03:41.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:03:41.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:03:41.266 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.266 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.266 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.266 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.258292126, 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:41.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:41.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:41.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.258275825, 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:03:41.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:03:41.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:03:41.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:03:41.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:03:41.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:03:41.275 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.275 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.275 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.275 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.275 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:41.275 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.25825604000001, 2024-09-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:03:41.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:03:41.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:03:41.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:03:41.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.280 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.280 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:41.280 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.25823976700001, 2024-09-13T07:03:41.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:03:41.280 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.280 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.280 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:03:41.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.281 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:41.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:41.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:03:41.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:03:41.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:03:41.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:03:41.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:03:41.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:03:41.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:03:41.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:03:41.284 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.284 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.284 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.284 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.285 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:41.285 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.25822194200001, 2024-09-13T07:03:41.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:03:41.285 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.285 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.285 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.285 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.285 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.285 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:03:41.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.285 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:03:41.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:03:41.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:03:41.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:03:41.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:03:41.288 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.288 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.289 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.289 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.289 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:41.289 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.25820584900001, 2024-09-13T07:03:41.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:03:41.289 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.289 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.289 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.289 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.289 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.289 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:03:41.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.290 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:41.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:41.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:03:41.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:03:41.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.25818466299999, 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.293 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:03:41.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:03:41.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:03:41.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:03:41.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:03:41.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.25816863, 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.297 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.25811939099999, 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.301 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:41.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:03:41.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:03:41.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:41.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:03:41.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:03:41.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.25810133100001, 2024-09-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:03:41.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:03:41.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:03:41.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:03:41.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:03:41.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.25808481599999, 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:03:41.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:03:41.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:03:41.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:03:41.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.258066451, 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:03:41.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:03:41.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:03:41.318 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.318 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.318 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.318 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.258050353, 2024-09-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.25803441799999, 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:41.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:41.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:03:41.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:03:41.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:03:41.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:03:41.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.258017921, 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.327 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:41.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.328 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:03:41.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.328 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:41.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:41.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:41.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:03:41.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:03:41.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:03:41.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:03:41.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:03:41.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:03:41.331 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.331 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.331 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.331 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.331 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.331 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:41.332 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:41.333 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:41.333 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:41.333 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:41.333 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:41.333 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:41.333 INFO:teuthology.orchestra.run.smithi118.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T07:03:41.333 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:41.333 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:41.334 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:41.335 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:41.329+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:41.434 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:41.431+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:41.484 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:41.484 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:41.484 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:41.484 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:41.494 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:41.535 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:41.533+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:41.603 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:41.603 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:41.603 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:41.603 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:41.603 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:41.613 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:41.636 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:41.633+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:41.720 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:41.720 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:41.720 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:41.720 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:41.720 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.720 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:03:41.720 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:41.720 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.66245329500001, 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.721 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:03:41.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:41.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:03:41.722 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:03:41.722 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.722 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.722 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.722 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.723 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:03:41.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:03:41.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:41.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:03:41.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:03:41.724 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.724 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.724 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.724 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:03:41.724 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:03:41.724 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.724 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.844236938000002, 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:03:41.725 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.726 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:03:41.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:41.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:03:41.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:03:41.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.727 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:03:41.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:03:41.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:03:41.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:03:41.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:03:41.729 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.729 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.729 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.729 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:41.729 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:41.729 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.844187550000001, 2024-09-13T07:03:41.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:03:41.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:03:41.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:03:41.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:03:41.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:03:41.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:03:41.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:03:41.733 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.733 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.733 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.734 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:41.734 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:41.734 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.844164843000001, 2024-09-13T07:03:41.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:03:41.734 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.734 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.734 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.735 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.735 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.735 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:03:41.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.735 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:41.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:41.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:03:41.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:03:41.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:03:41.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:03:41.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.738 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.844120650000001, 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:41.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:03:41.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.742 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.813141561000002, 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:03:41.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:03:41.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:03:41.746 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137457 3.18 3:1a596dcc:::rbd_data.1194af11889b.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.067519750999999, 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033734403000000003, 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137457 2024-09-13T07:03:41.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651523+0000", 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651531+0000", 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.328e-06 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651545+0000", 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4270000000000001e-05 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651555+0000", 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5210000000000008e-06 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651626+0000", 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1915999999999994e-05 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651761+0000", 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000134838 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:41.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.658577+0000", 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068151820000000004 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685181+0000", 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026604200000000001 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685193+0000", 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2128e-05 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:41.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685226+0000", 2024-09-13T07:03:41.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3504999999999999e-05 2024-09-13T07:03:41.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685259+0000", 2024-09-13T07:03:41.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2866e-05 2024-09-13T07:03:41.752 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.752 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.752 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.752 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138565 3.0 3:002d0425:::rbd_data.11949f62878b.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:41.752 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:41.753 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.054149258999999, 2024-09-13T07:03:41.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033394489999999999, 2024-09-13T07:03:41.753 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.753 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:41.753 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.753 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.753 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.753 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138565 2024-09-13T07:03:41.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.753 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664893+0000", 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664906+0000", 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2996000000000001e-05 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664936+0000", 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9609999999999999e-05 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664949+0000", 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383e-05 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665046+0000", 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6459000000000006e-05 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665259+0000", 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000213141 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.672522+0000", 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072632110000000003 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:41.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.697982+0000", 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025459699999999998 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698018+0000", 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6375999999999999e-05 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698166+0000", 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014818700000000001 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698290+0000", 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012333999999999999 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138587 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 33.005683591999997, 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019224502000000001, 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138587 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713360+0000", 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713368+0000", 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713383+0000", 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5176e-05 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713390+0000", 2024-09-13T07:03:41.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8669999999999996e-06 2024-09-13T07:03:41.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713449+0000", 2024-09-13T07:03:41.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8938999999999999e-05 2024-09-13T07:03:41.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713549+0000", 2024-09-13T07:03:41.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010009 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.716087+0000", 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025383820000000001 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732351+0000", 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016263564000000001 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732450+0000", 2024-09-13T07:03:41.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9529000000000002e-05 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732532+0000", 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1526000000000005e-05 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732585+0000", 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3819000000000001e-05 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138786 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.571468336999999, 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018847452000000001, 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138786 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147575+0000", 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:41.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147583+0000", 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147601+0000", 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7961e-05 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147610+0000", 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-13T07:03:41.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147661+0000", 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0447e-05 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147754+0000", 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2974999999999997e-05 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.150580+0000", 2024-09-13T07:03:41.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002826566 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166135+0000", 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015554772 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166225+0000", 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0258000000000002e-05 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166334+0000", 2024-09-13T07:03:41.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010816500000000001 2024-09-13T07:03:41.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166424+0000", 2024-09-13T07:03:41.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0116000000000003e-05 2024-09-13T07:03:41.769 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.769 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.769 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.769 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:139127 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:41.769 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:41.769 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.944417373, 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019295931999999998, 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139127 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774626+0000", 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774635+0000", 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5419999999999994e-06 2024-09-13T07:03:41.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774654+0000", 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8359000000000002e-05 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774663+0000", 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774718+0000", 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5315000000000001e-05 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774805+0000", 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7284e-05 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.779319+0000", 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045138310000000003 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793772+0000", 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014452692 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793821+0000", 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9563e-05 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793878+0000", 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6953999999999997e-05 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793923+0000", 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4781999999999999e-05 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.774 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:164691 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.660457499, 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015985102000000001, 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 164691 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058589+0000", 2024-09-13T07:03:41.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2510000000000001e-06 2024-09-13T07:03:41.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058594+0000", 2024-09-13T07:03:41.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9089999999999999e-06 2024-09-13T07:03:41.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058598+0000", 2024-09-13T07:03:41.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2069999999999997e-06 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064175+0000", 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055769909999999999 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064210+0000", 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4966999999999999e-05 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074572+0000", 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010362219000000001 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165201 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.65792957, 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013054266, 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.779 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165201 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:03:41.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061117+0000", 2024-09-13T07:03:41.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.12e-06 2024-09-13T07:03:41.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061125+0000", 2024-09-13T07:03:41.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8539999999999997e-06 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061129+0000", 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074092+0000", 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012963166999999999 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074136+0000", 2024-09-13T07:03:41.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3578000000000002e-05 2024-09-13T07:03:41.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074169+0000", 2024-09-13T07:03:41.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3581000000000003e-05 2024-09-13T07:03:41.783 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.783 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.783 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.783 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165202 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7831552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:41.783 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:03:41.783 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.657911895, 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013116245, 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165202 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061132+0000", 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061135+0000", 2024-09-13T07:03:41.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3410000000000001e-06 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061142+0000", 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7079999999999997e-06 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061147+0000", 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8930000000000004e-06 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074180+0000", 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013032346 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074217+0000", 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074249+0000", 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1711000000000002e-05 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.788 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165203 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:41.788 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:03:41.788 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.657893781, 2024-09-13T07:03:41.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013180211000000001, 2024-09-13T07:03:41.788 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.788 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.788 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165203 2024-09-13T07:03:41.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.788 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061150+0000", 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061153+0000", 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.108e-06 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061158+0000", 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0649999999999998e-06 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061162+0000", 2024-09-13T07:03:41.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0080000000000004e-06 2024-09-13T07:03:41.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074262+0000", 2024-09-13T07:03:41.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013100557000000001 2024-09-13T07:03:41.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074299+0000", 2024-09-13T07:03:41.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6981999999999999e-05 2024-09-13T07:03:41.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074331+0000", 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1801000000000003e-05 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165204 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8093696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.65787907, 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0132388, 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165204 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:03:41.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061167+0000", 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1409999999999999e-06 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061175+0000", 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5519999999999999e-06 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061179+0000", 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.172e-06 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074342+0000", 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013162821999999999 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074373+0000", 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1103999999999999e-05 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074404+0000", 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1359999999999998e-05 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165205 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:41.796 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:03:41.797 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.657862111, 2024-09-13T07:03:41.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013300635, 2024-09-13T07:03:41.797 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.797 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.797 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165205 2024-09-13T07:03:41.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.797 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061184+0000", 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0779999999999998e-06 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061192+0000", 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7209999999999994e-06 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061196+0000", 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4469999999999999e-06 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074417+0000", 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013220750999999999 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074452+0000", 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4814999999999998e-05 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074483+0000", 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1118000000000003e-05 2024-09-13T07:03:41.800 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165206 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.65784399, 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013364454, 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.801 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165206 2024-09-13T07:03:41.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.802 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:03:41.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:03:41.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061200+0000", 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061203+0000", 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.559e-06 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061210+0000", 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061214+0000", 2024-09-13T07:03:41.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0999999999999997e-06 2024-09-13T07:03:41.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074494+0000", 2024-09-13T07:03:41.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013279094 2024-09-13T07:03:41.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074530+0000", 2024-09-13T07:03:41.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6538999999999997e-05 2024-09-13T07:03:41.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074565+0000", 2024-09-13T07:03:41.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4972999999999999e-05 2024-09-13T07:03:41.805 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.805 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.805 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169377 3.0 3:07b65a69:::rbd_data.1194e181a50b.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 9.3621775950000004, 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.032744151999999999, 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169377 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:03:41.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:03:41.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356864+0000", 2024-09-13T07:03:41.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:41.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356880+0000", 2024-09-13T07:03:41.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6022999999999999e-05 2024-09-13T07:03:41.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356916+0000", 2024-09-13T07:03:41.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5921999999999999e-05 2024-09-13T07:03:41.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356931+0000", 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4311000000000001e-05 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357011+0000", 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9992000000000003e-05 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357143+0000", 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000131822 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.361578+0000", 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00443525 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389466+0000", 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027888342999999999 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389529+0000", 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2460999999999997e-05 2024-09-13T07:03:41.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:41.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389574+0000", 2024-09-13T07:03:41.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5405999999999999e-05 2024-09-13T07:03:41.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389611+0000", 2024-09-13T07:03:41.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7122000000000001e-05 2024-09-13T07:03:41.811 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.811 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.811 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169731 3.0 3:031f8e93:::rbd_data.1194e181a50b.0000000000000006:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 8.4377517970000007, 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.021389756999999999, 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169731 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281291+0000", 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281302+0000", 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0651e-05 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281324+0000", 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1713999999999999e-05 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281340+0000", 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281450+0000", 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110086 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281593+0000", 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014306 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.286356+0000", 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047626320000000002 2024-09-13T07:03:41.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302404+0000", 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016048605000000001 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302476+0000", 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1483000000000007e-05 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302578+0000", 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101883 2024-09-13T07:03:41.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302683+0000", 2024-09-13T07:03:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104681 2024-09-13T07:03:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:41.817 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:41.817 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:41.818 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:41.819 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:41.817+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:41.919 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:41.917+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:41.933 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:41.933 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:41.934 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:41.934 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:41.944 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:42.021 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:42.017+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:42.040 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:03:42.059 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:42.059 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:42.059 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:42.059 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:42.059 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:42.069 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:42.122 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:42.117+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:42.174 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:42.174 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:42.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.052609132000001, 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.175 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:03:42.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:03:42.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:03:42.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:03:42.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:03:42.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:03:42.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:03:42.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:03:42.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:03:42.179 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.179 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.179 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.179 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.179 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:42.180 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.052588654999994, 2024-09-13T07:03:42.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:03:42.180 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.180 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.180 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.180 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.180 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.180 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:03:42.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.180 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:03:42.183 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.184 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.184 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.184 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.184 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:42.184 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045892051999999, 2024-09-13T07:03:42.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:03:42.184 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.184 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.185 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.185 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.185 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.185 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:03:42.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.185 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:42.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:42.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:42.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:42.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:03:42.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:03:42.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.187 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.187 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:03:42.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:03:42.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:03:42.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:03:42.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:03:42.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:03:42.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:03:42.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:03:42.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:03:42.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:03:42.189 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.189 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.189 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.190 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.190 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:42.190 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045879983000006, 2024-09-13T07:03:42.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:03:42.190 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.190 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.190 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.190 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.190 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.190 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:03:42.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.191 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:42.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:42.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:42.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:42.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:03:42.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:03:42.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:42.195 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045867540000003, 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:03:42.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.199 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045849942000004, 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:42.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:42.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:42.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:03:42.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:03:42.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:03:42.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:03:42.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:03:42.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:03:42.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:03:42.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:03:42.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:03:42.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:03:42.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:03:42.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:03:42.205 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.205 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.205 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.205 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.205 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045839239000003, 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:42.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:42.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:03:42.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:03:42.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:03:42.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:03:42.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:03:42.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:03:42.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:03:42.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:03:42.210 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.210 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.210 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.210 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.210 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:42.211 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045795306000002, 2024-09-13T07:03:42.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:03:42.211 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.211 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.211 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.211 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.211 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.211 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:03:42.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.211 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:42.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:42.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:03:42.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:03:42.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:03:42.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:03:42.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:03:42.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:03:42.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045775520000007, 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.215 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:03:42.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:03:42.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045759770000004, 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:03:42.219 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:42.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:42.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:42.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:03:42.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:03:42.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:03:42.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:03:42.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:03:42.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:03:42.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:03:42.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:03:42.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:03:42.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:03:42.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045748517999996, 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.224 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:42.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:03:42.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:03:42.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:03:42.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:03:42.228 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.228 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.228 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.228 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.229 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:42.229 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045737919000004, 2024-09-13T07:03:42.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:03:42.229 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.229 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.229 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.229 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.229 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.229 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:03:42.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.230 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:42.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:42.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:42.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:42.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:03:42.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:03:42.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:03:42.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:03:42.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:03:42.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:03:42.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:03:42.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:03:42.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:03:42.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:03:42.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:03:42.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:03:42.234 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.234 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.234 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.234 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.234 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:42.235 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045728425999997, 2024-09-13T07:03:42.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:03:42.235 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.235 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.235 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.235 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.235 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.235 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:03:42.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.236 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:42.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:03:42.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:03:42.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:03:42.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:03:42.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:03:42.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:03:42.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:03:42.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:03:42.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:03:42.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:03:42.240 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.240 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.240 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.240 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.241 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:42.241 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045718882000003, 2024-09-13T07:03:42.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:03:42.241 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.241 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.241 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.241 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.241 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.241 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:03:42.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.242 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:42.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:42.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:42.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:42.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:03:42.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:03:42.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:03:42.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:03:42.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:03:42.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:03:42.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:03:42.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:03:42.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:03:42.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:03:42.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:03:42.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:03:42.245 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045686278999995, 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.246 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.045673256000001, 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.250 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.251 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.251 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.251 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.251 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:03:42.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.251 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:42.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:03:42.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:03:42.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:03:42.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:03:42.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:03:42.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:03:42.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:03:42.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:03:42.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:03:42.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:03:42.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:03:42.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:03:42.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.042322271000003, 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.255 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:03:42.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.256 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:42.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:42.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:03:42.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:42.259 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.042301993999999, 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:03:42.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.263 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.264 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:42.264 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:42.264 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.042281793000001, 2024-09-13T07:03:42.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:03:42.264 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.264 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:42.264 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:42.264 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:42.264 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.264 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:03:42.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:03:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:03:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:03:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:03:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:03:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:03:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:03:42.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:03:42.268 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.268 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.268 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.268 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:166663 3.b e135/117 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head v 135'409, mlcod=132'407)", 2024-09-13T07:03:42.268 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:03:42.268 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 12.759692828, 2024-09-13T07:03:42.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045108144000000003, 2024-09-13T07:03:42.268 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:42.269 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:42.269 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:42.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:42.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:03:42.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:42.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:03:42.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:42.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414172+0000", 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427200+0000", 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013028098 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427229+0000", 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9178000000000001e-05 2024-09-13T07:03:42.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427239+0000", 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6870000000000006e-06 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427393+0000", 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015382899999999999 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427451+0000", 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8301000000000003e-05 2024-09-13T07:03:42.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459232+0000", 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031780695999999997 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459282+0000", 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0701000000000001e-05 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:42.272 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:42.273 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:42.274 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:42.275 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:03:42.269+0000 7f02732a5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22301) UID: 0 2024-09-13T07:03:42.375 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:42.371+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:42.389 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:42.389 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:42.389 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:42.389 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:42.399 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:42.476 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:42.473+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:42.502 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:42.502 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:42.502 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:42.502 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:42.502 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:42.512 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:42.577 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:42.575+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:42.618 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.015059604000001, 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:03:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:03:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:03:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:03:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:03:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.015028503000003, 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:03:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.015006188999998, 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:03:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:03:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:03:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:03:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:03:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:03:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:03:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:03:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014983158, 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:03:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014958608000001, 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:03:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:03:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:03:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:03:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:03:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:03:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:03:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:03:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:03:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:03:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:03:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014922898999998, 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:03:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014898219999999, 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:03:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:03:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014864946000003, 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:03:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.656 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.656 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:42.656 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014838158000003, 2024-09-13T07:03:42.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:03:42.656 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.656 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.656 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:03:42.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:42.660 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014813603999997, 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:03:42.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:03:42.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014783835000003, 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:03:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:42.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:03:42.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.669 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014723992, 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:03:42.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:03:42.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014657141999997, 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:03:42.674 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:03:42.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:03:42.678 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014624640000001, 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.679 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:03:42.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.680 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:42.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:42.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:03:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014590019000003, 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:03:42.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:03:42.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:03:42.688 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.688 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.688 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.688 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.688 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:42.688 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014539788, 2024-09-13T07:03:42.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:03:42.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:03:42.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:03:42.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:03:42.693 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.693 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.693 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.693 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.693 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:42.693 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014491602, 2024-09-13T07:03:42.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:03:42.693 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.693 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:42.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:03:42.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:03:42.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014464963999998, 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:03:42.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:03:42.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:03:42.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:03:42.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:03:42.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.014418128999999, 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.702 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:03:42.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:03:42.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:03:42.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:03:42.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:03:42.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:03:42.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.011063712000002, 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:42.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:42.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:42.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:42.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:42.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:42.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:42.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:42.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:42.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:42.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:03:42.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:03:42.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:42.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:42.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:42.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:03:42.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:03:42.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.711 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:42.711 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:42.711 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:42.713 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:03:42.713 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:42.713+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:42.814 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:42.809+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:42.824 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:42.824 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:42.825 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:42.825 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:42.835 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:03:42.914 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:42.913+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:42.939 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:42.940 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:42.940 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:42.940 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:42.940 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:42.949 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:03:43.015 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:43.015+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:43.063 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:43.063 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:43.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:16688 3.17 3:eea0a1d0:::rbd_data.1194b0f2ab1d.0000000000000055:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1376969~6179] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 113.944484614, 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96435641000000005, 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 16688 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.064 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:43.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118588+0000", 2024-09-13T07:03:43.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118586+0000", 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118605+0000", 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8414999999999999e-05 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118634+0000", 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9723999999999999e-05 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118647+0000", 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.274e-05 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118771+0000", 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012371999999999999 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:49.118798+0000", 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7483000000000001e-05 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079299+0000", 2024-09-13T07:03:43.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96050073999999996 2024-09-13T07:03:43.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:43.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.079415+0000", 2024-09-13T07:03:43.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116079 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.080112+0000", 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069638700000000003 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.081678+0000", 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001566797 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082880+0000", 2024-09-13T07:03:43.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012020900000000001 2024-09-13T07:03:43.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:43.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082918+0000", 2024-09-13T07:03:43.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7299999999999999e-05 2024-09-13T07:03:43.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:50.082945+0000", 2024-09-13T07:03:43.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6772e-05 2024-09-13T07:03:43.070 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.070 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.070 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.071 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129532 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.071 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:43.071 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.448081727000002, 2024-09-13T07:03:43.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89889205500000002, 2024-09-13T07:03:43.071 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.071 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.071 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.071 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.071 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.072 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129532 2024-09-13T07:03:43.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.072 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:43.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:43.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614991+0000", 2024-09-13T07:03:43.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.614994+0000", 2024-09-13T07:03:43.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.044e-06 2024-09-13T07:03:43.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615000+0000", 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9229999999999996e-06 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615005+0000", 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7670000000000003e-06 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615060+0000", 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4561000000000002e-05 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615069+0000", 2024-09-13T07:03:43.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1540000000000008e-06 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513869+0000", 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89879977700000002 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.513883+0000", 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4361e-05 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.075 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129533 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447986393999997, 2024-09-13T07:03:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.899270177, 2024-09-13T07:03:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129533 2024-09-13T07:03:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:43.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:43.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615086+0000", 2024-09-13T07:03:43.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615090+0000", 2024-09-13T07:03:43.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.856e-06 2024-09-13T07:03:43.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615096+0000", 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.703e-06 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615102+0000", 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.378e-06 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615140+0000", 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7966999999999997e-05 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615147+0000", 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8800000000000002e-06 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514342+0000", 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919565300000004 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514357+0000", 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4355000000000001e-05 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.080 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.081 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.081 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129534 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.081 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:43.081 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447913495999998, 2024-09-13T07:03:43.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89923080600000005, 2024-09-13T07:03:43.081 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.082 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.082 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129534 2024-09-13T07:03:43.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.082 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:43.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:43.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615159+0000", 2024-09-13T07:03:43.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615162+0000", 2024-09-13T07:03:43.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3569999999999998e-06 2024-09-13T07:03:43.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615169+0000", 2024-09-13T07:03:43.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.455e-06 2024-09-13T07:03:43.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615175+0000", 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3679999999999998e-06 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615212+0000", 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6733999999999998e-05 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615218+0000", 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7569999999999998e-06 2024-09-13T07:03:43.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514383+0000", 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916408000000003 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514390+0000", 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5429999999999996e-06 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129535 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.086 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447840079999999, 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89919205499999999, 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129535 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615233+0000", 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615232+0000", 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615236+0000", 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:43.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615244+0000", 2024-09-13T07:03:43.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7369999999999995e-06 2024-09-13T07:03:43.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615250+0000", 2024-09-13T07:03:43.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2270000000000001e-06 2024-09-13T07:03:43.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4627000000000002e-05 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5089999999999993e-06 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514412+0000", 2024-09-13T07:03:43.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89907890599999996 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514425+0000", 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3138e-05 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129536 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447806602999997, 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89918503500000002, 2024-09-13T07:03:43.091 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.092 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.092 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.092 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.092 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.092 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129536 2024-09-13T07:03:43.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.092 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615266+0000", 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615268+0000", 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8370000000000001e-06 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615288+0000", 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9945999999999998e-05 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615294+0000", 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3340000000000001e-06 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615374+0000", 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9988999999999996e-05 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615381+0000", 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2930000000000003e-06 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514444+0000", 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89906347499999995 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514451+0000", 2024-09-13T07:03:43.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7630000000000003e-06 2024-09-13T07:03:43.096 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.096 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.096 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.096 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129537 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.096 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:43.096 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447758051000001, 2024-09-13T07:03:43.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89916833399999996, 2024-09-13T07:03:43.096 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.096 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129537 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615315+0000", 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615314+0000", 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615317+0000", 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1410000000000001e-06 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615324+0000", 2024-09-13T07:03:43.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3589999999999996e-06 2024-09-13T07:03:43.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615329+0000", 2024-09-13T07:03:43.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-13T07:03:43.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615408+0000", 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8678999999999997e-05 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615414+0000", 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514471+0000", 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905719200000001 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514483+0000", 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1691999999999999e-05 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129538 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447739511999998, 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89917603999999995, 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.101 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.102 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.102 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.102 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129538 2024-09-13T07:03:43.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.102 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:43.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:43.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615333+0000", 2024-09-13T07:03:43.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615335+0000", 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4090000000000001e-06 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615346+0000", 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0866000000000001e-05 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615424+0000", 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7628999999999998e-05 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615459+0000", 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5046999999999998e-05 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615466+0000", 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7610000000000002e-06 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514503+0000", 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89903750299999996 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514509+0000", 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2140000000000001e-06 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129539 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447594760999998, 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89905735799999997, 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.106 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.107 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.107 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129539 2024-09-13T07:03:43.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.107 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:43.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615478+0000", 2024-09-13T07:03:43.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615477+0000", 2024-09-13T07:03:43.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615481+0000", 2024-09-13T07:03:43.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6490000000000001e-06 2024-09-13T07:03:43.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615488+0000", 2024-09-13T07:03:43.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4359999999999999e-06 2024-09-13T07:03:43.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615493+0000", 2024-09-13T07:03:43.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3120000000000001e-06 2024-09-13T07:03:43.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615529+0000", 2024-09-13T07:03:43.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6468999999999999e-05 2024-09-13T07:03:43.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615536+0000", 2024-09-13T07:03:43.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9789999999999996e-06 2024-09-13T07:03:43.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514529+0000", 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89899276400000006 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514535+0000", 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2920000000000001e-06 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129540 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:43.111 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447523767, 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89901438700000003, 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129540 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615549+0000", 2024-09-13T07:03:43.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615548+0000", 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615552+0000", 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8349999999999997e-06 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615559+0000", 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615564+0000", 2024-09-13T07:03:43.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2360000000000004e-06 2024-09-13T07:03:43.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615600+0000", 2024-09-13T07:03:43.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6637000000000002e-05 2024-09-13T07:03:43.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615607+0000", 2024-09-13T07:03:43.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.669e-06 2024-09-13T07:03:43.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514557+0000", 2024-09-13T07:03:43.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89894939399999996 2024-09-13T07:03:43.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514563+0000", 2024-09-13T07:03:43.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8560000000000001e-06 2024-09-13T07:03:43.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129541 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447451155000003, 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89896824600000003, 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.117 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129541 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615622+0000", 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615621+0000", 2024-09-13T07:03:43.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615625+0000", 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615631+0000", 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3890000000000002e-06 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615636+0000", 2024-09-13T07:03:43.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3489999999999999e-06 2024-09-13T07:03:43.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615671+0000", 2024-09-13T07:03:43.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4570000000000003e-05 2024-09-13T07:03:43.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615678+0000", 2024-09-13T07:03:43.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8510000000000001e-06 2024-09-13T07:03:43.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514583+0000", 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89890530599999996 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514590+0000", 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7469999999999999e-06 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129542 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447377664999998, 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89892092400000001, 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129542 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:43.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:43.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615695+0000", 2024-09-13T07:03:43.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615698+0000", 2024-09-13T07:03:43.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.664e-06 2024-09-13T07:03:43.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615705+0000", 2024-09-13T07:03:43.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:43.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615711+0000", 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6470000000000003e-06 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615752+0000", 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0389999999999998e-05 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615759+0000", 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9230000000000002e-06 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514609+0000", 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89884984899999998 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514616+0000", 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7299999999999999e-06 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129543 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:43.126 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447298791999998, 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89886872600000001, 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129543 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615774+0000", 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615773+0000", 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615777+0000", 2024-09-13T07:03:43.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1860000000000002e-06 2024-09-13T07:03:43.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615785+0000", 2024-09-13T07:03:43.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2710000000000003e-06 2024-09-13T07:03:43.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615790+0000", 2024-09-13T07:03:43.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3619999999999997e-06 2024-09-13T07:03:43.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615828+0000", 2024-09-13T07:03:43.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8059999999999998e-05 2024-09-13T07:03:43.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615836+0000", 2024-09-13T07:03:43.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8809999999999996e-06 2024-09-13T07:03:43.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514636+0000", 2024-09-13T07:03:43.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89880009599999999 2024-09-13T07:03:43.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514643+0000", 2024-09-13T07:03:43.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:43.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.131 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.131 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129544 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.131 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:43.131 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447223270000002, 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89881970799999999, 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129544 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615849+0000", 2024-09-13T07:03:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615853+0000", 2024-09-13T07:03:43.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8600000000000003e-06 2024-09-13T07:03:43.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615860+0000", 2024-09-13T07:03:43.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1709999999999996e-06 2024-09-13T07:03:43.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615867+0000", 2024-09-13T07:03:43.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7020000000000004e-06 2024-09-13T07:03:43.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615905+0000", 2024-09-13T07:03:43.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8265e-05 2024-09-13T07:03:43.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615913+0000", 2024-09-13T07:03:43.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-13T07:03:43.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514663+0000", 2024-09-13T07:03:43.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89874952500000005 2024-09-13T07:03:43.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514669+0000", 2024-09-13T07:03:43.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-13T07:03:43.136 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.136 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.136 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.137 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129545 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.137 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:43.137 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447145097000003, 2024-09-13T07:03:43.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89876792599999999, 2024-09-13T07:03:43.137 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.137 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.137 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129545 2024-09-13T07:03:43.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.137 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615928+0000", 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615927+0000", 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:43.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615931+0000", 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8010000000000003e-06 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615939+0000", 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6310000000000003e-06 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615947+0000", 2024-09-13T07:03:43.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7090000000000002e-06 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615985+0000", 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8389000000000001e-05 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.615993+0000", 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2209999999999997e-06 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514689+0000", 2024-09-13T07:03:43.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89869535199999995 2024-09-13T07:03:43.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514696+0000", 2024-09-13T07:03:43.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8179999999999998e-06 2024-09-13T07:03:43.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.141 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129546 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.447065008999999, 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89872448199999999, 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129546 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:43.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616008+0000", 2024-09-13T07:03:43.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616007+0000", 2024-09-13T07:03:43.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616011+0000", 2024-09-13T07:03:43.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1659999999999996e-06 2024-09-13T07:03:43.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616018+0000", 2024-09-13T07:03:43.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0829999999999998e-06 2024-09-13T07:03:43.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616024+0000", 2024-09-13T07:03:43.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8170000000000004e-06 2024-09-13T07:03:43.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616063+0000", 2024-09-13T07:03:43.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8708000000000003e-05 2024-09-13T07:03:43.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616070+0000", 2024-09-13T07:03:43.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6299999999999998e-06 2024-09-13T07:03:43.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514721+0000", 2024-09-13T07:03:43.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89865028800000002 2024-09-13T07:03:43.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514732+0000", 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1452e-05 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129547 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.446989680999998, 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89868529799999997, 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.147 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.148 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129547 2024-09-13T07:03:43.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.148 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:43.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:43.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616083+0000", 2024-09-13T07:03:43.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616086+0000", 2024-09-13T07:03:43.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.394e-06 2024-09-13T07:03:43.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616093+0000", 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1130000000000003e-06 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616100+0000", 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1520000000000003e-06 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616138+0000", 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8084000000000003e-05 2024-09-13T07:03:43.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616146+0000", 2024-09-13T07:03:43.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3010000000000001e-06 2024-09-13T07:03:43.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:43.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89861259299999996 2024-09-13T07:03:43.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514768+0000", 2024-09-13T07:03:43.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0175e-05 2024-09-13T07:03:43.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.152 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.153 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129548 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.153 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:43.153 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.446905622000003, 2024-09-13T07:03:43.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89862847899999998, 2024-09-13T07:03:43.153 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.153 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.154 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129548 2024-09-13T07:03:43.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.154 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:43.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616167+0000", 2024-09-13T07:03:43.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616166+0000", 2024-09-13T07:03:43.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616170+0000", 2024-09-13T07:03:43.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8619999999999996e-06 2024-09-13T07:03:43.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616177+0000", 2024-09-13T07:03:43.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0450000000000003e-06 2024-09-13T07:03:43.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616183+0000", 2024-09-13T07:03:43.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.139e-06 2024-09-13T07:03:43.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616221+0000", 2024-09-13T07:03:43.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7320000000000002e-05 2024-09-13T07:03:43.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616228+0000", 2024-09-13T07:03:43.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2980000000000003e-06 2024-09-13T07:03:43.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514789+0000", 2024-09-13T07:03:43.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89856044599999996 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514796+0000", 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0809999999999996e-06 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129549 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.446832159000003, 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89859381900000002, 2024-09-13T07:03:43.159 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129549 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616241+0000", 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616240+0000", 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616244+0000", 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4149999999999999e-06 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616251+0000", 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616256+0000", 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3299999999999998e-06 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616302+0000", 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5646999999999999e-05 2024-09-13T07:03:43.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616309+0000", 2024-09-13T07:03:43.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5530000000000004e-06 2024-09-13T07:03:43.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514825+0000", 2024-09-13T07:03:43.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89851528400000003 2024-09-13T07:03:43.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514834+0000", 2024-09-13T07:03:43.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8349999999999999e-06 2024-09-13T07:03:43.164 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.164 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.165 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129550 3.1f 3.8c45a89f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:43.165 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:43.165 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.446747780000003, 2024-09-13T07:03:43.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89854296199999995, 2024-09-13T07:03:43.166 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.166 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.167 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129550 2024-09-13T07:03:43.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.167 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:43.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616325+0000", 2024-09-13T07:03:43.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616324+0000", 2024-09-13T07:03:43.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:43.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616328+0000", 2024-09-13T07:03:43.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7479999999999999e-06 2024-09-13T07:03:43.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616335+0000", 2024-09-13T07:03:43.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9280000000000004e-06 2024-09-13T07:03:43.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616342+0000", 2024-09-13T07:03:43.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2690000000000002e-06 2024-09-13T07:03:43.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616380+0000", 2024-09-13T07:03:43.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:03:43.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.616387+0000", 2024-09-13T07:03:43.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.075e-06 2024-09-13T07:03:43.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514861+0000", 2024-09-13T07:03:43.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.89847450799999995 2024-09-13T07:03:43.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514868+0000", 2024-09-13T07:03:43.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6050000000000003e-06 2024-09-13T07:03:43.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.175 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.175 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.175 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:43.176 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 7, 2, 0, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:03:43.176 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:03:43.176 INFO:tasks.thrashosds.thrasher:Killing osd 5, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T07:03:43.177 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T07:03:43.177 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:03:43.178 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:43.175+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:43.178 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:43.278 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:43.279+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:43.300 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172852 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [call lock.unlock in=52b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-13T07:03:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:03:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.10247790399999999, 2024-09-13T07:03:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.102502041, 2024-09-13T07:03:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for sub ops", 2024-09-13T07:03:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172852 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198696+0000", 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198710+0000", 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3548e-05 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198739+0000", 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9606000000000001e-05 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198752+0000", 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3033e-05 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198869+0000", 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116552 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199018+0000", 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014923700000000001 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199308+0000", 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028947199999999998 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.200731+0000", 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014229799999999999 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-13T07:03:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-13T07:03:43.306 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:43.310 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:43.336 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:03:43.381 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:43.383+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:43.416 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:43.416 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:43.416 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:43.416 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:43.416 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:43.426 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:43.480 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:43.477+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:43.490 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:43.509 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:43.531 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:43.540 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:43.541 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:43.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:43.541 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:43.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.541 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:03:43.541 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:43.541 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 137.485130881, 2024-09-13T07:03:43.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:43.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:03:43.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:03:43.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:03:43.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:03:43.545 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.545 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.545 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.545 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:43.545 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:43.546 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.327249647, 2024-09-13T07:03:43.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:03:43.546 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.546 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:43.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.546 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:03:43.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:03:43.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:03:43.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:03:43.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:03:43.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:43.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:03:43.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:03:43.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.551 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.30147993600001, 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:03:43.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:03:43.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:43.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:03:43.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:03:43.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:43.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:03:43.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:03:43.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:03:43.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.28032877, 2024-09-13T07:03:43.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:43.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:03:43.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:43.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:03:43.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:03:43.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:03:43.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:03:43.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:03:43.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:03:43.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:03:43.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:03:43.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:43.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:03:43.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:03:43.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:43.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:03:43.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:03:43.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:03:43.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:03:43.563 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.563 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.563 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.564 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:03:43.564 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:43.564 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 95.163284007000001, 2024-09-13T07:03:43.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:03:43.564 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.564 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:43.564 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.564 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.564 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.564 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:03:43.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.564 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:03:43.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:03:43.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:43.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:03:43.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:03:43.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:43.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:03:43.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:03:43.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:43.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:03:43.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:03:43.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:03:43.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:03:43.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.569 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.569 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:43.569 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 76.878286762000002, 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:43.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:03:43.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:03:43.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:03:43.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:43.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:03:43.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:03:43.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:03:43.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:03:43.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.626577624000006, 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:03:43.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:03:43.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.626559589999999, 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:43.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:03:43.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:03:43.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:03:43.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:03:43.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.582 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.582 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:43.583 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:43.583 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.626543510000005, 2024-09-13T07:03:43.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:03:43.583 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.583 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:43.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.583 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:03:43.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.583 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:43.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:43.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:03:43.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:03:43.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:03:43.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:03:43.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:03:43.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:43.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:03:43.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:03:43.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.626527834000001, 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.587 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:43.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:03:43.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:03:43.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:03:43.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:03:43.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.626504912000001, 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:03:43.591 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:03:43.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:03:43.595 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.595 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.595 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.595 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:43.595 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:43.595 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.626488815000002, 2024-09-13T07:03:43.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:03:43.595 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:43.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:03:43.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:03:43.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:03:43.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:03:43.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:03:43.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:03:43.599 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.599 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.599 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.599 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107067 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:43.599 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:43.599 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.625928798000004, 2024-09-13T07:03:43.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04441055, 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107067 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913375+0000", 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912832+0000", 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9994574 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913379+0000", 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00054719999999999997 2024-09-13T07:03:43.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913386+0000", 2024-09-13T07:03:43.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4549999999999998e-06 2024-09-13T07:03:43.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913393+0000", 2024-09-13T07:03:43.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6259999999999997e-06 2024-09-13T07:03:43.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957776+0000", 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044383143999999999 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957785+0000", 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9220000000000003e-06 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107068 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:43.603 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.625904367000004, 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044410502999999997, 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107068 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913399+0000", 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913401+0000", 2024-09-13T07:03:43.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7480000000000002e-06 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913407+0000", 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7520000000000002e-06 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913412+0000", 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8550000000000001e-06 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957802+0000", 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044389606999999998 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957809+0000", 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9160000000000001e-06 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:43.607 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.625887203000005, 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:43.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:43.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:03:43.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:03:43.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:03:43.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:03:43.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:03:43.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.611 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.611 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:43.612 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:43.612 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.625844427000004, 2024-09-13T07:03:43.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:03:43.612 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.612 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.612 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:03:43.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.612 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:43.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:43.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:03:43.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.468826657000001, 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:03:43.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:03:43.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:03:43.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:03:43.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.619 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.620 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:03:43.620 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:43.620 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.616449726999999, 2024-09-13T07:03:43.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:03:43.620 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.620 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:43.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.620 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:03:43.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.620 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:03:43.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:03:43.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.624 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.624 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:164179 3.14 e135/67 3:2fc485bd:::rbd_data.11942e4f5fc7.0000000000000002:head v 135'401, mlcod=132'398)", 2024-09-13T07:03:43.624 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:43.624 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.593509273, 2024-09-13T07:03:43.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063492174999999998, 2024-09-13T07:03:43.624 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945792+0000", 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954811+0000", 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090187940000000001 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954840+0000", 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8659000000000001e-05 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954852+0000", 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1966e-05 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954938+0000", 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6539000000000006e-05 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954975+0000", 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6303999999999997e-05 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009206+0000", 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.054231044999999999 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009286+0000", 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700999999999999e-05 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:165953 3.d 3:b3a20b45:::rbd_data.11942e4f5fc7.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8796117~39260] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:43.628 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.317061714999999, 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.095290105, 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 165953 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222240+0000", 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258929+0000", 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.036688401000000002 2024-09-13T07:03:43.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258971+0000", 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2212000000000002e-05 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258989+0000", 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7674000000000001e-05 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259072+0000", 2024-09-13T07:03:43.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3656e-05 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259297+0000", 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022450700000000001 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.260463+0000", 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011665340000000001 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:43.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.316214+0000", 2024-09-13T07:03:43.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.055750268999999998 2024-09-13T07:03:43.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:43.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317335+0000", 2024-09-13T07:03:43.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011211719999999999 2024-09-13T07:03:43.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:43.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317451+0000", 2024-09-13T07:03:43.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011616 2024-09-13T07:03:43.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:43.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:43.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:43.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317532+0000", 2024-09-13T07:03:43.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0860999999999997e-05 2024-09-13T07:03:43.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.634 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:43.634 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:43.634 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:43.635 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:43.635 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:43.636 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:43.633+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:43.636 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:43.736 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:43.733+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:43.750 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:43.750 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:43.760 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:43.837 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:43.835+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:43.865 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:43.866 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:43.875 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:43.938 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:43.937+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:43.977 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:43.977 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:43.986 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:44.039 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:44.035+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:44.056 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:03:44.100 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:44.100 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:44.111 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:44.153 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:44.151+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:44.241 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:44.237+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:44.257 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:44.257 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:44.267 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:44.342 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:44.337+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:44.367 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:44.367 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:44.376 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:44.443 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:44.443+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:44.477 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:44.477 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:44.487 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:44.544 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:44.543+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:44.589 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:44.589 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:44.598 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:44.644 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:44.641+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:44.702 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:44.702 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:44.712 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:44.757 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:44.759+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:44.821 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:44.821 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:44.832 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:44.846 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:44.847+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:44.938 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:44.938 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:44.947 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:44.948 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:44.945+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:45.048 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:45.047+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:45.098 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:45.099 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:45.107 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:03:45.110 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:45.155 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:45.153+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:45.218 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:45.218 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:45.228 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:45.250 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:45.251+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:45.338 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:45.338 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:45.349 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:45.351 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:45.351+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:45.452 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:45.451+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:45.454 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:45.455 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:45.464 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:45.553 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:45.549+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:45.574 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:45.574 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:45.574 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:45.574 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:45.584 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:45.654 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:45.655+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:45.737 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:45.738 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:45.738 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:45.738 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:45.738 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:45.747 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:45.755 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:45.755+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:45.856 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:45.857 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:45.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:45.857 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:45.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.857 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:45.857 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:45.857 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 139.79562297800001, 2024-09-13T07:03:45.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:03:45.857 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.857 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:45.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.858 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:03:45.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.858 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:45.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:45.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:03:45.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:03:45.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.237839100000002, 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:03:45.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:03:45.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.237816132999995, 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.867 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:45.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:03:45.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:03:45.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:03:45.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:03:45.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:03:45.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:03:45.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:03:45.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:03:45.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:03:45.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:03:45.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:03:45.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.237798955000002, 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.872 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:03:45.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:03:45.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:03:45.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:03:45.876 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.876 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.876 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.876 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:45.876 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:45.877 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.237785090000003, 2024-09-13T07:03:45.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:03:45.877 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.877 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.877 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.877 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.877 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.877 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:03:45.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.877 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.881 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:45.881 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:45.881 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.237771945000006, 2024-09-13T07:03:45.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:03:45.881 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.881 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.881 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.881 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.881 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.881 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:03:45.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.881 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:03:45.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:03:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:03:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:03:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:03:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:03:45.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:03:45.885 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.885 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.885 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.885 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:45.885 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:45.885 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.237758737999997, 2024-09-13T07:03:45.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:03:45.885 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.885 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.886 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.886 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.886 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.886 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:03:45.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.886 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:45.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:03:45.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:03:45.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:03:45.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:03:45.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:03:45.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:03:45.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:03:45.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:03:45.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:03:45.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:03:45.889 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.889 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.889 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.890 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:45.890 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:45.890 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.237744671000002, 2024-09-13T07:03:45.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:03:45.890 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.890 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.890 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.890 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.890 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.890 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:03:45.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:45.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:03:45.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:03:45.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:03:45.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:03:45.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:03:45.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:03:45.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:03:45.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.894 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.894 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:45.894 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:45.894 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.237724657000001, 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:45.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:45.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:03:45.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:03:45.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:03:45.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:03:45.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:03:45.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.237704906999994, 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.237680100999995, 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:03:45.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:03:45.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:03:45.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:03:45.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:03:45.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:45.907 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.237655836000002, 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:45.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:03:45.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.237614102999999, 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:45.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:45.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:03:45.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:03:45.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:03:45.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:03:45.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:03:45.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:03:45.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:03:45.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:03:45.916 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.916 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.916 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.916 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:45.916 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.237577814000005, 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:45.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:45.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:03:45.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:03:45.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:03:45.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:03:45.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:03:45.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:03:45.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:03:45.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 63.794474481999998, 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:03:45.921 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:03:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:03:45.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:03:45.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:03:45.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:03:45.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:03:45.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:03:45.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:03:45.925 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.925 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.925 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 63.794457600000001, 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:03:45.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:03:45.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:03:45.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:03:45.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:03:45.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:03:45.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:03:45.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:03:45.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:03:45.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:03:45.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:03:45.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:03:45.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.930 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.930 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:45.930 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 63.793407043000002, 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:45.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:03:45.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:03:45.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:03:45.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:03:45.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:03:45.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:03:45.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:03:45.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:03:45.935 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.935 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.935 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.935 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:45.935 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:45.935 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 63.791732494000001, 2024-09-13T07:03:45.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:03:45.935 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.935 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:45.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:03:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:03:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:03:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:03:45.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:03:45.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:03:45.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:03:45.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:03:45.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:45.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:03:45.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:03:45.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:03:45.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:03:45.939 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.939 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.939 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.939 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.271150104, 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:03:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:03:45.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:03:45.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:03:45.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:03:45.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:03:45.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:03:45.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:03:45.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:03:45.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:45.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:03:45.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:03:45.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:03:45.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:03:45.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:03:45.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:03:45.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172880 2.2 2:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e149)", 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.7168538579999999, 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.97218948500000002, 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:45.945 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172880 2024-09-13T07:03:45.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.946 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:45.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:45.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:03:45.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:45.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:03:45.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:45.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137687+0000", 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137702+0000", 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5023e-05 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137730+0000", 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7719e-05 2024-09-13T07:03:45.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137748+0000", 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8189000000000002e-05 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137891+0000", 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014330699999999999 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137903+0000", 2024-09-13T07:03:45.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1688999999999999e-05 2024-09-13T07:03:45.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142887+0000", 2024-09-13T07:03:45.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004984006 2024-09-13T07:03:45.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:45.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142898+0000", 2024-09-13T07:03:45.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.079e-05 2024-09-13T07:03:45.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:45.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109703+0000", 2024-09-13T07:03:45.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96680553899999999 2024-09-13T07:03:45.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:45.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:45.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:45.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109879+0000", 2024-09-13T07:03:45.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017579800000000001 2024-09-13T07:03:45.950 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.950 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:45.951 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:45.951 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:45.952 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:45.855+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:45.953 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:45.957 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:45.953+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:46.058 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:46.059+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:46.064 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:46.065 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:46.065 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:46.065 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:46.075 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:46.152 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-13T07:03:46.159 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:46.159+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:46.200 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:46.201 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:46.211 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:46.260 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:46.259+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:46.317 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:46.317 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:46.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:46.317 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 140.26204703600001, 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:03:46.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:03:46.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:03:46.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:03:46.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:03:46.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:03:46.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:46.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:03:46.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:03:46.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:46.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:03:46.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:03:46.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.104165802, 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:03:46.322 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.327 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.078396091, 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.328 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:46.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:03:46.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:03:46.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:46.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:46.333 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.05724492500001, 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:46.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:03:46.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:03:46.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:46.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:03:46.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:03:46.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:46.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:03:46.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:03:46.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:03:46.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.339 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.339 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:03:46.339 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:46.339 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.940200161999996, 2024-09-13T07:03:46.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:03:46.339 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.339 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:03:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:46.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:46.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:46.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:03:46.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:03:46.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:03:46.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:03:46.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:46.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:03:46.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:03:46.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:46.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:03:46.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:03:46.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:46.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:03:46.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:03:46.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:46.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:03:46.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:03:46.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:46.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:03:46.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:03:46.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:03:46.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:03:46.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.345 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.345 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:46.345 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:46.346 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.655202916999997, 2024-09-13T07:03:46.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:03:46.346 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.346 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:46.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.346 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:03:46.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.346 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:03:46.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:03:46.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:03:46.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:03:46.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:03:46.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:03:46.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:03:46.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:03:46.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:46.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:03:46.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:03:46.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.403493779000001, 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:46.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.351 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:03:46.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.351 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:46.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:46.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:03:46.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:46.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:03:46.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:03:46.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:03:46.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:46.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:03:46.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:03:46.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:03:46.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:03:46.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.355 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.355 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:46.355 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:46.355 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.403475744999994, 2024-09-13T07:03:46.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:03:46.355 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:03:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.403459665, 2024-09-13T07:03:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:03:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:03:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:46.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:46.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:03:46.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:03:46.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:03:46.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.364 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.364 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:46.364 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:46.364 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.403443988999996, 2024-09-13T07:03:46.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:03:46.364 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.364 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:46.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:03:46.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:46.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:03:46.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:03:46.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:03:46.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:03:46.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:03:46.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.368 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.368 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:46.368 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:46.368 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.403421066999996, 2024-09-13T07:03:46.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:03:46.369 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.369 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:46.369 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.369 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.369 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.369 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:03:46.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.369 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:46.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:03:46.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:03:46.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:03:46.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:03:46.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:03:46.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:46.372 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.403404969999997, 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:46.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:46.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:03:46.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:46.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:03:46.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:03:46.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:03:46.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:03:46.376 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.376 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.376 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.402803358, 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:46.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:46.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:46.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:46.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:03:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:03:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:03:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:03:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:03:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:03:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:03:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:03:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:03:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:03:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.380 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.402760581999999, 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:03:46.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.382 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:46.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:46.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:03:46.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:03:46.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:03:46.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:03:46.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:03:46.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.245742812000003, 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.385 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:03:46.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:03:46.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:03:46.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:03:46.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:03:46.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:03:46.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:03:46.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.389 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.389 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:03:46.389 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:46.389 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.393365881999998, 2024-09-13T07:03:46.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:03:46.389 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.389 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:46.389 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:46.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:03:46.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:03:46.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:03:46.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:03:46.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:03:46.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:03:46.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:164179 3.14 e135/67 3:2fc485bd:::rbd_data.11942e4f5fc7.0000000000000002:head v 135'401, mlcod=132'398)", 2024-09-13T07:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.370425428000001, 2024-09-13T07:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063492174999999998, 2024-09-13T07:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945792+0000", 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:46.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954811+0000", 2024-09-13T07:03:46.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090187940000000001 2024-09-13T07:03:46.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954840+0000", 2024-09-13T07:03:46.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8659000000000001e-05 2024-09-13T07:03:46.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954852+0000", 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1966e-05 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954938+0000", 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6539000000000006e-05 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954975+0000", 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6303999999999997e-05 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009206+0000", 2024-09-13T07:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.054231044999999999 2024-09-13T07:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009286+0000", 2024-09-13T07:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700999999999999e-05 2024-09-13T07:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:165953 3.d 3:b3a20b45:::rbd_data.11942e4f5fc7.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8796117~39260] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:46.399 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:46.399 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.09397787, 2024-09-13T07:03:46.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.095290105, 2024-09-13T07:03:46.399 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.399 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:46.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.399 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 165953 2024-09-13T07:03:46.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.399 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222240+0000", 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258929+0000", 2024-09-13T07:03:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.036688401000000002 2024-09-13T07:03:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258971+0000", 2024-09-13T07:03:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2212000000000002e-05 2024-09-13T07:03:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258989+0000", 2024-09-13T07:03:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7674000000000001e-05 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259072+0000", 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3656e-05 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259297+0000", 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022450700000000001 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:46.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.260463+0000", 2024-09-13T07:03:46.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011665340000000001 2024-09-13T07:03:46.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:46.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.316214+0000", 2024-09-13T07:03:46.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.055750268999999998 2024-09-13T07:03:46.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:46.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317335+0000", 2024-09-13T07:03:46.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011211719999999999 2024-09-13T07:03:46.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:46.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317451+0000", 2024-09-13T07:03:46.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011616 2024-09-13T07:03:46.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317532+0000", 2024-09-13T07:03:46.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0860999999999997e-05 2024-09-13T07:03:46.405 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.405 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.405 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.405 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172852 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [call lock.unlock in=52b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-13T07:03:46.405 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:03:46.405 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.1175207230000002, 2024-09-13T07:03:46.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.90390594199999996, 2024-09-13T07:03:46.405 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.406 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:46.406 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.406 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.406 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.406 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172852 2024-09-13T07:03:46.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.406 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:03:46.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:03:46.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198696+0000", 2024-09-13T07:03:46.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:46.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198710+0000", 2024-09-13T07:03:46.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3548e-05 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198739+0000", 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9606000000000001e-05 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198752+0000", 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3033e-05 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198869+0000", 2024-09-13T07:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116552 2024-09-13T07:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199018+0000", 2024-09-13T07:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014923700000000001 2024-09-13T07:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199308+0000", 2024-09-13T07:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028947199999999998 2024-09-13T07:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.200731+0000", 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014229799999999999 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059443+0000", 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.85871189999999997 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059467+0000", 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.478e-05 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102380+0000", 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042912463999999997 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102605+0000", 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022489499999999999 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172853 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [watch reconnect cookie 140335574242864 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-13T07:03:46.411 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.265771864, 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.056267725999999997, 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172853 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:03:46.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050445+0000", 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:46.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050461+0000", 2024-09-13T07:03:46.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5783000000000001e-05 2024-09-13T07:03:46.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:46.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050497+0000", 2024-09-13T07:03:46.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5995999999999999e-05 2024-09-13T07:03:46.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:46.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050516+0000", 2024-09-13T07:03:46.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9579999999999999e-05 2024-09-13T07:03:46.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050637+0000", 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120536 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050647+0000", 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6989999999999998e-06 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059533+0000", 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088858749999999997 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:46.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059545+0000", 2024-09-13T07:03:46.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2473000000000001e-05 2024-09-13T07:03:46.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:46.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102962+0000", 2024-09-13T07:03:46.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043416656999999997 2024-09-13T07:03:46.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:46.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.103612+0000", 2024-09-13T07:03:46.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000650374 2024-09-13T07:03:46.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:46.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:46.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:46.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.106715+0000", 2024-09-13T07:03:46.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031033229999999998 2024-09-13T07:03:46.417 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.417 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.417 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.417 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:46.417 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:46.417 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:46.419 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:46.419 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:46.417+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:46.520 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:46.519+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:46.538 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:46.538 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:46.548 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:46.620 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:46.619+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:46.650 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:46.650 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:46.660 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:46.721 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:46.723+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:46.811 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:46.811 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:46.821 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:46.822 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:46.821+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:46.924 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:46.923+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:46.928 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:46.928 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:46.938 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:47.025 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:47.021+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:47.046 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:47.046 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:47.056 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:47.119 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:03:47.127 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:47.125+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:47.168 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:47.169 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:47.179 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:47.226 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:47.227+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:47.290 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:47.290 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:47.300 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:47.327 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:47.327+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:47.400 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:47.400 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:47.410 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:47.453 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:47.455+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:47.530 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:47.525+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:47.562 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:47.562 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:47.572 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:47.630 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:47.631+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:47.677 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:47.677 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:47.687 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:47.732 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:47.729+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:47.788 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:47.788 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:47.798 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:47.832 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:47.829+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:47.904 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:47.904 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:47.914 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:47.933 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:47.935+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:48.034 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:48.033+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:48.035 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:48.035 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:48.045 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:48.136 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:48.133+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:48.136 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:03:48.163 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:48.163 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:48.173 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:48.236 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:48.233+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:48.334 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:48.334 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:48.337 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:48.335+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:48.344 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:48.438 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:48.439+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:48.446 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:48.446 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:48.447 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:48.447 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:48.456 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:48.492 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:48.513 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:48.513 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:48.513 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:48.513 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:48.514 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:48.514 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:48.514 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:03:48.514 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:48.514 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:48.519 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:03:48.519 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:03:48.539 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:48.537+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:48.561 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:48.561 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:48.571 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:48.640 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:48.637+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:48.721 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:48.721 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:48.730 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:48.740 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:48.739+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:48.836 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:48.836 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:48.841 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:48.843+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:48.845 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:48.942 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:48.937+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:48.949 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:48.949 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:48.958 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:49.043 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:49.041+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:49.062 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:49.062 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:49.072 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:49.144 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:49.141+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:49.193 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:49.193 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:49.204 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:49.244 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:49.243+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:49.278 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T07:03:49.278 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.5 2024-09-13T07:03:49.278 INFO:tasks.thrashosds.thrasher:remote for osd 5 is ubuntu@smithi164.front.sepia.ceph.com 2024-09-13T07:03:49.278 DEBUG:teuthology.orchestra.run.smithi164:> type ceph-objectstore-tool 2024-09-13T07:03:49.284 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T07:03:49.284 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-09-13T07:03:49.316 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:49.316 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:49.326 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:49.345 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:49.347+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:49.430 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:49.430 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:49.439 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:49.446 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:49.441+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:49.547 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:49.545+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:49.584 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:49.584 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:49.594 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:49.648 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:49.647+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:49.700 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:49.700 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:49.710 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:49.749 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:49.747+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:49.818 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:49.819 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:49.828 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:49.850 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:49.845+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:49.951 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:49.951+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:49.981 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:49.982 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:49.991 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:50.052 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:50.051+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:50.102 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:50.102 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:50.112 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:50.150 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:03:50.152 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:50.151+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:50.250 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:50.250 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:50.253 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:50.255+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:50.261 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:50.354 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:50.353+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:50.436 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:50.437 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:50.446 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:50.455 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:50.455+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:50.520 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:03:50.547 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:50.547 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:50.556 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:50.553+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:50.557 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:50.651 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:50.656 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:50.653+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:50.671 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:50.689 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:50.708 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:50.708 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:50.716 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:50.717 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:50.736 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:50.755 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:50.776 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:50.776 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:50.776 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:50.776 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:50.776 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:50.776 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:50.776 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:03:50.776 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:50.776 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:50.780 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:03:50.781 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:03:50.831 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:50.832 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:50.841 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:50.851 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:50.761+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:50.857 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:50.859+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:50.944 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:50.944 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:50.944 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:50.944 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:50.944 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:50.953 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:50.958 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:50.959+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:51.059 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:51.059+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:51.064 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461496306000001, 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:03:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:03:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:03:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:03:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:03:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:03:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:03:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:03:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:03:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461465204999996, 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:03:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:03:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:03:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:03:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:03:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:03:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461442891000004, 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:03:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:03:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:03:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:03:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:03:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:03:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:03:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:03:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:03:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:03:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:03:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461419860000007, 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:03:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:03:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:03:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:03:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:03:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:03:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:03:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:03:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:03:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:03:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.46139531, 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:03:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:03:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:03:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:03:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:03:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:03:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:03:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:03:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:03:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:03:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:03:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:03:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:03:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:03:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:03:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461359600999998, 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:03:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:03:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:03:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:03:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:03:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:03:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:03:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:03:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461334922000006, 2024-09-13T07:03:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:03:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:03:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:03:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461301648000003, 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:03:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:03:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461274860000003, 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:03:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461250305999997, 2024-09-13T07:03:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:03:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:03:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:03:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:03:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:03:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:03:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:03:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:03:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:03:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:03:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:03:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:03:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461220537000003, 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:03:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:03:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:03:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:03:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:03:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:03:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:03:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:03:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461160694, 2024-09-13T07:03:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:03:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:03:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:03:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:03:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:03:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:03:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461093844000004, 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:03:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:03:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:03:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461061341999994, 2024-09-13T07:03:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:03:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:03:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:03:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:03:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:03:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:03:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:03:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:03:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:03:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:03:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:03:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:03:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:03:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.461026720999996, 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:03:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:03:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:03:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:03:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:03:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:03:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:03:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:03:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:03:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:03:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:03:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.460976489999993, 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:03:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:03:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:03:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:03:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:03:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:03:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:03:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:03:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:03:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:03:51.142 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.460928304000007, 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.143 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:03:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:03:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:03:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:03:51.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:03:51.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.460901665999998, 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:03:51.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:03:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:03:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:03:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:03:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:03:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:03:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:03:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:03:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.152 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.460854831000006, 2024-09-13T07:03:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:03:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:03:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:03:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:03:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:03:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:03:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:03:51.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.457500413999995, 2024-09-13T07:03:51.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:03:51.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:03:51.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:03:51.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:03:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.162 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.162 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:51.163 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:03:51.164 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:51.165 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:51.167+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:51.264 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:51.263+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:51.268 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:51.269 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:51.280 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:51.365 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:51.367+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:51.383 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:51.383 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:51.383 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:51.383 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:51.383 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:51.394 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:51.466 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:51.461+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:51.499 INFO:teuthology.orchestra.run.smithi164.stdout:3.d 2024-09-13T07:03:51.499 INFO:teuthology.orchestra.run.smithi164.stdout:3.5 2024-09-13T07:03:51.499 INFO:teuthology.orchestra.run.smithi164.stdout:3.4 2024-09-13T07:03:51.499 INFO:teuthology.orchestra.run.smithi164.stdout:3.c 2024-09-13T07:03:51.499 INFO:teuthology.orchestra.run.smithi164.stdout:3.2 2024-09-13T07:03:51.499 INFO:teuthology.orchestra.run.smithi164.stdout:3.1f 2024-09-13T07:03:51.500 INFO:teuthology.orchestra.run.smithi164.stdout:3.1d 2024-09-13T07:03:51.500 INFO:teuthology.orchestra.run.smithi164.stdout:3.1c 2024-09-13T07:03:51.500 INFO:teuthology.orchestra.run.smithi164.stdout:3.17 2024-09-13T07:03:51.500 INFO:teuthology.orchestra.run.smithi164.stdout:3.1 2024-09-13T07:03:51.500 INFO:teuthology.orchestra.run.smithi164.stdout:3.16 2024-09-13T07:03:51.500 INFO:teuthology.orchestra.run.smithi164.stdout:2.3 2024-09-13T07:03:51.500 INFO:teuthology.orchestra.run.smithi164.stdout:2.2 2024-09-13T07:03:51.504 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:51.504 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:51.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:51.504 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 145.44524262600001, 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:03:51.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:03:51.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:03:51.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:03:51.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:03:51.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:03:51.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:03:51.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:51.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:03:51.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:03:51.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:03:51.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:03:51.510 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.510 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.511 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.512 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.512 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:51.512 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.887458748, 2024-09-13T07:03:51.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:03:51.512 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.512 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.512 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:03:51.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:03:51.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:03:51.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:03:51.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:03:51.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.516 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.516 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.516 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.887435780999994, 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:51.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:03:51.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:03:51.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:03:51.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:03:51.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:03:51.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:03:51.520 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.887418603, 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.521 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.887404738000001, 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.525 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:03:51.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:51.529 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.887391593000004, 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:03:51.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:03:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:03:51.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:03:51.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:03:51.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:03:51.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:03:51.533 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.533 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.533 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.887378385999995, 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:03:51.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:03:51.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:03:51.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:03:51.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:03:51.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:03:51.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:03:51.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:03:51.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:03:51.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:03:51.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:03:51.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:03:51.538 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.538 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.538 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.538 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.887364319, 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:51.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:51.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:03:51.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:03:51.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.887344304999999, 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.543 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:03:51.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.887324555000006, 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:03:51.547 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:03:51.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.552 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.552 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:51.552 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.887299748999993, 2024-09-13T07:03:51.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:03:51.552 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.552 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.552 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:03:51.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.552 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:03:51.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:03:51.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:03:51.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:03:51.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:03:51.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:03:51.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:03:51.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.556 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.556 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.556 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:51.556 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.887275484, 2024-09-13T07:03:51.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:03:51.556 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.556 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.556 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.556 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:03:51.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:03:51.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:03:51.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:03:51.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:03:51.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.887233750999997, 2024-09-13T07:03:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:03:51.561 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.561 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.561 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:03:51.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.561 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:51.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:03:51.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:03:51.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:03:51.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:03:51.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:03:51.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:03:51.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:03:51.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.887197462000003, 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.565 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.444094129999996, 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.569 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.573 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.444077247999999, 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:03:51.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:03:51.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:03:51.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:03:51.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:03:51.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:03:51.577 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.443026691, 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:03:51.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.441352142, 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.582 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:51.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:51.586 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.920769751999998, 2024-09-13T07:03:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:03:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:03:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:51.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:03:51.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:03:51.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:03:51.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:03:51.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:03:51.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:03:51.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:03:51.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172880 2.2 2:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e149)", 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.3664735060000002, 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.97218948500000002, 2024-09-13T07:03:51.591 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172880 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137687+0000", 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137702+0000", 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5023e-05 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137730+0000", 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7719e-05 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137748+0000", 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8189000000000002e-05 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137891+0000", 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014330699999999999 2024-09-13T07:03:51.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137903+0000", 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1688999999999999e-05 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142887+0000", 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004984006 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142898+0000", 2024-09-13T07:03:51.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.079e-05 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109703+0000", 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96680553899999999 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109879+0000", 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017579800000000001 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.596 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.597 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:51.598 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:51.598 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:51.599+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:51.699 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:51.699+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:51.710 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:51.710 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:51.710 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:51.710 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:51.720 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:51.799 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:51.799+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:51.828 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:51.828 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:51.828 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:51.828 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:51.828 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:51.838 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:51.901 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:51.903+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:51.947 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:51.947 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:51.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:51.947 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:51.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.948 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:03:51.948 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:51.948 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 145.89152032499999, 2024-09-13T07:03:51.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:03:51.948 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.948 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:51.948 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:51.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:03:51.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:03:51.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:51.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:03:51.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:03:51.952 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.952 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.952 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.952 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:51.952 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:51.952 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 113.733639091, 2024-09-13T07:03:51.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:03:51.952 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.952 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:51.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.953 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:03:51.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.953 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:51.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:03:51.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:03:51.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:51.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:03:51.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:03:51.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:03:51.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:03:51.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:03:51.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:03:51.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:03:51.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:03:51.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:03:51.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:03:51.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:51.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:03:51.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:03:51.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:51.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:03:51.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:03:51.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:51.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:03:51.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:03:51.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:51.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:03:51.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:03:51.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:03:51.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:03:51.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.961 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.962 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.962 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:51.962 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:51.963 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 113.70786938000001, 2024-09-13T07:03:51.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:03:51.963 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.963 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:51.963 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.963 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.964 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.964 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:03:51.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.964 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:51.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:03:51.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:03:51.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:51.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:03:51.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:03:51.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:03:51.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:03:51.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:03:51.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:03:51.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:03:51.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:03:51.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:03:51.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:03:51.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:51.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:03:51.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:03:51.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:51.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:03:51.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:03:51.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:51.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:03:51.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:03:51.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:51.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:03:51.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:03:51.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:03:51.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:03:51.976 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.976 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.976 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.977 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:51.977 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:51.977 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 113.686718214, 2024-09-13T07:03:51.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:03:51.978 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.978 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:51.978 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.978 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.979 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:03:51.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.979 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:51.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:03:51.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:03:51.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:51.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:03:51.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:03:51.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:03:51.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:03:51.984 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.984 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.984 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.985 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:03:51.985 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:51.985 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.569673451, 2024-09-13T07:03:51.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:03:51.985 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.985 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:51.986 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.986 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.986 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.986 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:03:51.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.986 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:51.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:51.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:03:51.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:03:51.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:03:51.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:03:51.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:03:51.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:03:51.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:03:51.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:03:51.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:51.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:03:51.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:03:51.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:51.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:03:51.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:03:51.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:51.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:03:51.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:03:51.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:03:51.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:03:51.992 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.992 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.992 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.993 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:03:51.993 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:51.993 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.284676206, 2024-09-13T07:03:51.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:03:51.993 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.993 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.993 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:03:51.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.993 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:03:51.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:51.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:03:51.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:03:51.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.997 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:51.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:51.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.997 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:51.997 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:51.997 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.032967068000005, 2024-09-13T07:03:51.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:03:51.997 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:51.997 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:51.997 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:03:51.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:51.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:03:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:03:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:03:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:03:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:03:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:03:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:03:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:03:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:03:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:03:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.032949033999998, 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:52.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:03:52.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:03:52.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:52.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:03:52.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:03:52.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:03:52.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.032932954000003, 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:52.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:52.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:03:52.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:52.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:03:52.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:03:52.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:52.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:03:52.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:03:52.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:03:52.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:03:52.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:03:52.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:03:52.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:52.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:03:52.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:03:52.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:03:52.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:03:52.010 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.010 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.010 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.011 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:52.011 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:52.011 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.032917277999999, 2024-09-13T07:03:52.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:03:52.011 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:52.011 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:52.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:52.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.011 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:03:52.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.012 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:52.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:52.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:52.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:52.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:03:52.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:03:52.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.032894356, 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:52.015 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:52.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:03:52.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:52.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:03:52.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:03:52.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:52.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.032878259, 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:52.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:03:52.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:03:52.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:03:52.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:03:52.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:03:52.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:03:52.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.032276647000003, 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.023 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:52.024 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:52.024 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.024 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:03:52.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.024 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:52.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:52.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:52.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:52.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:52.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:52.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:03:52.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:52.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:03:52.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:03:52.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:52.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:03:52.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:03:52.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:03:52.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:03:52.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:03:52.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:03:52.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.032233871000003, 2024-09-13T07:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:03:52.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:03:52.032 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.032 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.032 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.032 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:03:52.032 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:52.032 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.875216101000007, 2024-09-13T07:03:52.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:03:52.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:03:52.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:03:52.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:03:52.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:03:52.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.036 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.036 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:03:52.036 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:52.036 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.022839171000001, 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:52.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:03:52.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:52.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:03:52.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:52.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:03:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:03:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:03:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:03:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:03:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:03:52.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:03:52.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:03:52.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:52.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:03:52.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:03:52.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:03:52.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:03:52.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:03:52.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:03:52.041 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.042 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.042 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.042 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:164179 3.14 e135/67 3:2fc485bd:::rbd_data.11942e4f5fc7.0000000000000002:head v 135'401, mlcod=132'398)", 2024-09-13T07:03:52.042 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:52.042 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.999898717000001, 2024-09-13T07:03:52.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063492174999999998, 2024-09-13T07:03:52.042 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:52.042 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:52.042 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945792+0000", 2024-09-13T07:03:52.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:52.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:52.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954811+0000", 2024-09-13T07:03:52.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090187940000000001 2024-09-13T07:03:52.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:52.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954840+0000", 2024-09-13T07:03:52.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8659000000000001e-05 2024-09-13T07:03:52.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954852+0000", 2024-09-13T07:03:52.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1966e-05 2024-09-13T07:03:52.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954938+0000", 2024-09-13T07:03:52.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6539000000000006e-05 2024-09-13T07:03:52.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:52.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954975+0000", 2024-09-13T07:03:52.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6303999999999997e-05 2024-09-13T07:03:52.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:52.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009206+0000", 2024-09-13T07:03:52.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.054231044999999999 2024-09-13T07:03:52.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009286+0000", 2024-09-13T07:03:52.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700999999999999e-05 2024-09-13T07:03:52.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.047 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.047 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:165953 3.d 3:b3a20b45:::rbd_data.11942e4f5fc7.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8796117~39260] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.047 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:52.047 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.723451159, 2024-09-13T07:03:52.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.095290105, 2024-09-13T07:03:52.048 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:52.048 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:52.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:52.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:52.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.048 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 165953 2024-09-13T07:03:52.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.048 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:52.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:52.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:52.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222240+0000", 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:52.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258929+0000", 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.036688401000000002 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258971+0000", 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2212000000000002e-05 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258989+0000", 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7674000000000001e-05 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259072+0000", 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3656e-05 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259297+0000", 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022450700000000001 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.260463+0000", 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011665340000000001 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:52.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.316214+0000", 2024-09-13T07:03:52.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.055750268999999998 2024-09-13T07:03:52.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:52.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317335+0000", 2024-09-13T07:03:52.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011211719999999999 2024-09-13T07:03:52.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:03:52.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317451+0000", 2024-09-13T07:03:52.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011616 2024-09-13T07:03:52.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317532+0000", 2024-09-13T07:03:52.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0860999999999997e-05 2024-09-13T07:03:52.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.053 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.053 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172852 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [call lock.unlock in=52b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-13T07:03:52.053 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:03:52.053 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.746994012, 2024-09-13T07:03:52.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.90390594199999996, 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172852 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198696+0000", 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198710+0000", 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3548e-05 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:52.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198739+0000", 2024-09-13T07:03:52.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9606000000000001e-05 2024-09-13T07:03:52.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198752+0000", 2024-09-13T07:03:52.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3033e-05 2024-09-13T07:03:52.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198869+0000", 2024-09-13T07:03:52.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116552 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199018+0000", 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014923700000000001 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199308+0000", 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028947199999999998 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:03:52.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.200731+0000", 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014229799999999999 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059443+0000", 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.85871189999999997 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059467+0000", 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.478e-05 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102380+0000", 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042912463999999997 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102605+0000", 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022489499999999999 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172853 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [watch reconnect cookie 140335574242864 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:03:52.059 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8952451530000003, 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.056267725999999997, 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172853 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050445+0000", 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050461+0000", 2024-09-13T07:03:52.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5783000000000001e-05 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050497+0000", 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5995999999999999e-05 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050516+0000", 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9579999999999999e-05 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050637+0000", 2024-09-13T07:03:52.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120536 2024-09-13T07:03:52.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:03:52.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050647+0000", 2024-09-13T07:03:52.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6989999999999998e-06 2024-09-13T07:03:52.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059533+0000", 2024-09-13T07:03:52.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088858749999999997 2024-09-13T07:03:52.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:52.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059545+0000", 2024-09-13T07:03:52.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2473000000000001e-05 2024-09-13T07:03:52.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:52.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102962+0000", 2024-09-13T07:03:52.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043416656999999997 2024-09-13T07:03:52.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:03:52.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.103612+0000", 2024-09-13T07:03:52.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000650374 2024-09-13T07:03:52.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:52.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:52.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:52.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.106715+0000", 2024-09-13T07:03:52.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031033229999999998 2024-09-13T07:03:52.066 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.066 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:52.067 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:52.067 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:52.068 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:52.065+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:52.068 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:52.084 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op export --pgid 2.3 --file /var/log/ceph/exp.2.3.5 2024-09-13T07:03:52.169 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:52.167+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:52.193 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:52.193 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:52.193 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:52.193 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:52.205 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:52.269 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:52.265+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:52.309 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:52.310 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:52.310 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:52.310 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:52.310 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:52.320 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:52.371 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:52.371+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:52.427 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:52.428 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:52.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:52.428 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:52.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.428 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137474 3.6 3:65c31370:::rbd_data.1194af11889b.000000000000013b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 186185~11715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-13T07:03:52.428 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:52.428 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.682638619000002, 2024-09-13T07:03:52.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0054357140000001, 2024-09-13T07:03:52.428 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.428 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:52.428 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.428 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.428 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137474 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743934+0000", 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743949+0000", 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5705999999999998e-05 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743960+0000", 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0084e-05 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743971+0000", 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0942e-05 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744045+0000", 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4626e-05 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744062+0000", 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7339000000000001e-05 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746366+0000", 2024-09-13T07:03:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.002303457 2024-09-13T07:03:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746520+0000", 2024-09-13T07:03:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015385900000000001 2024-09-13T07:03:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746734+0000", 2024-09-13T07:03:52.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000214253 2024-09-13T07:03:52.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:52.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749128+0000", 2024-09-13T07:03:52.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002394129 2024-09-13T07:03:52.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:52.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749267+0000", 2024-09-13T07:03:52.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013878299999999999 2024-09-13T07:03:52.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749328+0000", 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0918000000000002e-05 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749371+0000", 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3188000000000001e-05 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163305 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2387968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.748420138, 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044745869000000001, 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163305 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678153+0000", 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678156+0000", 2024-09-13T07:03:52.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6350000000000002e-06 2024-09-13T07:03:52.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678161+0000", 2024-09-13T07:03:52.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:03:52.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678166+0000", 2024-09-13T07:03:52.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.498e-06 2024-09-13T07:03:52.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722751+0000", 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044585134999999998 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722836+0000", 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4672999999999994e-05 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722900+0000", 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4042999999999997e-05 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.438 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.439 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.439 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163307 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.439 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:03:52.439 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.748399674000002, 2024-09-13T07:03:52.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045060944999999998, 2024-09-13T07:03:52.439 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.439 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.439 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.439 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.440 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.440 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163307 2024-09-13T07:03:52.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.440 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:03:52.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:03:52.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:03:52.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678177+0000", 2024-09-13T07:03:52.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:03:52.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678183+0000", 2024-09-13T07:03:52.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0599999999999996e-06 2024-09-13T07:03:52.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678188+0000", 2024-09-13T07:03:52.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:03:52.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.443 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.443 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722941+0000", 2024-09-13T07:03:52.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044752844999999999 2024-09-13T07:03:52.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.443 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.443 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723073+0000", 2024-09-13T07:03:52.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013128599999999999 2024-09-13T07:03:52.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.443 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723235+0000", 2024-09-13T07:03:52.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000162671 2024-09-13T07:03:52.444 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.444 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.444 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.444 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163310 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 544768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.444 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:03:52.444 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.748380722, 2024-09-13T07:03:52.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045290534, 2024-09-13T07:03:52.444 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.444 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.445 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.445 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.445 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.445 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163310 2024-09-13T07:03:52.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.445 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.445 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.445 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.445 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:03:52.445 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.445 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.445 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:03:52.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:03:52.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678195+0000", 2024-09-13T07:03:52.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.785e-06 2024-09-13T07:03:52.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.447 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.447 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.447 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678201+0000", 2024-09-13T07:03:52.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:03:52.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.447 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.447 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.447 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678205+0000", 2024-09-13T07:03:52.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6999999999999999e-06 2024-09-13T07:03:52.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723299+0000", 2024-09-13T07:03:52.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045093964 2024-09-13T07:03:52.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723384+0000", 2024-09-13T07:03:52.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4562999999999997e-05 2024-09-13T07:03:52.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723484+0000", 2024-09-13T07:03:52.449 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100045 2024-09-13T07:03:52.449 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.449 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.449 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163313 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1093632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.748364736999999, 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045562975999999998, 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163313 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.450 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:03:52.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678212+0000", 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8650000000000001e-06 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678217+0000", 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2660000000000001e-06 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678221+0000", 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-06 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723552+0000", 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045330489000000002 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723660+0000", 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010770800000000001 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.453 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723772+0000", 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112608 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163317 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1642496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.748335929, 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045770165000000002, 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.454 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163317 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.455 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678237+0000", 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678241+0000", 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2669999999999998e-06 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678247+0000", 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7309999999999996e-06 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678252+0000", 2024-09-13T07:03:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-13T07:03:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723830+0000", 2024-09-13T07:03:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045577234000000001 2024-09-13T07:03:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723936+0000", 2024-09-13T07:03:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106277 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724008+0000", 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2422999999999994e-05 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163320 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.748315505000001, 2024-09-13T07:03:52.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045968883000000002, 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163320 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.459 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678261+0000", 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8090000000000001e-06 2024-09-13T07:03:52.460 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678267+0000", 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8860000000000001e-06 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678271+0000", 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4549999999999997e-06 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724061+0000", 2024-09-13T07:03:52.461 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045789656999999997 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724151+0000", 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9734999999999994e-05 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724227+0000", 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6848999999999996e-05 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.462 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163324 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.748031172000001, 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045988788000000003, 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163324 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.463 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:03:52.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.464 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:03:52.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.464 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.464 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.464 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678542+0000", 2024-09-13T07:03:52.464 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:52.464 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678547+0000", 2024-09-13T07:03:52.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6829999999999999e-06 2024-09-13T07:03:52.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.465 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.465 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678556+0000", 2024-09-13T07:03:52.465 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1929999999999999e-06 2024-09-13T07:03:52.465 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.465 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678562+0000", 2024-09-13T07:03:52.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.083e-06 2024-09-13T07:03:52.466 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.466 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.466 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724267+0000", 2024-09-13T07:03:52.466 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045705427 2024-09-13T07:03:52.466 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.466 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.466 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.467 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724417+0000", 2024-09-13T07:03:52.467 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014984699999999999 2024-09-13T07:03:52.467 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.467 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.467 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.467 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724532+0000", 2024-09-13T07:03:52.467 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011430900000000001 2024-09-13T07:03:52.467 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.467 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.467 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.467 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.467 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.467 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163327 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 733184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.748006138000001, 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046185902000000001, 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163327 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.468 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678567+0000", 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.469 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678571+0000", 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3799999999999998e-06 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678576+0000", 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7760000000000003e-06 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678581+0000", 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8640000000000004e-06 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.470 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724561+0000", 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045979178000000002 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724649+0000", 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8677000000000005e-05 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724754+0000", 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104527 2024-09-13T07:03:52.471 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163331 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.747987984000002, 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046361773000000002, 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.472 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163331 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.473 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.474 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678585+0000", 2024-09-13T07:03:52.474 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.474 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.474 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.474 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678588+0000", 2024-09-13T07:03:52.474 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0469999999999998e-06 2024-09-13T07:03:52.474 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.474 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.474 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.475 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678595+0000", 2024-09-13T07:03:52.475 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0399999999999998e-06 2024-09-13T07:03:52.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.475 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.475 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.475 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678599+0000", 2024-09-13T07:03:52.475 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:03:52.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.475 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.475 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.475 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724780+0000", 2024-09-13T07:03:52.475 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046181168000000002 2024-09-13T07:03:52.475 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.476 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.476 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724871+0000", 2024-09-13T07:03:52.476 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0562000000000005e-05 2024-09-13T07:03:52.476 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.476 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.476 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.476 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724948+0000", 2024-09-13T07:03:52.476 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6985999999999998e-05 2024-09-13T07:03:52.476 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.476 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.476 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.476 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.477 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.477 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163334 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1662976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.477 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:03:52.477 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.747970780999999, 2024-09-13T07:03:52.477 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046544502000000001, 2024-09-13T07:03:52.477 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.477 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.477 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.477 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.477 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.477 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163334 2024-09-13T07:03:52.477 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.477 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:03:52.478 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678605+0000", 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5380000000000001e-06 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678611+0000", 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7389999999999996e-06 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.479 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678615+0000", 2024-09-13T07:03:52.480 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-13T07:03:52.480 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.480 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.480 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.480 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724987+0000", 2024-09-13T07:03:52.480 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046371969999999998 2024-09-13T07:03:52.480 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.480 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.480 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.480 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725075+0000", 2024-09-13T07:03:52.480 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8091999999999996e-05 2024-09-13T07:03:52.480 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.480 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.481 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.481 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725148+0000", 2024-09-13T07:03:52.481 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2206999999999995e-05 2024-09-13T07:03:52.481 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.481 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.481 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.481 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.481 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.481 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163338 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.481 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:03:52.481 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.747954051000001, 2024-09-13T07:03:52.481 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046738452999999999, 2024-09-13T07:03:52.481 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.482 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.482 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.482 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.482 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.482 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163338 2024-09-13T07:03:52.482 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.482 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.482 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.482 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.482 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678619+0000", 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.483 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678622+0000", 2024-09-13T07:03:52.484 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.649e-06 2024-09-13T07:03:52.484 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.484 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.484 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.484 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678628+0000", 2024-09-13T07:03:52.484 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5029999999999997e-06 2024-09-13T07:03:52.484 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.484 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.484 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.484 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678632+0000", 2024-09-13T07:03:52.484 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4020000000000004e-06 2024-09-13T07:03:52.484 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.484 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725182+0000", 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046550389999999997 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725280+0000", 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7260000000000001e-05 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725358+0000", 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8720000000000005e-05 2024-09-13T07:03:52.485 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.486 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.486 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.486 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.486 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.486 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163339 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2363392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.486 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:03:52.486 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.747938522999998, 2024-09-13T07:03:52.486 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046920835000000001, 2024-09-13T07:03:52.486 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.486 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.486 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.486 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.487 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.487 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163339 2024-09-13T07:03:52.487 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.487 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.487 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.487 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.487 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:03:52.487 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.487 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.487 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.487 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.487 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:03:52.487 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.488 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.488 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.488 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.488 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:03:52.488 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.488 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.488 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.488 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.488 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678637+0000", 2024-09-13T07:03:52.488 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2479999999999999e-06 2024-09-13T07:03:52.488 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.488 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.488 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678642+0000", 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0030000000000002e-06 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678646+0000", 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9870000000000001e-06 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725397+0000", 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046750127000000002 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.489 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.490 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.490 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725482+0000", 2024-09-13T07:03:52.490 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5928999999999997e-05 2024-09-13T07:03:52.490 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.490 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.490 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.491 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725556+0000", 2024-09-13T07:03:52.491 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3823000000000001e-05 2024-09-13T07:03:52.491 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.491 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.491 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.491 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.491 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.491 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163342 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.491 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:03:52.491 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.747923883999999, 2024-09-13T07:03:52.491 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047212991000000003, 2024-09-13T07:03:52.491 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.491 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.492 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.492 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.492 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.492 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163342 2024-09-13T07:03:52.492 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.492 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.492 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.492 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.492 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:03:52.492 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.492 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.492 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.492 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678652+0000", 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.486e-06 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.493 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678657+0000", 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6909999999999996e-06 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678660+0000", 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8009999999999999e-06 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725597+0000", 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046937372999999998 2024-09-13T07:03:52.494 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.495 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.495 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.495 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725745+0000", 2024-09-13T07:03:52.495 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000148314 2024-09-13T07:03:52.495 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.495 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.495 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.495 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725863+0000", 2024-09-13T07:03:52.495 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011768 2024-09-13T07:03:52.495 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.495 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.496 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.496 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.496 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.496 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163345 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.496 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:03:52.496 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.747062321000001, 2024-09-13T07:03:52.496 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046569193000000002, 2024-09-13T07:03:52.496 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.496 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163345 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678662+0000", 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9991498 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679516+0000", 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000854789 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.498 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679530+0000", 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3492999999999999e-05 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679537+0000", 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.323e-06 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725892+0000", 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046354244000000003 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.499 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.500 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725975+0000", 2024-09-13T07:03:52.500 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3356000000000006e-05 2024-09-13T07:03:52.500 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.500 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.500 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.500 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726081+0000", 2024-09-13T07:03:52.500 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105992 2024-09-13T07:03:52.500 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.500 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.500 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.500 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.500 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.500 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163349 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1298432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.501 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:03:52.501 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.747030257999999, 2024-09-13T07:03:52.501 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046744247000000003, 2024-09-13T07:03:52.501 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.501 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.501 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.501 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.501 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.501 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163349 2024-09-13T07:03:52.501 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.501 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.501 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.501 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679543+0000", 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.502 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.503 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679546+0000", 2024-09-13T07:03:52.503 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:03:52.503 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.503 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.503 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.503 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679551+0000", 2024-09-13T07:03:52.503 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:03:52.503 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.503 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.503 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.503 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679554+0000", 2024-09-13T07:03:52.503 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8629999999999999e-06 2024-09-13T07:03:52.503 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726117+0000", 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046563081999999999 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726213+0000", 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6207999999999998e-05 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726288+0000", 2024-09-13T07:03:52.504 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4741999999999997e-05 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163350 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1449984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.747017631999999, 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046935635000000003, 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.505 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163350 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.506 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679558+0000", 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.928e-06 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679561+0000", 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7659999999999999e-06 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.507 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.508 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.508 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679564+0000", 2024-09-13T07:03:52.508 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4960000000000001e-06 2024-09-13T07:03:52.508 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.508 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.508 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.508 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726327+0000", 2024-09-13T07:03:52.508 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046762321000000003 2024-09-13T07:03:52.508 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.508 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.508 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.508 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726415+0000", 2024-09-13T07:03:52.508 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8487999999999997e-05 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726492+0000", 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6995000000000004e-05 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163353 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.747007523000001, 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047130755000000003, 2024-09-13T07:03:52.509 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163353 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.510 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679568+0000", 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5969999999999999e-06 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.511 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.512 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.512 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679571+0000", 2024-09-13T07:03:52.512 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:03:52.512 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.512 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.512 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.512 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679573+0000", 2024-09-13T07:03:52.512 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:03:52.512 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.512 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.512 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.513 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726523+0000", 2024-09-13T07:03:52.513 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046949882999999998 2024-09-13T07:03:52.513 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.513 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.513 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.513 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726616+0000", 2024-09-13T07:03:52.513 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3236e-05 2024-09-13T07:03:52.513 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.513 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.513 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.513 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726697+0000", 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0834000000000006e-05 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163356 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2379776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.746997840999999, 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047304662999999997, 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.514 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163356 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.515 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.516 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:03:52.516 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.516 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.516 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.516 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.516 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679578+0000", 2024-09-13T07:03:52.516 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:03:52.516 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.516 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.516 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.516 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679582+0000", 2024-09-13T07:03:52.516 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679589+0000", 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1149999999999996e-06 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726723+0000", 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047133768999999999 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.517 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726809+0000", 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5402000000000007e-05 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726881+0000", 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2152999999999998e-05 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:166663 3.b 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8129196~12979] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:03:52.518 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.028623326999998, 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.061564718999999997, 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 166663 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.519 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.520 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.520 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.520 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:03:52.520 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.520 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.520 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.520 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.520 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397947+0000", 2024-09-13T07:03:52.520 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999967 2024-09-13T07:03:52.520 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.520 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.520 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.520 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412144+0000", 2024-09-13T07:03:52.521 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014196972 2024-09-13T07:03:52.521 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.521 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.521 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.521 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412172+0000", 2024-09-13T07:03:52.521 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7781999999999999e-05 2024-09-13T07:03:52.521 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.521 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.521 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.521 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412182+0000", 2024-09-13T07:03:52.521 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6069999999999997e-06 2024-09-13T07:03:52.521 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.521 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412239+0000", 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.732e-05 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412365+0000", 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125356 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.413178+0000", 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081355299999999995 2024-09-13T07:03:52.522 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.443176+0000", 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029997903999999999 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459446+0000", 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016269921999999999 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459484+0000", 2024-09-13T07:03:52.523 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8488000000000001e-05 2024-09-13T07:03:52.524 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.524 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.524 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.524 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459515+0000", 2024-09-13T07:03:52.524 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0981e-05 2024-09-13T07:03:52.524 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.524 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.524 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.524 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.524 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.524 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:52.525 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-13T07:03:52.526 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:52.526 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:52.521+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:52.627 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:52.627+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:52.642 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:52.643 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:52.643 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:52.643 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:52.653 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:52.728 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:52.725+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:52.781 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:03:52.822 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:52.822 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:52.822 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:52.822 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:52.822 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:52.828 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:52.829+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:52.832 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:52.919 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:52.929 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:52.931+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:52.933 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:52.934 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:52.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:52.934 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:52.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.934 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:03:52.934 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:52.934 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 129.43980315100001, 2024-09-13T07:03:52.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:03:52.934 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.934 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.934 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.934 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.934 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:03:52.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:03:52.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:03:52.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:03:52.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:03:52.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:52.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:03:52.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:03:52.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:03:52.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:03:52.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:03:52.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:03:52.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:03:52.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:03:52.939 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.939 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.939 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.939 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.939 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:52.939 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.953319587, 2024-09-13T07:03:52.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:52.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:52.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:03:52.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:03:52.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:03:52.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:03:52.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:03:52.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:03:52.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:03:52.944 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.944 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.945 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.945 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.945 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:52.945 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.953249658, 2024-09-13T07:03:52.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:03:52.945 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.945 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.945 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.945 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.945 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.945 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:03:52.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.946 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:52.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:52.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:03:52.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:03:52.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:52.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:03:52.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:03:52.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:52.949 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.953233786, 2024-09-13T07:03:52.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:03:52.950 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.950 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.950 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.950 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.950 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.950 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:03:52.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.950 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:52.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:52.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:03:52.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:03:52.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:03:52.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:03:52.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:03:52.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:03:52.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:03:52.954 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.954 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.954 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.954 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.954 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:52.954 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.95316332900001, 2024-09-13T07:03:52.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:03:52.954 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.954 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:52.954 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.954 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:03:52.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:03:52.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:03:52.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:03:52.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:03:52.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.957 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:52.957 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.949811291, 2024-09-13T07:03:52.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:03:52.959 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.959 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.959 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.959 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.959 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.959 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:03:52.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.959 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:52.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:03:52.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:03:52.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:03:52.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:03:52.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:03:52.961 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:03:52.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.962 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:52.963 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.94979499, 2024-09-13T07:03:52.963 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:03:52.963 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.963 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.963 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.963 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.963 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.963 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:03:52.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.963 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.963 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.963 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.963 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.964 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:03:52.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:03:52.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:03:52.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:52.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.965 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.965 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:03:52.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:03:52.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.94977520499999, 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:52.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:52.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:52.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:03:52.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:03:52.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:03:52.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.970 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.94975893199999, 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.971 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.972 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.972 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:03:52.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.972 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:52.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:52.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.94974110699999, 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.975 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.976 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.976 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.976 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:03:52.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.976 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:52.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.976 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.976 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:52.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:52.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.977 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.977 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:03:52.977 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:03:52.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.977 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:03:52.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.94972501399999, 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:03:52.979 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.980 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.980 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.980 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.980 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.980 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:03:52.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.980 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:52.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:52.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:03:52.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:03:52.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:03:52.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:03:52.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.983 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.983 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:03:52.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:03:52.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.983 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.983 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.983 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.983 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.949703828, 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:52.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:03:52.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:03:52.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:03:52.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:03:52.987 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.987 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.987 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.987 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.949687795, 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.989 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.990 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:03:52.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:03:52.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.991 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.991 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:03:52.991 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:03:52.991 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.991 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.991 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.991 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.949638556, 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.992 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:03:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:03:52.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:03:52.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:03:52.994 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:03:52.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.994 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.994 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.994 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:52.995 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:52.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.996 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.996 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:52.996 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:52.996 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.94962049599999, 2024-09-13T07:03:52.996 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:03:52.996 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:52.996 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:52.996 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:52.996 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:52.996 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:52.996 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:03:52.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.997 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:52.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:52.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:52.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:52.997 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:52.997 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:52.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.997 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.997 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:52.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:52.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:52.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:52.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:03:52.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:03:52.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.998 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.998 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:52.998 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:03:52.998 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:03:52.998 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:03:52.999 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:03:53.000 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:53.000 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:53.000 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:53.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.000 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.000 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:53.000 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:53.000 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.949603981, 2024-09-13T07:03:53.000 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:03:53.000 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:53.000 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:53.000 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:53.000 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:53.001 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:53.001 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:03:53.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.001 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:53.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:53.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:53.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:53.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.001 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.001 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:53.001 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:53.001 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:03:53.002 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:03:53.003 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.94958561599999, 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:53.004 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.005 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:03:53.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:53.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:03:53.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:03:53.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:53.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:03:53.007 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:03:53.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.007 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.007 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:53.007 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.949569518, 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:53.008 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:53.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:03:53.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:03:53.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:53.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:03:53.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:03:53.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:53.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:03:53.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:03:53.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:53.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:03:53.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:03:53.012 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:53.012 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:53.012 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:53.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.012 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:53.012 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:53.012 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.949553583, 2024-09-13T07:03:53.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:03:53.012 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:53.012 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:53.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:53.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:03:53.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:03:53.016 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:53.016 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:53.016 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:53.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.016 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:53.016 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:53.016 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 114.94953708600001, 2024-09-13T07:03:53.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:03:53.016 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:53.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:53.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:53.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:53.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:53.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:53.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.018 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:53.018 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:03:53.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:03:53.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:03:53.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:53.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:53.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:53.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:03:53.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:03:53.020 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:53.020 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:53.020 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:53.020 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:53.020 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:53.020 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:53.021 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:53.022 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:53.022 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:53.022 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:53.022 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:53.030 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:53.031+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:53.034 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:53.057 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:53.057 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:53.058 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:53.058 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:53.058 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:53.058 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:53.058 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:03:53.058 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:53.058 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:53.061 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:03:53.062 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:03:53.130 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:53.130 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:53.131 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:53.131+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:53.140 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:53.171 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:03:53.232 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:53.229+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:53.250 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:53.250 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:53.255 INFO:teuthology.orchestra.run.smithi164.stderr:Exporting 2.3 info 2.3( v 18'1 (0'0,18'1] local-lis/les=98/99 n=1 ec=17/17 lis/c=98/98 les/c/f=99/99/0 sis=98) 2024-09-13T07:03:53.261 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:53.261 INFO:teuthology.orchestra.run.smithi164.stderr:Read #2:c4c92e5a:::rbd_trash:head# 2024-09-13T07:03:53.261 INFO:teuthology.orchestra.run.smithi164.stderr:Export successful 2024-09-13T07:03:53.332 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:53.331+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:53.371 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:53.371 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:53.381 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:53.434 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:53.435+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:53.482 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:53.482 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:53.492 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:53.535 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:53.535+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:53.635 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:53.635+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:53.641 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:53.641 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:53.651 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:53.736 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:53.735+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:53.760 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:53.760 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:53.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:53.837 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:53.833+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:53.840 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --force --op remove --pgid 2.3 2024-09-13T07:03:53.895 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:53.895 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:53.906 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:53.938 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:53.933+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:54.003 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:54.003 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:54.012 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:54.039 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:54.039+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:54.140 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:54.139+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:54.167 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:54.167 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:54.174 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:03:54.179 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:54.240 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:54.239+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:54.286 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:54.287 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:54.296 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:54.342 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:54.343+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:54.407 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:54.407 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:54.417 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:54.442 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:54.441+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:54.520 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:54.520 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:54.530 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:54.543 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:54.543+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:54.644 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:54.643+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:54.683 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:54.684 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:54.693 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:54.745 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:54.747+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:54.794 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:54.794 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:54.804 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:54.846 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:54.847+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:54.907 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:54.907 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:54.916 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:54.961 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:54.963+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:55.033 INFO:teuthology.orchestra.run.smithi164.stdout: marking collection for removal 2024-09-13T07:03:55.033 INFO:teuthology.orchestra.run.smithi164.stdout:setting '_remove' omap key 2024-09-13T07:03:55.033 INFO:teuthology.orchestra.run.smithi164.stdout:finish_remove_pgs 2.3_head removing 2.3 2024-09-13T07:03:55.034 INFO:teuthology.orchestra.run.smithi164.stdout:Remove successful 2024-09-13T07:03:55.035 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:55.035 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:55.045 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:55.048 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:55.047+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:55.063 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:03:55.149 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:55.151+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:55.157 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:55.157 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:55.168 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:55.201 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:55.219 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:55.236 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:55.250 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:55.249+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:55.255 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:55.279 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:55.279 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:55.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:55.279 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:55.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.280 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137474 3.6 3:65c31370:::rbd_data.1194af11889b.000000000000013b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 186185~11715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-13T07:03:55.280 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:55.280 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.535796804999997, 2024-09-13T07:03:55.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0054357140000001, 2024-09-13T07:03:55.280 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.280 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:55.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.280 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137474 2024-09-13T07:03:55.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.280 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743934+0000", 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:55.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743949+0000", 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5705999999999998e-05 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743960+0000", 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0084e-05 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743971+0000", 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0942e-05 2024-09-13T07:03:55.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744045+0000", 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4626e-05 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744062+0000", 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7339000000000001e-05 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746366+0000", 2024-09-13T07:03:55.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.002303457 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746520+0000", 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015385900000000001 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746734+0000", 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000214253 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:55.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749128+0000", 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002394129 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749267+0000", 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013878299999999999 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749328+0000", 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0918000000000002e-05 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749371+0000", 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3188000000000001e-05 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163305 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2387968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.601578323999998, 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044745869000000001, 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163305 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678153+0000", 2024-09-13T07:03:55.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678156+0000", 2024-09-13T07:03:55.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6350000000000002e-06 2024-09-13T07:03:55.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678161+0000", 2024-09-13T07:03:55.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:03:55.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678166+0000", 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.498e-06 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722751+0000", 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044585134999999998 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722836+0000", 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4672999999999994e-05 2024-09-13T07:03:55.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722900+0000", 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4042999999999997e-05 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163307 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.60155786, 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045060944999999998, 2024-09-13T07:03:55.290 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163307 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678177+0000", 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678183+0000", 2024-09-13T07:03:55.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0599999999999996e-06 2024-09-13T07:03:55.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678188+0000", 2024-09-13T07:03:55.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:03:55.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722941+0000", 2024-09-13T07:03:55.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044752844999999999 2024-09-13T07:03:55.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723073+0000", 2024-09-13T07:03:55.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013128599999999999 2024-09-13T07:03:55.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723235+0000", 2024-09-13T07:03:55.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000162671 2024-09-13T07:03:55.294 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.294 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.294 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163310 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 544768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.601538907999998, 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045290534, 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163310 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.295 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:03:55.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678195+0000", 2024-09-13T07:03:55.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.785e-06 2024-09-13T07:03:55.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678201+0000", 2024-09-13T07:03:55.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:03:55.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678205+0000", 2024-09-13T07:03:55.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6999999999999999e-06 2024-09-13T07:03:55.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723299+0000", 2024-09-13T07:03:55.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045093964 2024-09-13T07:03:55.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723384+0000", 2024-09-13T07:03:55.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4562999999999997e-05 2024-09-13T07:03:55.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723484+0000", 2024-09-13T07:03:55.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100045 2024-09-13T07:03:55.299 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.299 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.299 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.299 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163313 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1093632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.299 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:03:55.299 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.601522923000001, 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045562975999999998, 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163313 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678212+0000", 2024-09-13T07:03:55.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8650000000000001e-06 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678217+0000", 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2660000000000001e-06 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678221+0000", 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-06 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723552+0000", 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045330489000000002 2024-09-13T07:03:55.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723660+0000", 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010770800000000001 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723772+0000", 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112608 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.303 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163317 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1642496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.601494115000001, 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045770165000000002, 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163317 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678237+0000", 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678241+0000", 2024-09-13T07:03:55.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2669999999999998e-06 2024-09-13T07:03:55.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678247+0000", 2024-09-13T07:03:55.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7309999999999996e-06 2024-09-13T07:03:55.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678252+0000", 2024-09-13T07:03:55.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-13T07:03:55.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723830+0000", 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045577234000000001 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723936+0000", 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106277 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724008+0000", 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2422999999999994e-05 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.307 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163320 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.601473690999999, 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045968883000000002, 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163320 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.308 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:03:55.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678261+0000", 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8090000000000001e-06 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678267+0000", 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8860000000000001e-06 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678271+0000", 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4549999999999997e-06 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724061+0000", 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045789656999999997 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724151+0000", 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9734999999999994e-05 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724227+0000", 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6848999999999996e-05 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163324 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.601189357999999, 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045988788000000003, 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.312 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163324 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678542+0000", 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678547+0000", 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6829999999999999e-06 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678556+0000", 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1929999999999999e-06 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678562+0000", 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.083e-06 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724267+0000", 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045705427 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724417+0000", 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014984699999999999 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724532+0000", 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011430900000000001 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163327 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 733184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.601164323999999, 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046185902000000001, 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.316 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163327 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678567+0000", 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678571+0000", 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3799999999999998e-06 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678576+0000", 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7760000000000003e-06 2024-09-13T07:03:55.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678581+0000", 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8640000000000004e-06 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724561+0000", 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045979178000000002 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724649+0000", 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8677000000000005e-05 2024-09-13T07:03:55.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724754+0000", 2024-09-13T07:03:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104527 2024-09-13T07:03:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163331 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:03:55.320 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.60114617, 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046361773000000002, 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163331 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:03:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678585+0000", 2024-09-13T07:03:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678588+0000", 2024-09-13T07:03:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0469999999999998e-06 2024-09-13T07:03:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678595+0000", 2024-09-13T07:03:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0399999999999998e-06 2024-09-13T07:03:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678599+0000", 2024-09-13T07:03:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:03:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724780+0000", 2024-09-13T07:03:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046181168000000002 2024-09-13T07:03:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724871+0000", 2024-09-13T07:03:55.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0562000000000005e-05 2024-09-13T07:03:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724948+0000", 2024-09-13T07:03:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6985999999999998e-05 2024-09-13T07:03:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163334 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1662976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:03:55.325 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.601128967000001, 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046544502000000001, 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163334 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678605+0000", 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5380000000000001e-06 2024-09-13T07:03:55.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678611+0000", 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7389999999999996e-06 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678615+0000", 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724987+0000", 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046371969999999998 2024-09-13T07:03:55.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725075+0000", 2024-09-13T07:03:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8091999999999996e-05 2024-09-13T07:03:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725148+0000", 2024-09-13T07:03:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2206999999999995e-05 2024-09-13T07:03:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.329 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163338 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.601112236999999, 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046738452999999999, 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163338 2024-09-13T07:03:55.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678619+0000", 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678622+0000", 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.649e-06 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678628+0000", 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5029999999999997e-06 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678632+0000", 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4020000000000004e-06 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725182+0000", 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046550389999999997 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725280+0000", 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7260000000000001e-05 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725358+0000", 2024-09-13T07:03:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8720000000000005e-05 2024-09-13T07:03:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163339 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2363392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:03:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.601096709, 2024-09-13T07:03:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046920835000000001, 2024-09-13T07:03:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163339 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678637+0000", 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2479999999999999e-06 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678642+0000", 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0030000000000002e-06 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678646+0000", 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9870000000000001e-06 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725397+0000", 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046750127000000002 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725482+0000", 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5928999999999997e-05 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725556+0000", 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3823000000000001e-05 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163342 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.60108207, 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047212991000000003, 2024-09-13T07:03:55.338 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163342 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678652+0000", 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.486e-06 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678657+0000", 2024-09-13T07:03:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6909999999999996e-06 2024-09-13T07:03:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678660+0000", 2024-09-13T07:03:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8009999999999999e-06 2024-09-13T07:03:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725597+0000", 2024-09-13T07:03:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046937372999999998 2024-09-13T07:03:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725745+0000", 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000148314 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725863+0000", 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011768 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.342 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163345 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.600220507, 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046569193000000002, 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163345 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:03:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:03:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678662+0000", 2024-09-13T07:03:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9991498 2024-09-13T07:03:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679516+0000", 2024-09-13T07:03:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000854789 2024-09-13T07:03:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679530+0000", 2024-09-13T07:03:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3492999999999999e-05 2024-09-13T07:03:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679537+0000", 2024-09-13T07:03:55.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.323e-06 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725892+0000", 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046354244000000003 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725975+0000", 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3356000000000006e-05 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726081+0000", 2024-09-13T07:03:55.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105992 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163349 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1298432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.600188444, 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046744247000000003, 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.347 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163349 2024-09-13T07:03:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:03:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:03:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679543+0000", 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679546+0000", 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679551+0000", 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:03:55.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679554+0000", 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8629999999999999e-06 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726117+0000", 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046563081999999999 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726213+0000", 2024-09-13T07:03:55.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6207999999999998e-05 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726288+0000", 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4741999999999997e-05 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163350 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1449984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.600175818, 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046935635000000003, 2024-09-13T07:03:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163350 2024-09-13T07:03:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:03:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:03:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:03:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679558+0000", 2024-09-13T07:03:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.928e-06 2024-09-13T07:03:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679561+0000", 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7659999999999999e-06 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679564+0000", 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4960000000000001e-06 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726327+0000", 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046762321000000003 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726415+0000", 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8487999999999997e-05 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726492+0000", 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6995000000000004e-05 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163353 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.356 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:03:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.600165708999999, 2024-09-13T07:03:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047130755000000003, 2024-09-13T07:03:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163353 2024-09-13T07:03:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:03:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:03:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:03:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:55.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679568+0000", 2024-09-13T07:03:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5969999999999999e-06 2024-09-13T07:03:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679571+0000", 2024-09-13T07:03:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:03:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679573+0000", 2024-09-13T07:03:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:03:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726523+0000", 2024-09-13T07:03:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046949882999999998 2024-09-13T07:03:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726616+0000", 2024-09-13T07:03:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3236e-05 2024-09-13T07:03:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726697+0000", 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0834000000000006e-05 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163356 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2379776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 26.600156027000001, 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047304662999999997, 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.361 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163356 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:03:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679578+0000", 2024-09-13T07:03:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:03:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679582+0000", 2024-09-13T07:03:55.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679589+0000", 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1149999999999996e-06 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726723+0000", 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047133768999999999 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726809+0000", 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5402000000000007e-05 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726881+0000", 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2152999999999998e-05 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:166663 3.b 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8129196~12979] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.881781513, 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.061564718999999997, 2024-09-13T07:03:55.365 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 166663 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397947+0000", 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999967 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412144+0000", 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014196972 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412172+0000", 2024-09-13T07:03:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7781999999999999e-05 2024-09-13T07:03:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412182+0000", 2024-09-13T07:03:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6069999999999997e-06 2024-09-13T07:03:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412239+0000", 2024-09-13T07:03:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.732e-05 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412365+0000", 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125356 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.413178+0000", 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081355299999999995 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.443176+0000", 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029997903999999999 2024-09-13T07:03:55.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459446+0000", 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016269921999999999 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459484+0000", 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8488000000000001e-05 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459515+0000", 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0981e-05 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.370 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.371 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.371 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:55.371 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:55.372 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:55.373 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:55.373 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:55.373 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:03:55.374 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:55.374 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:55.374 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:55.374 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:55.374 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:55.374 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:03:55.374 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:55.374 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:55.375 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:03:55.375 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:55.371+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:55.473 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:55.469+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:55.527 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:55.527 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:55.527 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:55.527 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:55.537 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:55.574 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:55.569+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:55.600 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.3.5 2024-09-13T07:03:55.639 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:55.639 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:55.640 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:55.640 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:55.640 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:55.649 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:55.675 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:55.674+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:55.758 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:55.758 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:55.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:55.758 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:55.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.759 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:03:55.759 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:55.759 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.26348645600001, 2024-09-13T07:03:55.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:03:55.759 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.759 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.759 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.759 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.759 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.759 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:03:55.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:03:55.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:55.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:03:55.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:03:55.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:03:55.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:03:55.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:03:55.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:03:55.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:03:55.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:03:55.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:03:55.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:03:55.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:03:55.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:03:55.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:03:55.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:03:55.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:03:55.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:03:55.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:03:55.764 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.764 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.764 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.764 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.764 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.777002892, 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:55.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:55.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:03:55.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:03:55.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:03:55.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:03:55.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:03:55.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:03:55.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:03:55.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:03:55.769 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.769 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.769 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.769 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.769 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:55.769 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.77693296299999, 2024-09-13T07:03:55.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:03:55.769 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:03:55.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:03:55.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:03:55.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:03:55.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:03:55.773 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.773 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.773 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.776917091, 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:03:55.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:03:55.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:03:55.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:03:55.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:03:55.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:03:55.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:03:55.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:03:55.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:03:55.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:03:55.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:03:55.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:03:55.778 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.778 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.778 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.778 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.778 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:55.778 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.77684663399999, 2024-09-13T07:03:55.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:03:55.778 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.778 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:55.779 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.779 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.779 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.779 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:03:55.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.779 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:55.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:55.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:03:55.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:03:55.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:03:55.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:03:55.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:03:55.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:03:55.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:03:55.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:03:55.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:03:55.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:55.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:03:55.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:03:55.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:03:55.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:03:55.783 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.783 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.783 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.783 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.783 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:55.783 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.77349459600001, 2024-09-13T07:03:55.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:03:55.783 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.784 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.784 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.784 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.784 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.784 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:03:55.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.784 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:55.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:55.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:03:55.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:03:55.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:03:55.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:03:55.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:03:55.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:03:55.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:03:55.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:03:55.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:03:55.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:03:55.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:03:55.788 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.788 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.788 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.788 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.788 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:55.788 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.773478295, 2024-09-13T07:03:55.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:03:55.788 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.789 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.789 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.789 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.789 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.789 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:03:55.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.789 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:55.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:03:55.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:03:55.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:03:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:03:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:03:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:03:55.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:03:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:03:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:03:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:03:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:03:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.793 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.793 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:55.793 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.77345851, 2024-09-13T07:03:55.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:03:55.793 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.793 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.793 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:03:55.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.794 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:55.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:55.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:03:55.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:03:55.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:03:55.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:03:55.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:03:55.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:03:55.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:03:55.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:03:55.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:03:55.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:03:55.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:03:55.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.797 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.797 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.797 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:55.797 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.773442237, 2024-09-13T07:03:55.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:03:55.797 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:03:55.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:03:55.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:03:55.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:03:55.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:03:55.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:03:55.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.773424412, 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:03:55.801 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:55.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:03:55.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:03:55.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:03:55.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:03:55.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:03:55.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:03:55.805 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.805 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.805 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.805 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.805 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.773408319, 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:03:55.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:03:55.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:03:55.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:55.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:03:55.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:03:55.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:03:55.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:03:55.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:03:55.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:03:55.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:03:55.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:03:55.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.809 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.773387133, 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.810 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:55.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:03:55.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:03:55.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:03:55.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:03:55.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:03:55.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:03:55.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:03:55.813 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.77337110000001, 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.814 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:03:55.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:03:55.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:03:55.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:03:55.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:03:55.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:03:55.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.818 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.818 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.818 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.818 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:55.818 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.773321861, 2024-09-13T07:03:55.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:03:55.818 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.818 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.818 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.818 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.818 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:03:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:03:55.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:03:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:03:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:03:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:03:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:03:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.773303801, 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.822 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:55.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:03:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:03:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:03:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:03:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:03:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:03:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.773287286, 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.826 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:03:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:03:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:03:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:03:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:03:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:03:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:03:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.830 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.773268921, 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:03:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:03:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:03:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:03:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:03:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:03:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:03:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:03:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:03:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:03:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.834 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.77325282300001, 2024-09-13T07:03:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:03:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.835 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:03:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:03:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:03:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:03:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:03:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:03:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:03:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:03:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:03:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:03:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.773236888, 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.839 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:03:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:03:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:03:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 117.773220391, 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:55.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:03:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:03:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:03:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:03:55.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:55.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:03:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:03:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:55.847 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:55.847 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:55.848 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:55.849 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:55.846+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:55.949 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:55.950+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:56.002 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:56.002 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:56.002 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:56.003 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:56.012 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:56.050 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:56.045+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:56.120 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:56.120 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:56.120 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:56.120 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:56.120 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:56.130 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:56.151 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:56.150+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:56.189 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:03:56.248 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.356 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:03:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.190408152, 2024-09-13T07:03:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:03:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:03:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:03:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:03:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:03:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:03:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:03:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:03:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:03:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:03:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:03:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:03:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:03:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:03:56.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 66.372191795000006, 2024-09-13T07:03:56.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:03:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:03:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:03:56.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:03:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:03:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:03:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:03:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:03:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.365 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 66.372142406999998, 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:03:56.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:03:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:03:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:03:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:03:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:03:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:03:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:03:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:03:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:03:56.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 66.372119699999999, 2024-09-13T07:03:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:03:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:03:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:03:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:03:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:03:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:03:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:03:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:03:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:03:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:03:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:03:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:03:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:03:56.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 66.372075507000005, 2024-09-13T07:03:56.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:03:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:03:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:03:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:03:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:03:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:03:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:03:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:03:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:03:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:03:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:03:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:03:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:03:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:03:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:03:56.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 66.341096418000006, 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:03:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:03:56.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:56.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137457 3.18 3:1a596dcc:::rbd_data.1194af11889b.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:56.384 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.595474608000004, 2024-09-13T07:03:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033734403000000003, 2024-09-13T07:03:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137457 2024-09-13T07:03:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:56.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:03:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651523+0000", 2024-09-13T07:03:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651531+0000", 2024-09-13T07:03:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.328e-06 2024-09-13T07:03:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651545+0000", 2024-09-13T07:03:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4270000000000001e-05 2024-09-13T07:03:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651555+0000", 2024-09-13T07:03:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5210000000000008e-06 2024-09-13T07:03:56.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651626+0000", 2024-09-13T07:03:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1915999999999994e-05 2024-09-13T07:03:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651761+0000", 2024-09-13T07:03:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000134838 2024-09-13T07:03:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:56.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.658577+0000", 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068151820000000004 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685181+0000", 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026604200000000001 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685193+0000", 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2128e-05 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685226+0000", 2024-09-13T07:03:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3504999999999999e-05 2024-09-13T07:03:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685259+0000", 2024-09-13T07:03:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2866e-05 2024-09-13T07:03:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138565 3.0 3:002d0425:::rbd_data.11949f62878b.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.582104115999996, 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033394489999999999, 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138565 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:03:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664893+0000", 2024-09-13T07:03:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:03:56.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664906+0000", 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2996000000000001e-05 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664936+0000", 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9609999999999999e-05 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664949+0000", 2024-09-13T07:03:56.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383e-05 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665046+0000", 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6459000000000006e-05 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665259+0000", 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000213141 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:56.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.672522+0000", 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072632110000000003 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.697982+0000", 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025459699999999998 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698018+0000", 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6375999999999999e-05 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:56.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698166+0000", 2024-09-13T07:03:56.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014818700000000001 2024-09-13T07:03:56.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698290+0000", 2024-09-13T07:03:56.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012333999999999999 2024-09-13T07:03:56.396 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.396 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.396 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.396 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138587 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:03:56.397 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:56.397 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.533638449000001, 2024-09-13T07:03:56.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019224502000000001, 2024-09-13T07:03:56.397 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.397 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:56.397 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.397 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.397 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.397 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138587 2024-09-13T07:03:56.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.397 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713360+0000", 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:56.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713368+0000", 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713383+0000", 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5176e-05 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713390+0000", 2024-09-13T07:03:56.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8669999999999996e-06 2024-09-13T07:03:56.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713449+0000", 2024-09-13T07:03:56.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8938999999999999e-05 2024-09-13T07:03:56.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713549+0000", 2024-09-13T07:03:56.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010009 2024-09-13T07:03:56.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:56.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.716087+0000", 2024-09-13T07:03:56.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025383820000000001 2024-09-13T07:03:56.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:56.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732351+0000", 2024-09-13T07:03:56.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016263564000000001 2024-09-13T07:03:56.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:56.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732450+0000", 2024-09-13T07:03:56.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9529000000000002e-05 2024-09-13T07:03:56.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:56.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732532+0000", 2024-09-13T07:03:56.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1526000000000005e-05 2024-09-13T07:03:56.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732585+0000", 2024-09-13T07:03:56.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3819000000000001e-05 2024-09-13T07:03:56.402 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.402 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.402 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.403 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138786 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:56.403 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:56.403 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.099423194000003, 2024-09-13T07:03:56.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018847452000000001, 2024-09-13T07:03:56.403 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.403 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:56.403 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.403 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.403 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.403 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138786 2024-09-13T07:03:56.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147575+0000", 2024-09-13T07:03:56.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:56.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147583+0000", 2024-09-13T07:03:56.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:03:56.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147601+0000", 2024-09-13T07:03:56.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7961e-05 2024-09-13T07:03:56.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147610+0000", 2024-09-13T07:03:56.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-13T07:03:56.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147661+0000", 2024-09-13T07:03:56.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0447e-05 2024-09-13T07:03:56.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147754+0000", 2024-09-13T07:03:56.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2974999999999997e-05 2024-09-13T07:03:56.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:56.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.150580+0000", 2024-09-13T07:03:56.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002826566 2024-09-13T07:03:56.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:56.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166135+0000", 2024-09-13T07:03:56.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015554772 2024-09-13T07:03:56.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166225+0000", 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0258000000000002e-05 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166334+0000", 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010816500000000001 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166424+0000", 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0116000000000003e-05 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.408 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.409 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.409 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:139127 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:03:56.409 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:56.409 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.472372229999998, 2024-09-13T07:03:56.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019295931999999998, 2024-09-13T07:03:56.409 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.409 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:56.409 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.409 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.409 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.409 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139127 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774626+0000", 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774635+0000", 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5419999999999994e-06 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774654+0000", 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8359000000000002e-05 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774663+0000", 2024-09-13T07:03:56.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-13T07:03:56.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774718+0000", 2024-09-13T07:03:56.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5315000000000001e-05 2024-09-13T07:03:56.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774805+0000", 2024-09-13T07:03:56.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7284e-05 2024-09-13T07:03:56.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:03:56.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.779319+0000", 2024-09-13T07:03:56.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045138310000000003 2024-09-13T07:03:56.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:56.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793772+0000", 2024-09-13T07:03:56.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014452692 2024-09-13T07:03:56.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:56.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793821+0000", 2024-09-13T07:03:56.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9563e-05 2024-09-13T07:03:56.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:56.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793878+0000", 2024-09-13T07:03:56.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6953999999999997e-05 2024-09-13T07:03:56.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793923+0000", 2024-09-13T07:03:56.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4781999999999999e-05 2024-09-13T07:03:56.414 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.415 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.415 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.415 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:164691 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:56.415 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:03:56.415 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.188412356000001, 2024-09-13T07:03:56.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015985102000000001, 2024-09-13T07:03:56.415 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.415 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.415 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.415 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.415 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 164691 2024-09-13T07:03:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:03:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:03:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:03:56.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058589+0000", 2024-09-13T07:03:56.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2510000000000001e-06 2024-09-13T07:03:56.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058594+0000", 2024-09-13T07:03:56.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9089999999999999e-06 2024-09-13T07:03:56.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058598+0000", 2024-09-13T07:03:56.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2069999999999997e-06 2024-09-13T07:03:56.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064175+0000", 2024-09-13T07:03:56.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055769909999999999 2024-09-13T07:03:56.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064210+0000", 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4966999999999999e-05 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074572+0000", 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010362219000000001 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.419 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165201 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:56.420 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:03:56.420 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.185884427000001, 2024-09-13T07:03:56.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013054266, 2024-09-13T07:03:56.420 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.420 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.420 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.420 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.420 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.420 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165201 2024-09-13T07:03:56.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.420 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:03:56.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:03:56.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:03:56.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061117+0000", 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.12e-06 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061125+0000", 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8539999999999997e-06 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061129+0000", 2024-09-13T07:03:56.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:03:56.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074092+0000", 2024-09-13T07:03:56.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012963166999999999 2024-09-13T07:03:56.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074136+0000", 2024-09-13T07:03:56.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3578000000000002e-05 2024-09-13T07:03:56.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074169+0000", 2024-09-13T07:03:56.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3581000000000003e-05 2024-09-13T07:03:56.424 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.424 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.424 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.424 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165202 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7831552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:56.424 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:03:56.424 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.185866751999999, 2024-09-13T07:03:56.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013116245, 2024-09-13T07:03:56.425 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.425 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.425 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.425 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.425 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.425 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165202 2024-09-13T07:03:56.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.425 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:03:56.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:03:56.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061132+0000", 2024-09-13T07:03:56.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061135+0000", 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3410000000000001e-06 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061142+0000", 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7079999999999997e-06 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061147+0000", 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8930000000000004e-06 2024-09-13T07:03:56.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074180+0000", 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013032346 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074217+0000", 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074249+0000", 2024-09-13T07:03:56.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1711000000000002e-05 2024-09-13T07:03:56.429 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.429 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.429 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.429 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165203 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:56.429 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:03:56.429 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.185848638, 2024-09-13T07:03:56.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013180211000000001, 2024-09-13T07:03:56.429 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.429 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.429 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.429 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165203 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061150+0000", 2024-09-13T07:03:56.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061153+0000", 2024-09-13T07:03:56.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.108e-06 2024-09-13T07:03:56.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061158+0000", 2024-09-13T07:03:56.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0649999999999998e-06 2024-09-13T07:03:56.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061162+0000", 2024-09-13T07:03:56.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0080000000000004e-06 2024-09-13T07:03:56.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074262+0000", 2024-09-13T07:03:56.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013100557000000001 2024-09-13T07:03:56.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074299+0000", 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6981999999999999e-05 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074331+0000", 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1801000000000003e-05 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.433 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165204 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8093696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.185833927000001, 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0132388, 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165204 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.435 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061167+0000", 2024-09-13T07:03:56.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1409999999999999e-06 2024-09-13T07:03:56.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061175+0000", 2024-09-13T07:03:56.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5519999999999999e-06 2024-09-13T07:03:56.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.436 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.436 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.436 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061179+0000", 2024-09-13T07:03:56.436 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.172e-06 2024-09-13T07:03:56.436 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074342+0000", 2024-09-13T07:03:56.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013162821999999999 2024-09-13T07:03:56.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074373+0000", 2024-09-13T07:03:56.437 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1103999999999999e-05 2024-09-13T07:03:56.437 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.437 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.437 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.437 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074404+0000", 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1359999999999998e-05 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165205 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.185816968000001, 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013300635, 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.438 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.439 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165205 2024-09-13T07:03:56.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.439 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:03:56.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.439 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.439 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.439 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:03:56.439 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.439 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:03:56.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061184+0000", 2024-09-13T07:03:56.440 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0779999999999998e-06 2024-09-13T07:03:56.440 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.440 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.440 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.440 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061192+0000", 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7209999999999994e-06 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061196+0000", 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4469999999999999e-06 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074417+0000", 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013220750999999999 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.441 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074452+0000", 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4814999999999998e-05 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074483+0000", 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1118000000000003e-05 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165206 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:03:56.442 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.185798847000001, 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013364454, 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165206 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:03:56.443 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061200+0000", 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.444 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061203+0000", 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.559e-06 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061210+0000", 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061214+0000", 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0999999999999997e-06 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.445 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074494+0000", 2024-09-13T07:03:56.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013279094 2024-09-13T07:03:56.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074530+0000", 2024-09-13T07:03:56.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6538999999999997e-05 2024-09-13T07:03:56.446 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.446 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.446 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.446 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074565+0000", 2024-09-13T07:03:56.446 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4972999999999999e-05 2024-09-13T07:03:56.446 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.447 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.447 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.447 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.447 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.447 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169377 3.0 3:07b65a69:::rbd_data.1194e181a50b.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-13T07:03:56.447 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:03:56.447 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 23.890132452, 2024-09-13T07:03:56.447 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.032744151999999999, 2024-09-13T07:03:56.447 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.447 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:56.447 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.447 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.447 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169377 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.448 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356864+0000", 2024-09-13T07:03:56.449 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:56.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356880+0000", 2024-09-13T07:03:56.449 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6022999999999999e-05 2024-09-13T07:03:56.449 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.449 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.449 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.449 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356916+0000", 2024-09-13T07:03:56.449 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5921999999999999e-05 2024-09-13T07:03:56.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.450 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.450 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356931+0000", 2024-09-13T07:03:56.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4311000000000001e-05 2024-09-13T07:03:56.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.450 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.450 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357011+0000", 2024-09-13T07:03:56.450 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9992000000000003e-05 2024-09-13T07:03:56.450 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.450 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.450 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357143+0000", 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000131822 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.361578+0000", 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00443525 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389466+0000", 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027888342999999999 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.451 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:56.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389529+0000", 2024-09-13T07:03:56.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2460999999999997e-05 2024-09-13T07:03:56.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:56.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389574+0000", 2024-09-13T07:03:56.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5405999999999999e-05 2024-09-13T07:03:56.452 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.452 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.452 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.452 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389611+0000", 2024-09-13T07:03:56.452 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7122000000000001e-05 2024-09-13T07:03:56.453 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.453 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.453 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.453 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.453 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.453 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169731 3.0 3:031f8e93:::rbd_data.1194e181a50b.0000000000000006:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-13T07:03:56.453 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:03:56.453 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 22.965706654000002, 2024-09-13T07:03:56.453 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.021389756999999999, 2024-09-13T07:03:56.453 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.453 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:56.453 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.453 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169731 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.454 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281291+0000", 2024-09-13T07:03:56.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:03:56.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.455 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281302+0000", 2024-09-13T07:03:56.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0651e-05 2024-09-13T07:03:56.455 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.455 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.455 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.455 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281324+0000", 2024-09-13T07:03:56.455 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1713999999999999e-05 2024-09-13T07:03:56.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.456 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281340+0000", 2024-09-13T07:03:56.456 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:03:56.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.456 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281450+0000", 2024-09-13T07:03:56.456 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110086 2024-09-13T07:03:56.456 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.456 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.456 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281593+0000", 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014306 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.286356+0000", 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047626320000000002 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302404+0000", 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016048605000000001 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.457 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:03:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302476+0000", 2024-09-13T07:03:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1483000000000007e-05 2024-09-13T07:03:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302578+0000", 2024-09-13T07:03:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101883 2024-09-13T07:03:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302683+0000", 2024-09-13T07:03:56.458 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104681 2024-09-13T07:03:56.459 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.459 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.459 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.459 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.459 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.459 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:56.460 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:56.461 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:56.457+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:56.562 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:56.557+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:56.567 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:56.567 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:56.567 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:03:56.567 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:56.577 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:56.663 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:56.661+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:56.682 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:56.682 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:03:56.682 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:03:56.682 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:03:56.682 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:56.691 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:56.764 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:56.762+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:56.801 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:03:56.801 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:03:56.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:03:56.801 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:03:56.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.801 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.801 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:56.801 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.678760123999993, 2024-09-13T07:03:56.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:03:56.801 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.801 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:03:56.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:03:56.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:03:56.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.678739647, 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:03:56.805 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:03:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:03:56.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:03:56.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:03:56.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:03:56.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:03:56.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:03:56.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.809 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.810 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.810 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.810 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:56.810 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.672043044000006, 2024-09-13T07:03:56.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:03:56.810 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.810 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.811 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:03:56.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.811 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:56.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:56.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:03:56.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:56.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:03:56.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:03:56.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:03:56.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:03:56.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:03:56.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:03:56.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:03:56.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:03:56.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:03:56.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:03:56.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:03:56.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:03:56.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.814 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.672030974999998, 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:56.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:56.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:03:56.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:56.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:03:56.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:03:56.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:03:56.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:03:56.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:03:56.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:03:56.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:03:56.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:03:56.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:03:56.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:03:56.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.672018531999996, 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:03:56.819 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.820 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.820 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:03:56.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.820 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:56.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:56.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:03:56.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:56.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:03:56.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:03:56.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:03:56.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:03:56.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:03:56.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:03:56.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:03:56.823 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.823 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.824 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.824 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.824 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:56.824 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.672000933999996, 2024-09-13T07:03:56.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:03:56.824 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.824 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.824 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:03:56.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.825 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:56.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:56.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:03:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:56.828 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.671990230999995, 2024-09-13T07:03:56.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:03:56.829 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.829 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.829 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:03:56.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.829 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:56.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:56.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:03:56.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:03:56.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:03:56.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:03:56.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:03:56.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:03:56.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:03:56.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.833 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.833 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.833 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:56.833 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.671946297999995, 2024-09-13T07:03:56.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:03:56.833 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.833 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.834 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:03:56.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.834 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:56.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:03:56.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.837 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.671926511999999, 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:56.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:03:56.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.841 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.671910761999996, 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:03:56.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:03:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:03:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:03:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:03:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:03:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:03:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:03:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:03:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:03:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:03:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:03:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:03:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:03:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.846 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.671899510000003, 2024-09-13T07:03:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:03:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:03:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:03:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:03:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:03:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:03:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:03:56.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.671888910999996, 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.851 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:03:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:03:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:03:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:03:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:03:56.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:03:56.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:03:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:03:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:03:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:56.855 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.671879418000003, 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:03:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:03:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:03:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:03:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:03:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:03:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:03:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:03:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.671869873999995, 2024-09-13T07:03:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:03:56.860 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:03:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:03:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:03:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:03:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:03:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:03:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:03:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:03:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:03:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:03:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:03:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.864 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.671837271000001, 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:03:56.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:03:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:03:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:03:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:03:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:03:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:03:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:03:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:03:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:03:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.869 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.671824247999993, 2024-09-13T07:03:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:03:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:03:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:03:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:03:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:03:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:03:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:03:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:03:56.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:03:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:03:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:03:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:03:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.668473262999996, 2024-09-13T07:03:56.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:03:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:03:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:03:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:03:56.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:03:56.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:03:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:03:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:03:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:03:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:03:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.878 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.668452986000005, 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.879 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:03:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:03:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:03:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:03:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:03:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:03:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:03:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:03:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:03:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:03:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:03:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:03:56.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 94.668432784999993, 2024-09-13T07:03:56.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:03:56.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:03:56.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:166663 3.b e135/117 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head v 135'409, mlcod=132'407)", 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 27.385843820000002, 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045108144000000003, 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414172+0000", 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427200+0000", 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013028098 2024-09-13T07:03:56.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427229+0000", 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9178000000000001e-05 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427239+0000", 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6870000000000006e-06 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427393+0000", 2024-09-13T07:03:56.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015382899999999999 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427451+0000", 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8301000000000003e-05 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459232+0000", 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031780695999999997 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:03:56.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459282+0000", 2024-09-13T07:03:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0701000000000001e-05 2024-09-13T07:03:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:03:56.892 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:03:56.892 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:03:56.894 INFO:teuthology.orchestra.run.smithi164.stdout:get_pg_num_history pg_num_history pg_num_history(e147 pg_nums {1={14=1},2={17=8},3={22=32}} deleted_pools ) 2024-09-13T07:03:56.894 INFO:teuthology.orchestra.run.smithi164.stdout:Importing pgid 2.3 2024-09-13T07:03:56.894 INFO:teuthology.orchestra.run.smithi164.stdout:Write #2:c4c92e5a:::rbd_trash:head# 2024-09-13T07:03:56.894 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 0=[]:{} 2024-09-13T07:03:56.894 INFO:teuthology.orchestra.run.smithi164.stdout:write_pg epoch 140 info 2.3( v 18'1 (0'0,18'1] local-lis/les=98/99 n=1 ec=17/17 lis/c=98/98 les/c/f=99/99/0 sis=98) 2024-09-13T07:03:56.894 INFO:teuthology.orchestra.run.smithi164.stdout:Import successful 2024-09-13T07:03:56.894 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:56.895 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:56.894+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:56.994 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:56.994+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:57.012 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:57.012 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:57.012 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:57.012 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:57.022 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:57.095 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:57.093+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:57.132 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:57.132 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:57.132 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:57.132 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:57.132 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:57.142 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:57.196 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:57.193+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:57.201 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:03:57.259 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:57.259 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:03:57.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:03:57.259 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:03:57.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.260 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.260 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:57.260 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.656000191000004, 2024-09-13T07:03:57.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:03:57.260 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.260 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.260 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:03:57.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.260 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:57.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:57.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:03:57.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:03:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:03:57.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:03:57.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:03:57.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:03:57.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:03:57.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.264 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:57.265 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655969089999999, 2024-09-13T07:03:57.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:03:57.265 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.265 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.265 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:03:57.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.265 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:03:57.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:03:57.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:03:57.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655946775999993, 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.269 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:03:57.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.270 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:57.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:03:57.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:03:57.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:03:57.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.274 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.274 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:57.274 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655923744999996, 2024-09-13T07:03:57.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:03:57.274 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.274 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.274 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:03:57.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.274 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:57.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:03:57.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655899195000003, 2024-09-13T07:03:57.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.283 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.283 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:57.283 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655863486000001, 2024-09-13T07:03:57.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:03:57.283 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.283 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.283 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.283 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.283 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.283 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:03:57.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.283 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:57.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:03:57.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:03:57.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:03:57.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:57.287 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655838806999995, 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:03:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:03:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:03:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:03:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:03:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655805533000006, 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:03:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:03:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:03:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:03:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:03:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655778745000006, 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:03:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:03:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:03:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655754191, 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:03:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:03:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:03:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655724422000006, 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:03:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655664579000003, 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:03:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:03:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:03:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:03:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:03:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:03:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:03:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:03:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:03:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:03:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:03:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655597728999993, 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:03:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:03:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:03:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:03:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655565226999997, 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655530605999999, 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:03:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:03:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:03:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:03:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:03:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:03:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655480374999996, 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:03:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655432188999995, 2024-09-13T07:03:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:03:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:03:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:03:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655405551000001, 2024-09-13T07:03:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:03:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:03:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:03:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:03:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:03:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:03:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.655358715999995, 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:03:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:03:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.652004298999998, 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:03:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:03:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:03:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:03:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:03:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:03:57.351 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:57.353 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:03:57.353 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:57.350+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:57.376 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:03:57.420 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /var/log/ceph/exp.2.3.5 2024-09-13T07:03:57.453 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:57.449+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:57.463 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:57.463 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:03:57.463 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:57.473 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:03:57.524 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:57.545 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:57.554 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:57.553+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:57.564 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:57.579 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:57.579 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:57.579 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:57.580 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:57.580 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:57.583 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:57.589 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:03:57.607 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:57.631 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:57.652 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:57.653 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:57.653 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:57.653 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:57.653 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:57.653 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:57.653 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:03:57.653 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:57.653 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:57.654 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:57.654+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:57.657 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:03:57.657 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:03:57.701 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:57.701 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:57.711 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:03:57.756 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:57.754+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:57.815 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:57.815 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:57.824 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:03:57.856 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:57.853+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:57.934 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:57.934 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:57.944 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:03:57.987 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:57.985+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:58.058 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:58.058+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:58.095 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:58.095 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:58.105 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:03:58.158 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:58.157+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:58.230 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:58.230 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:58.241 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:03:58.259 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:58.258+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:58.357 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:58.358 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:58.360 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:58.358+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:58.368 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:03:58.461 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:58.462+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:58.473 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:58.473 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:58.483 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:03:58.562 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:58.562+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:58.593 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:58.593 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:58.603 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:03:58.663 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:58.662+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:58.753 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:58.754 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:58.763 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:03:58.764 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:58.761+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:58.864 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:03:58.861+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:03:58.873 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:58.873 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:58.885 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:03:58.965 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:03:58.966+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:03:58.993 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:58.993 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:59.003 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:03:59.066 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:59.066+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:59.113 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:59.113 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:59.122 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:03:59.167 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:59.166+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:59.215 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:03:59.242 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:59.242 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:59.256 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:03:59.297 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:59.298+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:59.369 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:59.365+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:59.409 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:59.409 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:59.420 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:03:59.470 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:03:59.470+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:03:59.524 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:59.524 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:59.533 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:03:59.577 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:03:59.578+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:03:59.659 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:03:59.671 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:03:59.670+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:03:59.688 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:59.688 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:59.698 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:03:59.772 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:03:59.769+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:03:59.802 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:03:59.817 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:03:59.818 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:03:59.823 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:03:59.828 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:03:59.845 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:03:59.865 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:03:59.873 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:59.869+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:03:59.885 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:03:59.908 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:03:59.935 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:03:59.935 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:03:59.935 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:03:59.936 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:03:59.936 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:03:59.936 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:03:59.936 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:03:59.936 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:03:59.936 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:03:59.938 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:03:59.939 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:03:59.939 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:03:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:03:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:03:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:03:59.940 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:03:59.949 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:03:59.994 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:03:59.993+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:00.059 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:00.059 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:00.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:00.059 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:00.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.059 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457787331999995, 2024-09-13T07:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:00.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:00.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:00.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:04:00.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:04:00.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:04:00.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:04:00.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:04:00.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:04:00.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:04:00.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:04:00.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:04:00.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:04:00.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:04:00.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:04:00.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:04:00.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:04:00.065 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.065 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.065 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.066 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.066 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:00.066 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457756231000005, 2024-09-13T07:04:00.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:04:00.066 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.066 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.066 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.066 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.067 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.067 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:04:00.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.067 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:00.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:04:00.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:04:00.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:04:00.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:04:00.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:04:00.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:04:00.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:04:00.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:04:00.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:04:00.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:04:00.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:04:00.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:04:00.071 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.071 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.071 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.071 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.071 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:00.072 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457733916999999, 2024-09-13T07:04:00.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:04:00.072 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.072 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.072 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:04:00.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.072 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:00.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:00.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:04:00.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:04:00.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:04:00.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:04:00.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:04:00.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:04:00.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:04:00.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:04:00.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:04:00.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:04:00.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:04:00.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:04:00.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:04:00.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:04:00.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:04:00.080 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.080 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.080 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.081 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.081 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:00.081 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457710886000001, 2024-09-13T07:04:00.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:04:00.081 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.082 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.082 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:04:00.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.082 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:00.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:00.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:04:00.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:04:00.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:04:00.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:04:00.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:04:00.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:04:00.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:04:00.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:00.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:04:00.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:04:00.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:04:00.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:04:00.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:04:00.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:04:00.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:04:00.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.089 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.090 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.090 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:00.090 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457686335999995, 2024-09-13T07:04:00.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:04:00.090 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.090 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.090 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.091 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.091 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.091 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:04:00.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.091 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:00.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:00.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:00.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:04:00.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:04:00.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:04:00.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:04:00.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:04:00.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:04:00.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:04:00.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:04:00.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:04:00.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:04:00.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:04:00.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:04:00.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:04:00.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:04:00.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.100 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.100 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.101 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.101 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:00.101 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457650627000007, 2024-09-13T07:04:00.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:04:00.101 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.101 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.102 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.102 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.102 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.102 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:04:00.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.102 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:00.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:00.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:04:00.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:04:00.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:04:00.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:04:00.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:04:00.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:04:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:04:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:04:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:04:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:04:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:04:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457625948, 2024-09-13T07:04:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:04:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:04:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:04:00.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:04:00.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:04:00.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:04:00.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:04:00.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:04:00.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457592673999997, 2024-09-13T07:04:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:04:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:04:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:00.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:04:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:04:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:04:00.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:04:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:04:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:04:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:04:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:04:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:04:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:04:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:04:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:04:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:04:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:04:00.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:04:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.122 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457565885999998, 2024-09-13T07:04:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:04:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:04:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:04:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:04:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:04:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:04:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:04:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:04:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:04:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:04:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:04:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:04:00.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:04:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:04:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:04:00.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:04:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:04:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457541332000005, 2024-09-13T07:04:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:04:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.133 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:04:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:00.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:04:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:04:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:04:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:04:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:04:00.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:04:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:04:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:04:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:04:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:04:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:04:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:04:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457511562999997, 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.141 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:04:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:04:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:04:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:04:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:04:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:04:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:04:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457451719999995, 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:04:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:04:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457384869999999, 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457352368000002, 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457317747000005, 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:04:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:04:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:04:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:04:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:04:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:04:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457267516000002, 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:04:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:04:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:04:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:04:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:04:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:04:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457219330000001, 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:04:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457192692000007, 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:04:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.457145857, 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:04:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:04:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.453791440000003, 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:04:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:04:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:04:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:04:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:04:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.188 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:00.190 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:04:00.190 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:00.190+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:00.232 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:04:00.291 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:00.289+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:00.318 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:00.319 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:00.319 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:00.319 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:00.330 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:04:00.392 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:00.390+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:00.476 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:00.477 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:00.477 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:00.477 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:00.477 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:00.486 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:04:00.493 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:00.489+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:00.595 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:00.595 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:00.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:00.595 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:00.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.595 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:04:00.595 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:00.595 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 154.53460104199999, 2024-09-13T07:04:00.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:04:00.595 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.595 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:00.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.596 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:04:00.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.596 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:00.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:00.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:04:00.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:04:00.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:04:00.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:04:00.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:04:00.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:00.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:04:00.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:04:00.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.601 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.601 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:00.601 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:00.601 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.976817163999996, 2024-09-13T07:04:00.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:04:00.601 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.601 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:00.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:04:00.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:04:00.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:04:00.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:04:00.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:04:00.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:04:00.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:04:00.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:04:00.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:04:00.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:04:00.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:04:00.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:04:00.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:04:00.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.605 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.606 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:00.606 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:00.606 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.976794197000004, 2024-09-13T07:04:00.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:04:00.606 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.606 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:04:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:04:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:04:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:04:00.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.976777018999996, 2024-09-13T07:04:00.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:04:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:04:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:04:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:04:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.976763153999997, 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:04:00.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:04:00.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.976750009, 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.619 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:04:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.976736802000005, 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.623 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:04:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:04:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:00.627 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.976722734999996, 2024-09-13T07:04:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:04:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:04:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:04:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:04:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:04:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:04:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:04:00.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:04:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:04:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.976702720999995, 2024-09-13T07:04:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:04:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:00.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:04:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:04:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:04:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:04:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:04:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:04:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.636 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.976682971000002, 2024-09-13T07:04:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:04:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:04:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:04:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:04:00.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:04:00.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:04:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.976658165000003, 2024-09-13T07:04:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:04:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:04:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:04:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:04:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:04:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:04:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:04:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:04:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.645 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.976633899999996, 2024-09-13T07:04:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:04:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:04:00.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:00.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:04:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:04:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:04:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.650 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.976592167000007, 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:00.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:04:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:04:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:04:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:04:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:04:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:04:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:04:00.654 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.976555877999999, 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:04:00.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.533452546000007, 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:04:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.533435663999995, 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:04:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:04:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:04:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:04:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:04:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:04:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.532385106999996, 2024-09-13T07:04:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:04:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:04:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:04:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:04:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:04:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:04:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:04:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:04:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:04:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:04:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:04:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:04:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.530710557999996, 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:04:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:04:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:04:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:04:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:04:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:04:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:04:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:04:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.010128167999994, 2024-09-13T07:04:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:04:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:04:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:04:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:04:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:04:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:04:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:04:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172880 2.2 2:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e149)", 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.455831922000002, 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.97218948500000002, 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172880 2024-09-13T07:04:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137687+0000", 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137702+0000", 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5023e-05 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137730+0000", 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7719e-05 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137748+0000", 2024-09-13T07:04:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8189000000000002e-05 2024-09-13T07:04:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137891+0000", 2024-09-13T07:04:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014330699999999999 2024-09-13T07:04:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137903+0000", 2024-09-13T07:04:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1688999999999999e-05 2024-09-13T07:04:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142887+0000", 2024-09-13T07:04:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004984006 2024-09-13T07:04:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142898+0000", 2024-09-13T07:04:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.079e-05 2024-09-13T07:04:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109703+0000", 2024-09-13T07:04:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96680553899999999 2024-09-13T07:04:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109879+0000", 2024-09-13T07:04:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017579800000000001 2024-09-13T07:04:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:00.687 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:00.688 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:00.594+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:00.689 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:04:00.695 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:00.694+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:00.796 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:00.793+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:00.841 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:00.841 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:00.851 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:04:00.897 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:00.893+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:00.956 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:00.956 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:00.956 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:00.956 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:00.956 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:00.966 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:04:01.006 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:01.005+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:01.099 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:01.097+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:01.117 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:01.117 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 155.06335397199999, 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:01.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:04:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:04:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:04:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:04:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:04:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:04:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:04:01.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:04:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:04:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:04:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:04:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:01.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:01.122 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 122.905472738, 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:04:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:04:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:04:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:04:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:04:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:04:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:04:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 122.87970302700001, 2024-09-13T07:04:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:04:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.128 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:04:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:04:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:04:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:04:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:04:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:04:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:04:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:04:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:04:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:04:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 122.858551861, 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:01.134 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:01.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:04:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:04:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:04:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:04:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:04:01.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.741507098, 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:04:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:04:01.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:04:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:04:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:04:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:04:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:04:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:04:01.145 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.146 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.146 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.146 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:01.146 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:01.146 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 94.456509853, 2024-09-13T07:04:01.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:04:01.146 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.146 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:01.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.204800715000005, 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.150 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:04:01.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:04:01.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:04:01.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.204782680999998, 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.158 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.159 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:01.159 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:01.159 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.204766601000003, 2024-09-13T07:04:01.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:04:01.159 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.159 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:01.159 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.159 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.159 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.159 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:04:01.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:04:01.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.163 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.163 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.204750924999999, 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:01.167 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.204728003, 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:01.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:01.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:04:01.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:04:01.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:04:01.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:04:01.171 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.171 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.171 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.171 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.204711906, 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:01.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:01.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:04:01.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:04:01.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:04:01.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:04:01.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:04:01.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:04:01.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:04:01.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:04:01.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:04:01.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:04:01.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:04:01.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.176 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.176 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:01.176 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:01.176 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.204110294000003, 2024-09-13T07:04:01.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:04:01.176 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.176 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:04:01.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:04:01.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:04:01.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:04:01.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:04:01.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:04:01.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:04:01.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:04:01.180 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.180 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.180 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.180 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:01.180 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:01.181 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.204067518000002, 2024-09-13T07:04:01.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:04:01.181 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.181 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:01.181 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.181 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.181 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.181 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:04:01.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.181 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.047049748000006, 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:04:01.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:04:01.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:04:01.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:04:01.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:04:01.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.188 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.188 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:04:01.188 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:01.188 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.194672818000001, 2024-09-13T07:04:01.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:04:01.189 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.189 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:01.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.189 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:04:01.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.189 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:01.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:01.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:04:01.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:01.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:04:01.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:04:01.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:04:01.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:04:01.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:164179 3.14 e135/67 3:2fc485bd:::rbd_data.11942e4f5fc7.0000000000000002:head v 135'401, mlcod=132'398)", 2024-09-13T07:04:01.193 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.171732364, 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063492174999999998, 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945792+0000", 2024-09-13T07:04:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954811+0000", 2024-09-13T07:04:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090187940000000001 2024-09-13T07:04:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954840+0000", 2024-09-13T07:04:01.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8659000000000001e-05 2024-09-13T07:04:01.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954852+0000", 2024-09-13T07:04:01.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1966e-05 2024-09-13T07:04:01.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954938+0000", 2024-09-13T07:04:01.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6539000000000006e-05 2024-09-13T07:04:01.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954975+0000", 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6303999999999997e-05 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009206+0000", 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.054231044999999999 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009286+0000", 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700999999999999e-05 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.197 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:165953 3.d 3:b3a20b45:::rbd_data.11942e4f5fc7.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8796117~39260] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.895284805999999, 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.095290105, 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 165953 2024-09-13T07:04:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222240+0000", 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258929+0000", 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.036688401000000002 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258971+0000", 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2212000000000002e-05 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258989+0000", 2024-09-13T07:04:01.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7674000000000001e-05 2024-09-13T07:04:01.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259072+0000", 2024-09-13T07:04:01.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3656e-05 2024-09-13T07:04:01.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:01.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259297+0000", 2024-09-13T07:04:01.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022450700000000001 2024-09-13T07:04:01.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.260463+0000", 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011665340000000001 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.316214+0000", 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.055750268999999998 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317335+0000", 2024-09-13T07:04:01.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011211719999999999 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317451+0000", 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011616 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317532+0000", 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0860999999999997e-05 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.204 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172852 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [call lock.unlock in=52b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-13T07:04:01.204 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:01.204 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.918827659000002, 2024-09-13T07:04:01.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.90390594199999996, 2024-09-13T07:04:01.204 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.204 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:01.204 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.204 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.204 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.204 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172852 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198696+0000", 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198710+0000", 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3548e-05 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198739+0000", 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9606000000000001e-05 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198752+0000", 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3033e-05 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198869+0000", 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116552 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199018+0000", 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014923700000000001 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:01.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199308+0000", 2024-09-13T07:04:01.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028947199999999998 2024-09-13T07:04:01.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:01.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.200731+0000", 2024-09-13T07:04:01.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014229799999999999 2024-09-13T07:04:01.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059443+0000", 2024-09-13T07:04:01.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.85871189999999997 2024-09-13T07:04:01.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:01.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059467+0000", 2024-09-13T07:04:01.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.478e-05 2024-09-13T07:04:01.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102380+0000", 2024-09-13T07:04:01.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042912463999999997 2024-09-13T07:04:01.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102605+0000", 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022489499999999999 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172853 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [watch reconnect cookie 140335574242864 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.067078800000001, 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.056267725999999997, 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:01.210 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172853 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:01.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050445+0000", 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050461+0000", 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5783000000000001e-05 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050497+0000", 2024-09-13T07:04:01.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5995999999999999e-05 2024-09-13T07:04:01.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050516+0000", 2024-09-13T07:04:01.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9579999999999999e-05 2024-09-13T07:04:01.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050637+0000", 2024-09-13T07:04:01.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120536 2024-09-13T07:04:01.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050647+0000", 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6989999999999998e-06 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059533+0000", 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088858749999999997 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059545+0000", 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2473000000000001e-05 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:01.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102962+0000", 2024-09-13T07:04:01.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043416656999999997 2024-09-13T07:04:01.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:01.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.103612+0000", 2024-09-13T07:04:01.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000650374 2024-09-13T07:04:01.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:01.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:01.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:01.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.106715+0000", 2024-09-13T07:04:01.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031033229999999998 2024-09-13T07:04:01.216 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.216 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.216 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.216 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:01.216 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:01.216 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:01.217 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:01.213+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:01.218 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:04:01.318 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:01.318+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:01.332 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:01.333 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:01.333 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:01.333 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:01.345 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:04:01.419 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:01.418+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:01.451 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:01.451 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:01.451 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:01.451 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:01.451 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:01.461 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:04:01.520 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:01.517+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:01.617 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:01.618 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:01.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:01.618 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:01.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.618 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137474 3.6 3:65c31370:::rbd_data.1194af11889b.000000000000013b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 186185~11715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-13T07:04:01.618 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:01.618 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.872480662999997, 2024-09-13T07:04:01.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0054357140000001, 2024-09-13T07:04:01.618 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.618 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:01.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137474 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743934+0000", 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743949+0000", 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5705999999999998e-05 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743960+0000", 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0084e-05 2024-09-13T07:04:01.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743971+0000", 2024-09-13T07:04:01.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0942e-05 2024-09-13T07:04:01.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744045+0000", 2024-09-13T07:04:01.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4626e-05 2024-09-13T07:04:01.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:01.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744062+0000", 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7339000000000001e-05 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746366+0000", 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.002303457 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746520+0000", 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015385900000000001 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746734+0000", 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000214253 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749128+0000", 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002394129 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749267+0000", 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013878299999999999 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749328+0000", 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0918000000000002e-05 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749371+0000", 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3188000000000001e-05 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.624 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163305 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2387968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.625 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:01.625 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.938262182000003, 2024-09-13T07:04:01.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044745869000000001, 2024-09-13T07:04:01.625 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.625 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.625 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163305 2024-09-13T07:04:01.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.625 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:01.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:01.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678153+0000", 2024-09-13T07:04:01.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678156+0000", 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6350000000000002e-06 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678161+0000", 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678166+0000", 2024-09-13T07:04:01.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.498e-06 2024-09-13T07:04:01.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722751+0000", 2024-09-13T07:04:01.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044585134999999998 2024-09-13T07:04:01.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722836+0000", 2024-09-13T07:04:01.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4672999999999994e-05 2024-09-13T07:04:01.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722900+0000", 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4042999999999997e-05 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163307 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.938241718, 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045060944999999998, 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163307 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:01.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678177+0000", 2024-09-13T07:04:01.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:04:01.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678183+0000", 2024-09-13T07:04:01.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0599999999999996e-06 2024-09-13T07:04:01.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678188+0000", 2024-09-13T07:04:01.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:04:01.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722941+0000", 2024-09-13T07:04:01.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044752844999999999 2024-09-13T07:04:01.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723073+0000", 2024-09-13T07:04:01.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013128599999999999 2024-09-13T07:04:01.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723235+0000", 2024-09-13T07:04:01.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000162671 2024-09-13T07:04:01.633 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.633 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.633 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163310 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 544768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.938222766000003, 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045290534, 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163310 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:01.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:01.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:01.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678195+0000", 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.785e-06 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678201+0000", 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678205+0000", 2024-09-13T07:04:01.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6999999999999999e-06 2024-09-13T07:04:01.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723299+0000", 2024-09-13T07:04:01.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045093964 2024-09-13T07:04:01.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723384+0000", 2024-09-13T07:04:01.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4562999999999997e-05 2024-09-13T07:04:01.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723484+0000", 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100045 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163313 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1093632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.938206780999998, 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045562975999999998, 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.638 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163313 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:01.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:01.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678212+0000", 2024-09-13T07:04:01.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8650000000000001e-06 2024-09-13T07:04:01.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678217+0000", 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2660000000000001e-06 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678221+0000", 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-06 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723552+0000", 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045330489000000002 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723660+0000", 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010770800000000001 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723772+0000", 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112608 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.642 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163317 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1642496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.643 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:01.643 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.938177973000002, 2024-09-13T07:04:01.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045770165000000002, 2024-09-13T07:04:01.643 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.643 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.643 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.643 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.643 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.643 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163317 2024-09-13T07:04:01.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.643 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:01.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:01.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678237+0000", 2024-09-13T07:04:01.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678241+0000", 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2669999999999998e-06 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678247+0000", 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7309999999999996e-06 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678252+0000", 2024-09-13T07:04:01.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-13T07:04:01.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723830+0000", 2024-09-13T07:04:01.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045577234000000001 2024-09-13T07:04:01.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723936+0000", 2024-09-13T07:04:01.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106277 2024-09-13T07:04:01.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724008+0000", 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2422999999999994e-05 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163320 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.938157549000003, 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045968883000000002, 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.647 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.648 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.648 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.648 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163320 2024-09-13T07:04:01.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.648 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:01.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678261+0000", 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8090000000000001e-06 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678267+0000", 2024-09-13T07:04:01.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8860000000000001e-06 2024-09-13T07:04:01.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678271+0000", 2024-09-13T07:04:01.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4549999999999997e-06 2024-09-13T07:04:01.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724061+0000", 2024-09-13T07:04:01.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045789656999999997 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724151+0000", 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9734999999999994e-05 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724227+0000", 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6848999999999996e-05 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.652 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163324 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.652 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:01.652 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.937873216, 2024-09-13T07:04:01.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045988788000000003, 2024-09-13T07:04:01.652 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.652 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.652 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.652 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.652 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.652 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163324 2024-09-13T07:04:01.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.652 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678542+0000", 2024-09-13T07:04:01.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678547+0000", 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6829999999999999e-06 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678556+0000", 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1929999999999999e-06 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678562+0000", 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.083e-06 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724267+0000", 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045705427 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724417+0000", 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014984699999999999 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724532+0000", 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011430900000000001 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163327 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 733184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.937848182000003, 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046185902000000001, 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.656 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163327 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:01.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678567+0000", 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678571+0000", 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3799999999999998e-06 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678576+0000", 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7760000000000003e-06 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678581+0000", 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8640000000000004e-06 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724561+0000", 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045979178000000002 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724649+0000", 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8677000000000005e-05 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724754+0000", 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104527 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163331 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.660 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.937830028, 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046361773000000002, 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163331 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:01.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678585+0000", 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678588+0000", 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0469999999999998e-06 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678595+0000", 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0399999999999998e-06 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678599+0000", 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724780+0000", 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046181168000000002 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724871+0000", 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0562000000000005e-05 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724948+0000", 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6985999999999998e-05 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.664 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.665 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163334 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1662976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.665 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:01.665 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.937812825000002, 2024-09-13T07:04:01.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046544502000000001, 2024-09-13T07:04:01.665 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.665 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.665 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.665 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.665 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.665 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163334 2024-09-13T07:04:01.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:01.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678605+0000", 2024-09-13T07:04:01.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5380000000000001e-06 2024-09-13T07:04:01.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678611+0000", 2024-09-13T07:04:01.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7389999999999996e-06 2024-09-13T07:04:01.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678615+0000", 2024-09-13T07:04:01.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-13T07:04:01.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724987+0000", 2024-09-13T07:04:01.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046371969999999998 2024-09-13T07:04:01.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725075+0000", 2024-09-13T07:04:01.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8091999999999996e-05 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725148+0000", 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2206999999999995e-05 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163338 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.937796095000003, 2024-09-13T07:04:01.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046738452999999999, 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163338 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:01.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678619+0000", 2024-09-13T07:04:01.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678622+0000", 2024-09-13T07:04:01.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.649e-06 2024-09-13T07:04:01.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678628+0000", 2024-09-13T07:04:01.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5029999999999997e-06 2024-09-13T07:04:01.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678632+0000", 2024-09-13T07:04:01.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4020000000000004e-06 2024-09-13T07:04:01.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725182+0000", 2024-09-13T07:04:01.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046550389999999997 2024-09-13T07:04:01.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725280+0000", 2024-09-13T07:04:01.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7260000000000001e-05 2024-09-13T07:04:01.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725358+0000", 2024-09-13T07:04:01.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8720000000000005e-05 2024-09-13T07:04:01.674 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.674 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.674 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.674 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163339 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2363392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.674 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:01.674 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.937780566999997, 2024-09-13T07:04:01.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046920835000000001, 2024-09-13T07:04:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163339 2024-09-13T07:04:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:01.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:01.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678637+0000", 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2479999999999999e-06 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678642+0000", 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0030000000000002e-06 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678646+0000", 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9870000000000001e-06 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725397+0000", 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046750127000000002 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725482+0000", 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5928999999999997e-05 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725556+0000", 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3823000000000001e-05 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.678 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.679 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.679 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163342 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.679 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:01.679 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.937765927999997, 2024-09-13T07:04:01.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047212991000000003, 2024-09-13T07:04:01.679 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.679 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.679 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.679 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.679 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.679 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163342 2024-09-13T07:04:01.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.680 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:01.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:01.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678652+0000", 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.486e-06 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678657+0000", 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6909999999999996e-06 2024-09-13T07:04:01.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678660+0000", 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8009999999999999e-06 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725597+0000", 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046937372999999998 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725745+0000", 2024-09-13T07:04:01.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000148314 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725863+0000", 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011768 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163345 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.936904364999997, 2024-09-13T07:04:01.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046569193000000002, 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163345 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678662+0000", 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9991498 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679516+0000", 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000854789 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679530+0000", 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3492999999999999e-05 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679537+0000", 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.323e-06 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725892+0000", 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046354244000000003 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725975+0000", 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3356000000000006e-05 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726081+0000", 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105992 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.687 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163349 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1298432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.688 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:01.688 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.936872301999998, 2024-09-13T07:04:01.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046744247000000003, 2024-09-13T07:04:01.688 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.688 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.688 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.688 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.688 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.688 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163349 2024-09-13T07:04:01.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.688 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:01.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:01.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679543+0000", 2024-09-13T07:04:01.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679546+0000", 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679551+0000", 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679554+0000", 2024-09-13T07:04:01.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8629999999999999e-06 2024-09-13T07:04:01.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726117+0000", 2024-09-13T07:04:01.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046563081999999999 2024-09-13T07:04:01.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726213+0000", 2024-09-13T07:04:01.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6207999999999998e-05 2024-09-13T07:04:01.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726288+0000", 2024-09-13T07:04:01.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4741999999999997e-05 2024-09-13T07:04:01.692 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.692 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.692 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.692 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163350 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1449984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.692 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:01.692 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.936859675999997, 2024-09-13T07:04:01.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046935635000000003, 2024-09-13T07:04:01.692 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.692 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163350 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679558+0000", 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.928e-06 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.694 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679561+0000", 2024-09-13T07:04:01.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7659999999999999e-06 2024-09-13T07:04:01.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679564+0000", 2024-09-13T07:04:01.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4960000000000001e-06 2024-09-13T07:04:01.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.695 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.695 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.695 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726327+0000", 2024-09-13T07:04:01.695 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046762321000000003 2024-09-13T07:04:01.695 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726415+0000", 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8487999999999997e-05 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726492+0000", 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6995000000000004e-05 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.696 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.697 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163353 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.697 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:01.697 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.936849567000003, 2024-09-13T07:04:01.697 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047130755000000003, 2024-09-13T07:04:01.697 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.697 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.697 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.697 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.697 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.697 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163353 2024-09-13T07:04:01.697 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.697 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.697 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.698 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679568+0000", 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5969999999999999e-06 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679571+0000", 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679573+0000", 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:04:01.699 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726523+0000", 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046949882999999998 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726616+0000", 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3236e-05 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726697+0000", 2024-09-13T07:04:01.700 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0834000000000006e-05 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163356 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2379776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.936839884999998, 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047304662999999997, 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.701 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163356 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.702 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:01.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:01.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679578+0000", 2024-09-13T07:04:01.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:04:01.703 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.703 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.703 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.703 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679582+0000", 2024-09-13T07:04:01.703 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:04:01.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679589+0000", 2024-09-13T07:04:01.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1149999999999996e-06 2024-09-13T07:04:01.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.704 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726723+0000", 2024-09-13T07:04:01.704 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047133768999999999 2024-09-13T07:04:01.704 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.704 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.704 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726809+0000", 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5402000000000007e-05 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726881+0000", 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2152999999999998e-05 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:166663 3.b 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8129196~12979] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:01.705 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:01.706 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.218465371000001, 2024-09-13T07:04:01.706 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.061564718999999997, 2024-09-13T07:04:01.706 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:01.706 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:01.706 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:01.706 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:01.706 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:01.706 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 166663 2024-09-13T07:04:01.706 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:01.707 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397947+0000", 2024-09-13T07:04:01.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999967 2024-09-13T07:04:01.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:01.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412144+0000", 2024-09-13T07:04:01.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014196972 2024-09-13T07:04:01.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.708 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:01.708 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412172+0000", 2024-09-13T07:04:01.708 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7781999999999999e-05 2024-09-13T07:04:01.708 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.708 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412182+0000", 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6069999999999997e-06 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412239+0000", 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.732e-05 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412365+0000", 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125356 2024-09-13T07:04:01.709 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:01.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.413178+0000", 2024-09-13T07:04:01.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081355299999999995 2024-09-13T07:04:01.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.710 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.710 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:01.710 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.443176+0000", 2024-09-13T07:04:01.710 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029997903999999999 2024-09-13T07:04:01.710 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:01.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459446+0000", 2024-09-13T07:04:01.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016269921999999999 2024-09-13T07:04:01.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:01.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459484+0000", 2024-09-13T07:04:01.711 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8488000000000001e-05 2024-09-13T07:04:01.711 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:01.711 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:01.711 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:01.711 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459515+0000", 2024-09-13T07:04:01.712 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0981e-05 2024-09-13T07:04:01.712 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.712 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.712 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.712 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:01.712 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:01.712 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:01.713 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:04:01.714 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:01.714+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:01.814 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:01.813+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:01.828 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:01.828 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:01.828 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:01.829 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:01.838 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:04:01.915 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:01.913+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:01.940 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:04:01.944 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:01.944 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:01.945 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:01.945 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:01.945 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:01.954 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:04:02.016 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:02.014+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:02.080 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:02.101 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:02.101 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:02.101 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:02.110 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:04:02.116 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:02.114+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:02.119 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:02.140 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:02.162 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:02.185 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:02.206 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:02.206 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:02.209 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:02.213 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:02.214 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:02.217 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:04:02.217 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:02.213+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:02.250 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:04:02.318 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:02.317+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:02.378 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:02.378 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:02.388 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:04:02.419 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:02.418+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:02.435 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 7, 2, 0, 1] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-13T07:04:02.435 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:04:02.435 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T07:04:02.435 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T07:04:02.488 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:02.488 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:02.498 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:04:02.519 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:02.517+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:02.597 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:02.597 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:02.606 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:04:02.619 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:02.618+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:02.710 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:02.710 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:02.720 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:04:02.721 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:02.718+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:04:02.788 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "epoch": 167, 2024-09-13T07:04:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "fsid": "25bb30d1-9541-4e40-8659-18d83bb1bd8a", 2024-09-13T07:04:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "created": "2024-09-13T07:00:49.323883+0000", 2024-09-13T07:04:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "modified": "2024-09-13T07:04:02.227409+0000", 2024-09-13T07:04:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "last_up_change": "2024-09-13T07:03:44.039513+0000", 2024-09-13T07:04:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "last_in_change": "2024-09-13T07:03:32.957795+0000", 2024-09-13T07:04:02.788 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_num": 5804032, 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_set": [ 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "nodeep-scrub", 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "pglog_hardlimit", 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "purged_snapdirs", 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_deletes", 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "sortbitwise" 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_version": 4, 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_snapshot": "", 2024-09-13T07:04:02.789 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_max": 3, 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "max_osd": 8, 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "require_min_compat_client": "luminous", 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "min_compat_client": "luminous", 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "require_osd_release": "reef", 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "allow_crimson": false, 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "pools": [ 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 1, 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": ".mgr", 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:00.675365+0000", 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 1, 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool", 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:04:02.790 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 1, 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 1, 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 1, 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 1, 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 1, 2024-09-13T07:04:02.791 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "16", 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:04:02.792 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 0, 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 0, 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:04:02.793 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:04:02.794 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "options": { 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_max": 32, 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_min": 1 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": {} 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 8, 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 8, 2024-09-13T07:04:02.795 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.25, 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 2, 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 2, 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 0.92424774169921875, 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 0.92000001668930054, 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 0.92000001668930054 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 2, 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": "rbd", 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:07.404263+0000", 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 8193, 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:04:02.796 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 8, 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 8, 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 8, 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 8, 2024-09-13T07:04:02.797 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 8, 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "20", 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:04:02.798 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 3, 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 20, 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:04:02.799 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:04:02.800 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "options": {}, 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd": {} 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 4, 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 4, 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.75, 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 3, 2024-09-13T07:04:02.801 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 3, 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 0.92424774169921875, 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 0.92000001668930054, 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 0.92000001668930054 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 3, 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_name": "pool_client.0", 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "create_time": "2024-09-13T07:01:17.321011+0000", 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "flags": 8193, 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "type": 1, 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 2, 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "min_size": 1, 2024-09-13T07:04:02.802 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_rule": 0, 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_count": 0, 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_target": 0, 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "object_hash": 2, 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_autoscale_mode": "off", 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num": 32, 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num": 32, 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_placement_num_target": 32, 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_target": 32, 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_num_pending": 32, 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "last_pg_merge_meta": { 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "source_pgid": "0.0", 2024-09-13T07:04:02.803 INFO:teuthology.orchestra.run.smithi118.stdout: "ready_epoch": 0, 2024-09-13T07:04:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_started": 0, 2024-09-13T07:04:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "last_epoch_clean": 0, 2024-09-13T07:04:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "source_version": "0'0", 2024-09-13T07:04:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "target_version": "0'0" 2024-09-13T07:04:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "last_change": "167", 2024-09-13T07:04:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend": "0", 2024-09-13T07:04:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T07:04:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T07:04:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "auid": 0, 2024-09-13T07:04:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_mode": "selfmanaged", 2024-09-13T07:04:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_seq": 77, 2024-09-13T07:04:02.804 INFO:teuthology.orchestra.run.smithi118.stdout: "snap_epoch": 167, 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "pool_snaps": [], 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps": "[]", 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_bytes": 0, 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "quota_max_objects": 0, 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "tiers": [], 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "tier_of": -1, 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "read_tier": -1, 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "write_tier": -1, 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_mode": "none", 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_bytes": 0, 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "target_max_objects": 0, 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T07:04:02.805 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_flush_age": 0, 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "cache_min_evict_age": 0, 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profile": "", 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_params": { 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "none" 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_period": 0, 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_count": 0, 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "use_gmt_hitset": true, 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "min_read_recency_for_promote": 0, 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "min_write_recency_for_promote": 0, 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "hit_set_search_last_n": 0, 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "grade_table": [], 2024-09-13T07:04:02.806 INFO:teuthology.orchestra.run.smithi118.stdout: "stripe_width": 0, 2024-09-13T07:04:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "expected_num_objects": 0, 2024-09-13T07:04:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "fast_read": false, 2024-09-13T07:04:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "options": {}, 2024-09-13T07:04:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "application_metadata": { 2024-09-13T07:04:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "rbd": {} 2024-09-13T07:04:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "read_balance": { 2024-09-13T07:04:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "score_acting": 2.25, 2024-09-13T07:04:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "score_stable": 2.25, 2024-09-13T07:04:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "optimal_score": 0.75, 2024-09-13T07:04:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_acting": 1.690000057220459, 2024-09-13T07:04:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "raw_score_stable": 1.690000057220459, 2024-09-13T07:04:02.807 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity_weighted": 0.92424774169921875, 2024-09-13T07:04:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity": 0.92000001668930054, 2024-09-13T07:04:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "average_primary_affinity_weighted": 0.92000001668930054 2024-09-13T07:04:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:04:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "osds": [ 2024-09-13T07:04:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 0, 2024-09-13T07:04:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "21736eee-00cd-4eef-9725-eca7776957b4", 2024-09-13T07:04:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:04:02.808 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:04:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.826507568359375, 2024-09-13T07:04:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:04:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:04:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:04:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:04:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 149, 2024-09-13T07:04:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:04:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:04:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:04:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.809 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6817", 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6819", 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6821", 2024-09-13T07:04:02.810 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6823", 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6829", 2024-09-13T07:04:02.811 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6831", 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6825", 2024-09-13T07:04:02.812 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6827", 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 3731386737 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6819/3731386737", 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6823/3731386737", 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6831/3731386737", 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6827/3731386737", 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:04:02.813 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 1, 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "05818db2-841d-406d-be17-8fbdab4fff69", 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.1844329833984375, 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 84, 2024-09-13T07:04:02.814 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6816", 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6818", 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.815 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6820", 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6822", 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.816 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6828", 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6830", 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.817 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6824", 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6826", 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 876310814 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.818 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6818/876310814", 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6822/876310814", 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6830/876310814", 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6826/876310814", 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 2, 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "2387761c-42d2-481b-b5b5-b16d3a943b22", 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:04:02.819 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 148, 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6808", 2024-09-13T07:04:02.820 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6809", 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6810", 2024-09-13T07:04:02.821 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:04:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6811", 2024-09-13T07:04:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:04:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:04:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.822 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6814", 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6815", 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.823 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6812", 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6813", 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 690188762 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6809/690188762", 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6811/690188762", 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6815/690188762", 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6813/690188762", 2024-09-13T07:04:02.824 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:04:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:04:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:04:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 3, 2024-09-13T07:04:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "56500e90-f76b-47c8-a649-893400359c53", 2024-09-13T07:04:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:04:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:04:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.8596343994140625, 2024-09-13T07:04:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 0.39398193359375, 2024-09-13T07:04:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:04:02.825 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 148, 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6800", 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.826 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6801", 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6802", 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.827 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6803", 2024-09-13T07:04:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:04:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:04:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6806", 2024-09-13T07:04:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:04:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.828 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6807", 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6804", 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.829 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.118:6805", 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2198175481 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.118:6801/2198175481", 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.118:6803/2198175481", 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.118:6807/2198175481", 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.118:6805/2198175481", 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 4, 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "f594a612-aa07-4565-a09e-08368cf397b6", 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.9017486572265625, 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 149, 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:04:02.831 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:04:02.832 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.832 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.832 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6810", 2024-09-13T07:04:02.832 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:04:02.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.832 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.832 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6813", 2024-09-13T07:04:02.832 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:04:02.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.832 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6817", 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6819", 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.834 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:04:02.834 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.834 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.834 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6827", 2024-09-13T07:04:02.834 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:04:02.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.834 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.834 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6830", 2024-09-13T07:04:02.834 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:04:02.834 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.834 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6822", 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6825", 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1835601651 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.836 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6813/1835601651", 2024-09-13T07:04:02.836 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6819/1835601651", 2024-09-13T07:04:02.836 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6830/1835601651", 2024-09-13T07:04:02.836 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6825/1835601651", 2024-09-13T07:04:02.836 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:04:02.836 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:04:02.836 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:04:02.836 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.836 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 5, 2024-09-13T07:04:02.836 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "1013b81f-e6b4-430b-938d-88759d43b881", 2024-09-13T07:04:02.836 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 0, 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 0.95001220703125, 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 115, 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 148, 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.837 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6800", 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6801", 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.838 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6802", 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6803", 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.839 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6806", 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6807", 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.840 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6804", 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6805", 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2853696331 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6801/2853696331", 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6803/2853696331", 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6807/2853696331", 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6805/2853696331", 2024-09-13T07:04:02.841 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: "exists" 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 6, 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "367f5f5b-5919-4534-b88c-229f327e4950", 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:04:02.842 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 149, 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6808", 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6812", 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:04:02.843 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6815", 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6818", 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:04:02.844 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6828", 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6831", 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:04:02.845 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6821", 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6824", 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 131309660 2024-09-13T07:04:02.846 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6812/131309660", 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6818/131309660", 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6831/131309660", 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6824/131309660", 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 7, 2024-09-13T07:04:02.847 INFO:teuthology.orchestra.run.smithi118.stdout: "uuid": "90d9e486-e1ae-4305-80a6-6f483e534643", 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: "up": 1, 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: "in": 1, 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: "weight": 1, 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_affinity": 1, 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_begin": 0, 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: "last_clean_end": 0, 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: "up_from": 13, 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: "up_thru": 148, 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: "down_at": 0, 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: "lost_at": 0, 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addrs": { 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.848 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.849 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6809", 2024-09-13T07:04:02.849 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:04:02.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.849 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.849 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6811", 2024-09-13T07:04:02.849 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:04:02.849 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.849 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.849 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addrs": { 2024-09-13T07:04:02.849 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6814", 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6816", 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addrs": { 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6826", 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6829", 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addrs": { 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: "addrvec": [ 2024-09-13T07:04:02.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.852 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v2", 2024-09-13T07:04:02.852 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6820", 2024-09-13T07:04:02.852 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:04:02.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.852 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-09-13T07:04:02.852 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "172.21.15.164:6823", 2024-09-13T07:04:02.852 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 652173699 2024-09-13T07:04:02.852 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.852 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.852 INFO:teuthology.orchestra.run.smithi118.stdout: "public_addr": "172.21.15.164:6811/652173699", 2024-09-13T07:04:02.852 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_addr": "172.21.15.164:6816/652173699", 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_back_addr": "172.21.15.164:6829/652173699", 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: "heartbeat_front_addr": "172.21.15.164:6823/652173699", 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: "state": [ 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: "exists", 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: "up" 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_xinfo": [ 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 0, 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:04:02.853 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.059233+0000", 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 1, 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.083130+0000", 2024-09-13T07:04:02.854 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 2, 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.848896+0000", 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 3, 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:04:02.855 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.847456+0000", 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 4, 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:04:02.856 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.914135+0000", 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 5, 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:56.834284+0000", 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 148 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.857 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 6, 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.003715+0000", 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": 7, 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: "down_stamp": "0.000000", 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_probability": 0, 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: "laggy_interval": 0, 2024-09-13T07:04:02.858 INFO:teuthology.orchestra.run.smithi118.stdout: "features": 4540138322906710015, 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: "old_weight": 0, 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: "last_purged_snaps_scrub": "2024-09-13T07:00:57.033514+0000", 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: "dead_epoch": 0 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap": [ 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: "pgid": "2.5", 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: "osds": [ 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: 6, 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: 4 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:04:02.859 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap_items": [], 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_upmap_primaries": [], 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: "pg_temp": [], 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: "primary_temp": [], 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: "blocklist": {}, 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: "range_blocklist": {}, 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: "erasure_code_profiles": { 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: "default": { 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: "crush-failure-domain": "osd", 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: "k": "2", 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: "m": "1", 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: "plugin": "jerasure", 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: "technique": "reed_sol_van" 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: "removed_snaps_queue": [ 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 3, 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: "snaps": [ 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: "begin": 30, 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: "length": 17 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: "new_removed_snaps": [ 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.861 INFO:teuthology.orchestra.run.smithi118.stdout: "pool": 3, 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: "snaps": [ 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: "begin": 30, 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: "length": 1 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: ], 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: "new_purged_snaps": [], 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: "crush_node_flags": {}, 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: "device_class_flags": {}, 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode": { 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode_enabled": false, 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_bucket_count": 0, 2024-09-13T07:04:02.862 INFO:teuthology.orchestra.run.smithi118.stdout: "degraded_stretch_mode": 0, 2024-09-13T07:04:02.863 INFO:teuthology.orchestra.run.smithi118.stdout: "recovering_stretch_mode": 0, 2024-09-13T07:04:02.863 INFO:teuthology.orchestra.run.smithi118.stdout: "stretch_mode_bucket": 0 2024-09-13T07:04:02.863 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:02.863 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:02.865 INFO:tasks.thrashosds.thrasher:j is {'epoch': 167, 'fsid': '25bb30d1-9541-4e40-8659-18d83bb1bd8a', 'created': '2024-09-13T07:00:49.323883+0000', 'modified': '2024-09-13T07:04:02.227409+0000', 'last_up_change': '2024-09-13T07:03:44.039513+0000', 'last_in_change': '2024-09-13T07:03:32.957795+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T07:01:00.675365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9242477416992188, 'average_primary_affinity': 0.9200000166893005, 'average_primary_affinity_weighted': 0.9200000166893005}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T07:01:07.404263+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.75, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9242477416992188, 'average_primary_affinity': 0.9200000166893005, 'average_primary_affinity_weighted': 0.9200000166893005}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T07:01:17.321011+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 77, 'snap_epoch': 167, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 0.75, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9242477416992188, 'average_primary_affinity': 0.9200000166893005, 'average_primary_affinity_weighted': 0.9200000166893005}}], 'osds': [{'osd': 0, 'uuid': '21736eee-00cd-4eef-9725-eca7776957b4', 'up': 1, 'in': 1, 'weight': 0.826507568359375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 149, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6817', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6819', 'nonce': 3731386737}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6821', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6823', 'nonce': 3731386737}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6829', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6831', 'nonce': 3731386737}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6825', 'nonce': 3731386737}, {'type': 'v1', 'addr': '172.21.15.118:6827', 'nonce': 3731386737}]}, 'public_addr': '172.21.15.118:6819/3731386737', 'cluster_addr': '172.21.15.118:6823/3731386737', 'heartbeat_back_addr': '172.21.15.118:6831/3731386737', 'heartbeat_front_addr': '172.21.15.118:6827/3731386737', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '05818db2-841d-406d-be17-8fbdab4fff69', 'up': 1, 'in': 1, 'weight': 0.1844329833984375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6816', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6818', 'nonce': 876310814}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6820', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6822', 'nonce': 876310814}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6828', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6830', 'nonce': 876310814}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6824', 'nonce': 876310814}, {'type': 'v1', 'addr': '172.21.15.118:6826', 'nonce': 876310814}]}, 'public_addr': '172.21.15.118:6818/876310814', 'cluster_addr': '172.21.15.118:6822/876310814', 'heartbeat_back_addr': '172.21.15.118:6830/876310814', 'heartbeat_front_addr': '172.21.15.118:6826/876310814', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2387761c-42d2-481b-b5b5-b16d3a943b22', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 148, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6808', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6809', 'nonce': 690188762}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6810', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6811', 'nonce': 690188762}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6814', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6815', 'nonce': 690188762}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6812', 'nonce': 690188762}, {'type': 'v1', 'addr': '172.21.15.118:6813', 'nonce': 690188762}]}, 'public_addr': '172.21.15.118:6809/690188762', 'cluster_addr': '172.21.15.118:6811/690188762', 'heartbeat_back_addr': '172.21.15.118:6815/690188762', 'heartbeat_front_addr': '172.21.15.118:6813/690188762', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '56500e90-f76b-47c8-a649-893400359c53', 'up': 1, 'in': 1, 'weight': 0.8596343994140625, 'primary_affinity': 0.39398193359375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 148, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6800', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6801', 'nonce': 2198175481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6802', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6803', 'nonce': 2198175481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6806', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6807', 'nonce': 2198175481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6804', 'nonce': 2198175481}, {'type': 'v1', 'addr': '172.21.15.118:6805', 'nonce': 2198175481}]}, 'public_addr': '172.21.15.118:6801/2198175481', 'cluster_addr': '172.21.15.118:6803/2198175481', 'heartbeat_back_addr': '172.21.15.118:6807/2198175481', 'heartbeat_front_addr': '172.21.15.118:6805/2198175481', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f594a612-aa07-4565-a09e-08368cf397b6', 'up': 1, 'in': 1, 'weight': 0.9017486572265625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 149, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 1835601651}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6817', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 1835601651}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6827', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6830', 'nonce': 1835601651}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 1835601651}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 1835601651}]}, 'public_addr': '172.21.15.164:6813/1835601651', 'cluster_addr': '172.21.15.164:6819/1835601651', 'heartbeat_back_addr': '172.21.15.164:6830/1835601651', 'heartbeat_front_addr': '172.21.15.164:6825/1835601651', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1013b81f-e6b4-430b-938d-88759d43b881', 'up': 0, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 115, 'down_at': 148, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 2853696331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 2853696331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 2853696331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 2853696331}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 2853696331}]}, 'public_addr': '172.21.15.164:6801/2853696331', 'cluster_addr': '172.21.15.164:6803/2853696331', 'heartbeat_back_addr': '172.21.15.164:6807/2853696331', 'heartbeat_front_addr': '172.21.15.164:6805/2853696331', 'state': ['exists']}, {'osd': 6, 'uuid': '367f5f5b-5919-4534-b88c-229f327e4950', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 149, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6812', 'nonce': 131309660}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6815', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6818', 'nonce': 131309660}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 131309660}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6821', 'nonce': 131309660}, {'type': 'v1', 'addr': '172.21.15.164:6824', 'nonce': 131309660}]}, 'public_addr': '172.21.15.164:6812/131309660', 'cluster_addr': '172.21.15.164:6818/131309660', 'heartbeat_back_addr': '172.21.15.164:6831/131309660', 'heartbeat_front_addr': '172.21.15.164:6824/131309660', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '90d9e486-e1ae-4305-80a6-6f483e534643', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 148, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6809', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 652173699}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6816', 'nonce': 652173699}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 652173699}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 652173699}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 652173699}]}, 'public_addr': '172.21.15.164:6811/652173699', 'cluster_addr': '172.21.15.164:6816/652173699', 'heartbeat_back_addr': '172.21.15.164:6829/652173699', 'heartbeat_front_addr': '172.21.15.164:6823/652173699', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.059233+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.083130+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.848896+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.847456+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.914135+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:56.834284+0000', 'dead_epoch': 148}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.003715+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T07:00:57.033514+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.5', 'osds': [6, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 30, 'length': 17}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 30, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T07:04:02.865 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 2.5 2024-09-13T07:04:02.865 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 2.5 2024-09-13T07:04:02.866 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:02.866 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:02.866 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:04:02.867 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:02.865+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:02.965 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:02.966+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:02.984 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:02.985 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:02.995 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:04:03.066 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:03.066+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:03.106 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:03.106 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:03.116 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:04:03.167 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:03.165+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:03.225 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:03.225 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:03.234 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:04:03.255 INFO:teuthology.orchestra.run.smithi118.stderr:clear 2.5 pg_upmap mapping 2024-09-13T07:04:03.256 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:04:03.268 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:03.266+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:03.343 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:03.344 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:03.353 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:04:03.369 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:03.370+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:03.462 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:03.462 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:03.469 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:03.470+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:03.472 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:04:03.571 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:03.570+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:03.576 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:03.576 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:03.586 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:04:03.672 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:03.670+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:03.695 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:03.696 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:03.705 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:04:03.773 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:03.769+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:03.813 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:03.814 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:03.823 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:04:03.873 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:03.874+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:03.930 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:03.931 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:03.940 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:04:03.974 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:03.973+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:04.053 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:04.053 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:04.063 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:04:04.075 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:04.073+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:04.170 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:04.170 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:04.175 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:04.174+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:04.179 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:04:04.214 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:04:04.275 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:04.274+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:04.290 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:04.290 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:04.301 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:04:04.372 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:04.376 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:04.374+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:04.399 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:04.415 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:04.415 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:04.415 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:04.415 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:04.422 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:04.428 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:04:04.447 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:04.478 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:04.478+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:04.482 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:04.518 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:04.554 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:04.554 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:04.554 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:04.554 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:04.554 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:04.554 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:04.554 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:04.554 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:04.554 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:04.556 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:04.556 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:04.557 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:04.558 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:04.559 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:04.566 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:04:04.578 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:04.577+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:04.679 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:04.677+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:04.726 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:04.726 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:04.726 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:04.726 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:04.726 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.726 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:04:04.727 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:04.727 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.23186768900001, 2024-09-13T07:04:04.727 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:04:04.727 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.727 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:04.727 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.727 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.727 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.727 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:04:04.727 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.727 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.727 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.727 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.728 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:04:04.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:04:04.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:04:04.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:04:04.729 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.729 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.729 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.729 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:04:04.729 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.730 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.731 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.732 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.732 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:04.732 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.745384125, 2024-09-13T07:04:04.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:04:04.732 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.732 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:04.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.732 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:04:04.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.732 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:04.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:04:04.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:04:04.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:04:04.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:04:04.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:04:04.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:04:04.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:04:04.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:04:04.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:04.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:04:04.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:04:04.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:04:04.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:04:04.736 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.736 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.736 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.736 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.737 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:04.737 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.745314196, 2024-09-13T07:04:04.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:04:04.737 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.737 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:04.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.737 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.737 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:04:04.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.737 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.737 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.740 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.741 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.741 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.741 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:04.741 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.745298324, 2024-09-13T07:04:04.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:04:04.741 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.741 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:04.741 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.741 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.741 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.741 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.742 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:04:04.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:04:04.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:04:04.745 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.745 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.745 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.745 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.745 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:04.745 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.745227867, 2024-09-13T07:04:04.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:04:04.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.74187582899999, 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:04:04.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.74185952800001, 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:04:04.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:04:04.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:04:04.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:04:04.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:04:04.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:04:04.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.741839743, 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:04:04.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.74182347, 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:04.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:04.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:04:04.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:04:04.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:04:04.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:04:04.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:04:04.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.765 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.741805645, 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.766 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:04:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:04:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:04:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:04:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:04:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:04:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:04:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:04:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:04:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:04:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.741789552, 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:04:04.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:04:04.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.773 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.741768366, 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.774 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:04:04.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:04:04.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:04:04.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:04:04.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:04:04.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:04:04.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:04:04.777 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.777 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.74175233299999, 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.778 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:04:04.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.741703094, 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:04:04.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:04:04.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:04:04.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:04:04.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:04:04.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.786 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.786 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:04.786 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.741685034, 2024-09-13T07:04:04.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:04:04.786 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.786 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.786 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:04:04.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.786 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:04.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:04.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:04.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:04:04.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:04:04.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:04:04.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:04:04.790 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.790 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.790 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.790 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.790 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:04.790 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.741668519, 2024-09-13T07:04:04.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:04:04.790 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.790 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:04.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.741650154, 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.794 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:04.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:04:04.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:04:04.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:04:04.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:04:04.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:04:04.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:04:04.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:04:04.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:04:04.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:04:04.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:04:04.798 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.798 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.798 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.798 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.798 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:04.798 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.741634056, 2024-09-13T07:04:04.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:04:04.798 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.741618121, 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.802 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.803 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.803 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.803 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.803 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:04:04.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.803 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:04.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:04:04.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 126.741601624, 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:04:04.806 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:04.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:04.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:04.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:04.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:04.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:04.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:04.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:04:04.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:04:04.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:04:04.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:04:04.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:04.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:04.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:04.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:04:04.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:04:04.810 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.810 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.810 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.810 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:04.810 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:04.810 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:04.812 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:04:04.812 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:04.810+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:04.912 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:04.910+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:04.925 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:04.925 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:04.925 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:04.925 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:04.935 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:04:05.013 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:05.013+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:05.041 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:05.041 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:05.041 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:05.041 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:05.042 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:05.051 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:04:05.115 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:05.113+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:05.162 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:05.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 159.10437992300001, 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.166 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 75.286163565999999, 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:04:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:04:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:04:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:04:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:04:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:04:05.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:04:05.170 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 75.286114178000005, 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.171 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:04:05.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:04:05.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:04:05.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 75.286091471000006, 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.175 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:04:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:04:05.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:04:05.178 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 75.286047277999998, 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.179 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:04:05.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:04:05.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:04:05.182 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 75.255068188999999, 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.183 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:04:05.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:04:05.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137457 3.18 3:1a596dcc:::rbd_data.1194af11889b.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 60.509446379000003, 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033734403000000003, 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.187 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137457 2024-09-13T07:04:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651523+0000", 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651531+0000", 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.328e-06 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651545+0000", 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4270000000000001e-05 2024-09-13T07:04:05.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651555+0000", 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5210000000000008e-06 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651626+0000", 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1915999999999994e-05 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651761+0000", 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000134838 2024-09-13T07:04:05.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.658577+0000", 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068151820000000004 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685181+0000", 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026604200000000001 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685193+0000", 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2128e-05 2024-09-13T07:04:05.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685226+0000", 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3504999999999999e-05 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685259+0000", 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2866e-05 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.192 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138565 3.0 3:002d0425:::rbd_data.11949f62878b.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.496075887000003, 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033394489999999999, 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138565 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:05.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664893+0000", 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664906+0000", 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2996000000000001e-05 2024-09-13T07:04:05.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664936+0000", 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9609999999999999e-05 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664949+0000", 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383e-05 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665046+0000", 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6459000000000006e-05 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665259+0000", 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000213141 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.672522+0000", 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072632110000000003 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.697982+0000", 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025459699999999998 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698018+0000", 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6375999999999999e-05 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698166+0000", 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014818700000000001 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698290+0000", 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012333999999999999 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.197 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138587 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.447610220000001, 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019224502000000001, 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138587 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.198 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713360+0000", 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713368+0000", 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713383+0000", 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5176e-05 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713390+0000", 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8669999999999996e-06 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713449+0000", 2024-09-13T07:04:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8938999999999999e-05 2024-09-13T07:04:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713549+0000", 2024-09-13T07:04:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010009 2024-09-13T07:04:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.716087+0000", 2024-09-13T07:04:05.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025383820000000001 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732351+0000", 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016263564000000001 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732450+0000", 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9529000000000002e-05 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732532+0000", 2024-09-13T07:04:05.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1526000000000005e-05 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732585+0000", 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3819000000000001e-05 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138786 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.013394965000003, 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018847452000000001, 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.203 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138786 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147575+0000", 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147583+0000", 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147601+0000", 2024-09-13T07:04:05.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7961e-05 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147610+0000", 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147661+0000", 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0447e-05 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147754+0000", 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2974999999999997e-05 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.150580+0000", 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002826566 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166135+0000", 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015554772 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166225+0000", 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0258000000000002e-05 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166334+0000", 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010816500000000001 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166424+0000", 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0116000000000003e-05 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.208 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:139127 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.386344000999998, 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019295931999999998, 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139127 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:05.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774626+0000", 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774635+0000", 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5419999999999994e-06 2024-09-13T07:04:05.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774654+0000", 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8359000000000002e-05 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774663+0000", 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774718+0000", 2024-09-13T07:04:05.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5315000000000001e-05 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774805+0000", 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7284e-05 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.779319+0000", 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045138310000000003 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:05.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793772+0000", 2024-09-13T07:04:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014452692 2024-09-13T07:04:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793821+0000", 2024-09-13T07:04:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9563e-05 2024-09-13T07:04:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793878+0000", 2024-09-13T07:04:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6953999999999997e-05 2024-09-13T07:04:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793923+0000", 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4781999999999999e-05 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:164691 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.102384127000001, 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015985102000000001, 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.214 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 164691 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:05.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058589+0000", 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2510000000000001e-06 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058594+0000", 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9089999999999999e-06 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058598+0000", 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2069999999999997e-06 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064175+0000", 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055769909999999999 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064210+0000", 2024-09-13T07:04:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4966999999999999e-05 2024-09-13T07:04:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074572+0000", 2024-09-13T07:04:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010362219000000001 2024-09-13T07:04:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.218 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165201 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.099856197999998, 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013054266, 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165201 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:05.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061117+0000", 2024-09-13T07:04:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.12e-06 2024-09-13T07:04:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061125+0000", 2024-09-13T07:04:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8539999999999997e-06 2024-09-13T07:04:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061129+0000", 2024-09-13T07:04:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:04:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074092+0000", 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012963166999999999 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074136+0000", 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3578000000000002e-05 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074169+0000", 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3581000000000003e-05 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.222 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165202 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7831552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.099838523000003, 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013116245, 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165202 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061132+0000", 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061135+0000", 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3410000000000001e-06 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061142+0000", 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7079999999999997e-06 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061147+0000", 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8930000000000004e-06 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074180+0000", 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013032346 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074217+0000", 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074249+0000", 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1711000000000002e-05 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165203 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.099820409000003, 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013180211000000001, 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165203 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061150+0000", 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061153+0000", 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.108e-06 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061158+0000", 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0649999999999998e-06 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061162+0000", 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0080000000000004e-06 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074262+0000", 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013100557000000001 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074299+0000", 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6981999999999999e-05 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074331+0000", 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1801000000000003e-05 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165204 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8093696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.099805697999997, 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0132388, 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165204 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061167+0000", 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1409999999999999e-06 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061175+0000", 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5519999999999999e-06 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061179+0000", 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.172e-06 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074342+0000", 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013162821999999999 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074373+0000", 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1103999999999999e-05 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074404+0000", 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1359999999999998e-05 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.234 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165205 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.099788738999997, 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013300635, 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165205 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061184+0000", 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0779999999999998e-06 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061192+0000", 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7209999999999994e-06 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061196+0000", 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4469999999999999e-06 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074417+0000", 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013220750999999999 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074452+0000", 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4814999999999998e-05 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074483+0000", 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1118000000000003e-05 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.238 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165206 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.099770618000001, 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013364454, 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165206 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.239 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061200+0000", 2024-09-13T07:04:05.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061203+0000", 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.559e-06 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061210+0000", 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061214+0000", 2024-09-13T07:04:05.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0999999999999997e-06 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074494+0000", 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013279094 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074530+0000", 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6538999999999997e-05 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074565+0000", 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4972999999999999e-05 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169377 3.0 3:07b65a69:::rbd_data.1194e181a50b.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 32.804104223000003, 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.032744151999999999, 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.243 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169377 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356864+0000", 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356880+0000", 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6022999999999999e-05 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356916+0000", 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5921999999999999e-05 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356931+0000", 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4311000000000001e-05 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357011+0000", 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9992000000000003e-05 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357143+0000", 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000131822 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.361578+0000", 2024-09-13T07:04:05.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00443525 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389466+0000", 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027888342999999999 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389529+0000", 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2460999999999997e-05 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389574+0000", 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5405999999999999e-05 2024-09-13T07:04:05.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389611+0000", 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7122000000000001e-05 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169731 3.0 3:031f8e93:::rbd_data.1194e181a50b.0000000000000006:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 31.879678425000002, 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.021389756999999999, 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.248 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169731 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:05.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281291+0000", 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281302+0000", 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0651e-05 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281324+0000", 2024-09-13T07:04:05.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1713999999999999e-05 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281340+0000", 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281450+0000", 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110086 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281593+0000", 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014306 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.286356+0000", 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047626320000000002 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302404+0000", 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016048605000000001 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302476+0000", 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1483000000000007e-05 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302578+0000", 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101883 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302683+0000", 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104681 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.253 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.254 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.254 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:05.255 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:04:05.255 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:05.254+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:05.270 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:04:05.356 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:05.353+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:05.374 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:05.374 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:05.375 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:05.375 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:05.385 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:04:05.457 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:05.458+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:05.491 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:05.492 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:05.492 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:05.492 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:05.492 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:05.501 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:04:05.558 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:05.553+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:05.608 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86837 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8372224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:04:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.487448641, 2024-09-13T07:04:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042501729000000002, 2024-09-13T07:04:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.608 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86837 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121258+0000", 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121257+0000", 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121260+0000", 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9890000000000001e-06 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121267+0000", 2024-09-13T07:04:05.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2639999999999997e-06 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121273+0000", 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1959999999999996e-06 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163438+0000", 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042165616000000003 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163536+0000", 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8008000000000001e-05 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163760+0000", 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000223141 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.48742816399999, 2024-09-13T07:04:05.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:04:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:04:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:04:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:04:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87037 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7159808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.480731561, 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042602844000000001, 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87037 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127975+0000", 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127976+0000", 2024-09-13T07:04:05.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5400000000000001e-06 2024-09-13T07:04:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127981+0000", 2024-09-13T07:04:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.403e-06 2024-09-13T07:04:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127985+0000", 2024-09-13T07:04:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0659999999999997e-06 2024-09-13T07:04:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170449+0000", 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042464582000000001 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170487+0000", 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8039e-05 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170578+0000", 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0426999999999993e-05 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.620 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.48071949200001, 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.621 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:05.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.480707049, 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.625 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.480689451, 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:04:05.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:04:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:04:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:04:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:04:05.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:04:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:04:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:04:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:04:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.633 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.480678748, 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:04:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:04:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:04:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:04:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:04:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:04:05.637 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.480634815, 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.638 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:04:05.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.48061502900001, 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.642 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:04:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.646 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.647 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:05.647 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.480599279, 2024-09-13T07:04:05.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:04:05.647 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.647 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.647 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.647 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.647 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.647 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:04:05.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.647 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:04:05.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:04:05.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:04:05.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:04:05.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.480588027, 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.651 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:04:05.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:05.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:04:05.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:04:05.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:04:05.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:04:05.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.480577428, 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:04:05.655 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:05.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:05.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:05.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:04:05.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:04:05.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:04:05.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.660 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.660 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:05.660 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.480567935, 2024-09-13T07:04:05.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:04:05.660 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.660 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.660 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.660 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.660 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.660 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:04:05.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.660 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:04:05.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:04:05.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:04:05.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:04:05.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:04:05.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.480558391, 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:04:05.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:04:05.668 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.668 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.668 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.668 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.668 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:05.668 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.48052578799999, 2024-09-13T07:04:05.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:04:05.668 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.668 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.668 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.669 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.669 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.669 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:04:05.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.669 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:05.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:04:05.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:04:05.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:04:05.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:04:05.672 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.672 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.672 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.672 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.480512765, 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:04:05.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:04:05.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.676 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87094 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8208384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.47716178, 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042529351, 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87094 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131545+0000", 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131544+0000", 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131547+0000", 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6869999999999999e-06 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131553+0000", 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2439999999999998e-06 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131559+0000", 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6779999999999997e-06 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173895+0000", 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042336226999999997 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.173972+0000", 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7047999999999998e-05 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174074+0000", 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010182 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.477141503, 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.684 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 103.477121302, 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.685 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.687 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.688 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:166663 3.b e135/117 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head v 135'409, mlcod=132'407)", 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 36.194532336999998, 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045108144000000003, 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:05.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:05.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:05.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:05.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414172+0000", 2024-09-13T07:04:05.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:05.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427200+0000", 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013028098 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427229+0000", 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9178000000000001e-05 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427239+0000", 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6870000000000006e-06 2024-09-13T07:04:05.691 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:05.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427393+0000", 2024-09-13T07:04:05.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015382899999999999 2024-09-13T07:04:05.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:05.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427451+0000", 2024-09-13T07:04:05.692 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8301000000000003e-05 2024-09-13T07:04:05.692 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.692 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.692 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:05.692 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459232+0000", 2024-09-13T07:04:05.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031780695999999997 2024-09-13T07:04:05.693 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:05.693 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:05.693 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:05.693 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459282+0000", 2024-09-13T07:04:05.693 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0701000000000001e-05 2024-09-13T07:04:05.693 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.693 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.693 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.693 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:05.693 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:05.693 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:05.695 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:04:05.695 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:05.694+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:05.796 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:05.793+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:05.815 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:05.815 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:05.815 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:05.815 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:05.825 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:04:05.897 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:05.894+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:05.930 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:05.930 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:05.930 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:05.930 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:05.930 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:05.940 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:04:05.997 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:05.993+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:06.048 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445761938000004, 2024-09-13T07:04:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:04:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:04:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:04:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445730836999999, 2024-09-13T07:04:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:04:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:04:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:04:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:04:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:04:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:04:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445708522999993, 2024-09-13T07:04:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445685491999996, 2024-09-13T07:04:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:04:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:04:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445660942000004, 2024-09-13T07:04:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:04:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:04:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:04:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:04:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:04:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:04:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:04:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:04:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:04:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:04:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445625233000001, 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:04:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:04:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445600553999995, 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:04:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:04:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:04:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:04:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:04:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:04:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:04:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:04:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:04:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445567280000006, 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:06.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:06.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:04:06.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445540492000006, 2024-09-13T07:04:06.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:06.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:04:06.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:06.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:04:06.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:04:06.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:04:06.090 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.091 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.091 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.091 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.091 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:06.091 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445515938, 2024-09-13T07:04:06.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:04:06.091 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.091 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.091 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.091 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:04:06.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:04:06.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.095 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:06.096 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445486169000006, 2024-09-13T07:04:06.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:04:06.096 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.096 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.096 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.096 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.096 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.096 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:04:06.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.096 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:04:06.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:04:06.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:04:06.100 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.100 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.100 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.100 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.100 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:06.100 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445426326000003, 2024-09-13T07:04:06.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:04:06.100 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.100 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:06.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:04:06.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.104 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445359475999993, 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:06.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:04:06.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:04:06.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445326973999997, 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:04:06.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:06.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:04:06.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:04:06.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445292352999999, 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:04:06.113 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445242121999996, 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:06.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:04:06.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:04:06.121 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445193935999995, 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:04:06.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:06.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:04:06.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:04:06.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:04:06.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:04:06.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:04:06.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.126 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445167298000001, 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:04:06.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.445120462999995, 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.131 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:04:06.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:04:06.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.441766045999998, 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.135 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.136 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:04:06.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.136 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:06.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.139 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.140 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:06.141 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:04:06.141 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:06.137+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:06.242 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:06.241+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:06.275 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:04:06.297 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:06.298 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:06.298 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:06.298 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:06.308 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:04:06.343 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:06.341+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:06.414 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:06.414 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:06.414 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:06.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:04:06.466 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:06.465+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:06.544 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:06.542+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:06.560 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:04:06.583 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:06.584 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:06.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:06.584 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:06.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.584 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:04:06.584 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:06.584 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 160.52348303299999, 2024-09-13T07:04:06.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:04:06.584 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.584 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:06.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.585 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:04:06.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.585 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:06.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:06.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:04:06.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:04:06.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:06.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.589 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.965699154999996, 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.590 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:04:06.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:04:06.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:04:06.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:04:06.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:04:06.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.965676188000003, 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.594 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:04:06.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.965659009999996, 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.598 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:06.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:06.602 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.965645144999996, 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:06.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:04:06.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:04:06.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.606 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.965631999999999, 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:04:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.608 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:06.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:06.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.965618793000004, 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:06.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:04:06.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:04:06.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:04:06.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:04:06.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:04:06.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:04:06.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:04:06.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:04:06.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.615 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.965604725999995, 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:06.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.965584711999995, 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.620 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:06.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:06.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:06.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:04:06.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:04:06.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:04:06.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:04:06.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:04:06.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:04:06.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:04:06.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:04:06.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:04:06.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:04:06.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:04:06.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:04:06.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.624 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.625 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:06.625 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:06.625 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.965564962000002, 2024-09-13T07:04:06.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:04:06.625 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.625 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.625 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:04:06.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.625 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:04:06.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:04:06.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:04:06.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:04:06.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:04:06.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:04:06.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:04:06.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.965540156000003, 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.629 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.630 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:04:06.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.630 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:06.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:04:06.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:04:06.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:04:06.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:04:06.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:04:06.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.634 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:06.634 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:06.634 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.965515890999995, 2024-09-13T07:04:06.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:04:06.634 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.634 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.634 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:04:06.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:04:06.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:04:06.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.965474158000006, 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.638 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.965437868999999, 2024-09-13T07:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.522334537000006, 2024-09-13T07:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.648 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:04:06.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.648 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:06.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:06.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:04:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:04:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:04:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:04:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:04:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.522317654999995, 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:04:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:04:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:04:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:04:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:04:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:04:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:04:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.521267097999996, 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:04:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.519592548999995, 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:04:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:04:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 75.999010158999994, 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:04:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:04:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:04:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:04:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:04:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:04:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:04:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172880 2.2 2:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e149)", 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.444713913000001, 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.97218948500000002, 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172880 2024-09-13T07:04:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137687+0000", 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137702+0000", 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5023e-05 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137730+0000", 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7719e-05 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137748+0000", 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8189000000000002e-05 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137891+0000", 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014330699999999999 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137903+0000", 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1688999999999999e-05 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142887+0000", 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004984006 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142898+0000", 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.079e-05 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109703+0000", 2024-09-13T07:04:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96680553899999999 2024-09-13T07:04:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109879+0000", 2024-09-13T07:04:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017579800000000001 2024-09-13T07:04:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:06.674 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:06.676 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:04:06.676 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:06.674+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:06.711 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:06.731 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:06.748 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:06.768 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:06.777 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:06.773+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:06.789 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:06.793 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:06.793 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:06.803 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:04:06.813 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:06.835 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:06.835 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:06.836 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:06.836 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:06.836 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:06.836 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:06.836 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:06.836 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:06.836 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:06.838 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:06.839 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:06.878 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:06.878+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:06.953 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:06.953 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:06.963 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:04:06.979 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:06.978+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:07.070 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:07.070 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:07.080 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:07.077+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:07.080 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:04:07.181 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:07.181+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:07.193 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:07.193 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:07.203 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:04:07.282 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:07.282+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:07.321 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:07.321 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:07.331 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:04:07.382 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:07.382+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:07.443 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:07.444 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:07.454 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:04:07.484 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:07.482+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:07.560 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:07.560 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:07.569 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:04:07.585 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:07.586+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:07.677 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:07.678 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:07.686 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:07.686+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:07.687 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:04:07.787 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:07.785+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:07.841 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:07.851 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:04:07.888 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:07.886+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:07.957 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:07.958 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:07.967 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:04:07.988 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:07.985+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:08.075 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:08.075 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:08.085 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:04:08.088 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:08.086+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:08.189 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:08.190+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:08.194 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:08.194 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:08.204 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:04:08.271 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 7, 2, 0, 1] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-13T07:04:08.271 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:04:08.272 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T07:04:08.272 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.000000 2024-09-13T07:04:08.272 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 0 2024-09-13T07:04:08.288 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:04:08.294 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:08.289+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:08.316 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:08.316 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:08.327 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:04:08.390 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:08.389+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:08.482 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:08.482 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:08.491 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:08.489+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:08.492 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:04:08.592 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:08.589+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:08.597 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:08.597 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:08.607 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:04:08.694 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:08.694+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:08.724 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:08.724 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:08.734 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:04:08.795 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:08.794+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:08.840 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:04:08.843 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:08.843 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:08.853 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:04:08.895 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:08.893+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:08.959 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:08.959 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:08.969 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:04:08.988 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:08.997 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:08.994+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:09.014 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:09.035 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:09.053 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:09.074 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:09.078 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:09.078 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:09.088 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:04:09.097 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:09.120 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:09.120 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:09.120 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:09.120 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:09.120 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:09.120 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:09.121 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:09.121 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:09.121 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:09.122 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:09.123 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:09.136 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:09.133+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:09.195 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:09.195 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:09.196 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:09.198 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:09.198+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:09.206 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:04:09.301 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:09.297+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:09.302 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:04:09.304 INFO:teuthology.orchestra.run.smithi118.stderr:set osd.7 primary-affinity to 0 (802) 2024-09-13T07:04:09.319 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:04:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 163.26002860400001, 2024-09-13T07:04:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:04:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:04:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:04:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:04:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:04:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:04:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:04:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:04:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:04:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:04:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:04:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:04:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:04:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:04:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:04:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:04:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:04:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:04:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.702244726, 2024-09-13T07:04:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:04:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:04:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:04:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:04:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:04:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:04:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:04:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:04:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:04:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:04:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:04:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:04:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:04:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:04:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.702221759, 2024-09-13T07:04:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:04:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:04:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:04:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:04:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:04:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:04:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:04:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:04:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.702204581, 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:04:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.702190716, 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:04:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:04:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:04:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:04:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.70217757099999, 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:04:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:04:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.702164364, 2024-09-13T07:04:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:04:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:04:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:04:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:04:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:04:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:04:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:04:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:04:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.702150297, 2024-09-13T07:04:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:04:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:04:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:04:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:04:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:04:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:04:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.702130283, 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:04:09.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:04:09.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:04:09.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:04:09.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.365 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.366 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.366 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:09.366 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.702110533, 2024-09-13T07:04:09.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:04:09.366 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.366 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.367 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:04:09.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.367 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:09.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:09.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:04:09.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:04:09.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:04:09.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:04:09.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.702085727, 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:04:09.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:04:09.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:04:09.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:04:09.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:04:09.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:04:09.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:04:09.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:04:09.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:04:09.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:04:09.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:04:09.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:04:09.375 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.375 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.375 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.375 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.375 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:09.375 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.702061462, 2024-09-13T07:04:09.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:04:09.375 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.375 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.376 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:04:09.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.376 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:09.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:09.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:04:09.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:04:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:04:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:04:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:04:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:04:09.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:04:09.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:04:09.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:04:09.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.702019729, 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.70198344000001, 2024-09-13T07:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 87.258880108, 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 87.258863226000003, 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 87.257812669000003, 2024-09-13T07:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:04:09.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:04:09.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:04:09.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:04:09.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:04:09.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:04:09.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:04:09.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 87.256138120000003, 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.403 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:04:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:09.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:09.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:04:09.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:04:09.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:04:09.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:04:09.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:04:09.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:04:09.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:04:09.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:04:09.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:04:09.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:04:09.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:04:09.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:04:09.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.408 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.408 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:04:09.408 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:09.408 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.735555730000002, 2024-09-13T07:04:09.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:04:09.408 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.408 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:09.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.409 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:04:09.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.409 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:09.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:04:09.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:04:09.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172880 2.2 2:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e149)", 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 25.181259484000002, 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.97218948500000002, 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.413 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172880 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137687+0000", 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137702+0000", 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5023e-05 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137730+0000", 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7719e-05 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137748+0000", 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8189000000000002e-05 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137891+0000", 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014330699999999999 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137903+0000", 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1688999999999999e-05 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142887+0000", 2024-09-13T07:04:09.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004984006 2024-09-13T07:04:09.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:09.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142898+0000", 2024-09-13T07:04:09.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.079e-05 2024-09-13T07:04:09.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109703+0000", 2024-09-13T07:04:09.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96680553899999999 2024-09-13T07:04:09.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109879+0000", 2024-09-13T07:04:09.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017579800000000001 2024-09-13T07:04:09.418 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.418 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.418 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.418 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.418 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.418 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:09.420 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:04:09.420 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:09.418+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:09.520 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:09.517+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:09.570 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:09.570 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:09.571 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:09.571 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:09.581 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:04:09.630 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:09.629+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:09.688 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:09.688 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:09.689 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:09.689 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:09.689 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:09.698 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:04:09.722 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:09.722+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:09.807 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:04:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 163.751477558, 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 131.593596324, 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:04:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:04:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:04:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:04:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:04:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:04:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 131.56782661299999, 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:04:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:04:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:04:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:04:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:04:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:04:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:04:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 131.54667544700001, 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:04:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:04:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:04:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:04:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:04:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:04:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.429630684, 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:04:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:04:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:04:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:04:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:04:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:04:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:04:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:04:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:04:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:04:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:04:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:04:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:04:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:04:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:04:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:04:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.144633439, 2024-09-13T07:04:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:04:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.892924300999994, 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:04:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:04:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:04:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.892906267000001, 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:04:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.892890187000006, 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:04:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:04:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:04:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.892874511000002, 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:04:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.892851589000003, 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:04:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:04:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:04:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:04:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:04:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.892835492000003, 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:04:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.892233880000006, 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:04:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.892191104000005, 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:04:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:04:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 87.735173333999995, 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:04:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.882796403999997, 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:04:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:04:09.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:04:09.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:04:09.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:04:09.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:04:09.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:04:09.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:09.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:04:09.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:04:09.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:04:09.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:04:09.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:164179 3.14 e135/67 3:2fc485bd:::rbd_data.11942e4f5fc7.0000000000000002:head v 135'401, mlcod=132'398)", 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 40.859855949999996, 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063492174999999998, 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:09.878 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945792+0000", 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954811+0000", 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090187940000000001 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954840+0000", 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8659000000000001e-05 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954852+0000", 2024-09-13T07:04:09.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1966e-05 2024-09-13T07:04:09.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954938+0000", 2024-09-13T07:04:09.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6539000000000006e-05 2024-09-13T07:04:09.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954975+0000", 2024-09-13T07:04:09.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6303999999999997e-05 2024-09-13T07:04:09.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:09.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009206+0000", 2024-09-13T07:04:09.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.054231044999999999 2024-09-13T07:04:09.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009286+0000", 2024-09-13T07:04:09.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700999999999999e-05 2024-09-13T07:04:09.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.882 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.883 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:165953 3.d 3:b3a20b45:::rbd_data.11942e4f5fc7.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8796117~39260] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:09.883 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:09.883 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 40.583408392000003, 2024-09-13T07:04:09.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.095290105, 2024-09-13T07:04:09.883 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.883 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:09.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.883 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 165953 2024-09-13T07:04:09.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.883 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:09.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:09.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222240+0000", 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258929+0000", 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.036688401000000002 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258971+0000", 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2212000000000002e-05 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258989+0000", 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7674000000000001e-05 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259072+0000", 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3656e-05 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259297+0000", 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022450700000000001 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:09.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.260463+0000", 2024-09-13T07:04:09.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011665340000000001 2024-09-13T07:04:09.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:09.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.316214+0000", 2024-09-13T07:04:09.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.055750268999999998 2024-09-13T07:04:09.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:09.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317335+0000", 2024-09-13T07:04:09.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011211719999999999 2024-09-13T07:04:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317451+0000", 2024-09-13T07:04:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011616 2024-09-13T07:04:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317532+0000", 2024-09-13T07:04:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0860999999999997e-05 2024-09-13T07:04:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172852 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [call lock.unlock in=52b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 26.606951245000001, 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.90390594199999996, 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172852 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.889 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198696+0000", 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198710+0000", 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3548e-05 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198739+0000", 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9606000000000001e-05 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198752+0000", 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3033e-05 2024-09-13T07:04:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198869+0000", 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116552 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199018+0000", 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014923700000000001 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199308+0000", 2024-09-13T07:04:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028947199999999998 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.200731+0000", 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014229799999999999 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059443+0000", 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.85871189999999997 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059467+0000", 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.478e-05 2024-09-13T07:04:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102380+0000", 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042912463999999997 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102605+0000", 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022489499999999999 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172853 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [watch reconnect cookie 140335574242864 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 25.755202386000001, 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.056267725999999997, 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172853 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050445+0000", 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050461+0000", 2024-09-13T07:04:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5783000000000001e-05 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050497+0000", 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5995999999999999e-05 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050516+0000", 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9579999999999999e-05 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050637+0000", 2024-09-13T07:04:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120536 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050647+0000", 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6989999999999998e-06 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059533+0000", 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088858749999999997 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059545+0000", 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2473000000000001e-05 2024-09-13T07:04:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102962+0000", 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043416656999999997 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.103612+0000", 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000650374 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.106715+0000", 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031033229999999998 2024-09-13T07:04:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:09.900 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:09.901 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:04:09.901 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:09.902+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:10.002 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:10.002+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:10.019 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:10.019 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:10.019 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:10.029 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:04:10.103 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:10.102+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:10.135 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:10.135 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:10.136 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:10.136 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:10.136 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:10.145 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:04:10.204 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:10.202+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:10.255 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137474 3.6 3:65c31370:::rbd_data.1194af11889b.000000000000013b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 186185~11715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-13T07:04:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.510574160999994, 2024-09-13T07:04:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0054357140000001, 2024-09-13T07:04:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.256 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137474 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743934+0000", 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743949+0000", 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5705999999999998e-05 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743960+0000", 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0084e-05 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743971+0000", 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0942e-05 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744045+0000", 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4626e-05 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744062+0000", 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7339000000000001e-05 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746366+0000", 2024-09-13T07:04:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.002303457 2024-09-13T07:04:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746520+0000", 2024-09-13T07:04:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015385900000000001 2024-09-13T07:04:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746734+0000", 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000214253 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749128+0000", 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002394129 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749267+0000", 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013878299999999999 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749328+0000", 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0918000000000002e-05 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749371+0000", 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3188000000000001e-05 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163305 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2387968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.262 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.576355679999999, 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044745869000000001, 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163305 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:10.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678153+0000", 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678156+0000", 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6350000000000002e-06 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678161+0000", 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678166+0000", 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.498e-06 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722751+0000", 2024-09-13T07:04:10.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044585134999999998 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722836+0000", 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4672999999999994e-05 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722900+0000", 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4042999999999997e-05 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163307 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.576335215999997, 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045060944999999998, 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163307 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:10.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678177+0000", 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678183+0000", 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0599999999999996e-06 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678188+0000", 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722941+0000", 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044752844999999999 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723073+0000", 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013128599999999999 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723235+0000", 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000162671 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.270 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163310 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 544768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.576316263999999, 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045290534, 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163310 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678195+0000", 2024-09-13T07:04:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.785e-06 2024-09-13T07:04:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678201+0000", 2024-09-13T07:04:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:04:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678205+0000", 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6999999999999999e-06 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723299+0000", 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045093964 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723384+0000", 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4562999999999997e-05 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723484+0000", 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100045 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163313 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1093632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.576300279000002, 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045562975999999998, 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163313 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:10.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678212+0000", 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8650000000000001e-06 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678217+0000", 2024-09-13T07:04:10.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2660000000000001e-06 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678221+0000", 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-06 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723552+0000", 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045330489000000002 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723660+0000", 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010770800000000001 2024-09-13T07:04:10.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723772+0000", 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112608 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163317 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1642496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.576271470999998, 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045770165000000002, 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.279 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163317 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:10.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678237+0000", 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678241+0000", 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2669999999999998e-06 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678247+0000", 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7309999999999996e-06 2024-09-13T07:04:10.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678252+0000", 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723830+0000", 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045577234000000001 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723936+0000", 2024-09-13T07:04:10.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106277 2024-09-13T07:04:10.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724008+0000", 2024-09-13T07:04:10.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2422999999999994e-05 2024-09-13T07:04:10.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.283 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.283 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163320 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.283 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:10.283 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.576251047, 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045968883000000002, 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163320 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678261+0000", 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8090000000000001e-06 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678267+0000", 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8860000000000001e-06 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678271+0000", 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4549999999999997e-06 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724061+0000", 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045789656999999997 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724151+0000", 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9734999999999994e-05 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724227+0000", 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6848999999999996e-05 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163324 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.575966714000003, 2024-09-13T07:04:10.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045988788000000003, 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163324 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678542+0000", 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678547+0000", 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6829999999999999e-06 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678556+0000", 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1929999999999999e-06 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678562+0000", 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.083e-06 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724267+0000", 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045705427 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724417+0000", 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014984699999999999 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724532+0000", 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011430900000000001 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163327 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 733184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.57594168, 2024-09-13T07:04:10.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046185902000000001, 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163327 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678567+0000", 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678571+0000", 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3799999999999998e-06 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678576+0000", 2024-09-13T07:04:10.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7760000000000003e-06 2024-09-13T07:04:10.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678581+0000", 2024-09-13T07:04:10.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8640000000000004e-06 2024-09-13T07:04:10.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724561+0000", 2024-09-13T07:04:10.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045979178000000002 2024-09-13T07:04:10.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724649+0000", 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8677000000000005e-05 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724754+0000", 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104527 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.295 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163331 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.296 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:10.296 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.575923525999997, 2024-09-13T07:04:10.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046361773000000002, 2024-09-13T07:04:10.296 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.296 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.296 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163331 2024-09-13T07:04:10.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.296 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678585+0000", 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678588+0000", 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0469999999999998e-06 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678595+0000", 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0399999999999998e-06 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678599+0000", 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724780+0000", 2024-09-13T07:04:10.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046181168000000002 2024-09-13T07:04:10.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724871+0000", 2024-09-13T07:04:10.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0562000000000005e-05 2024-09-13T07:04:10.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724948+0000", 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6985999999999998e-05 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163334 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1662976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.575906322999998, 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046544502000000001, 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.300 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163334 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678605+0000", 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5380000000000001e-06 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678611+0000", 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7389999999999996e-06 2024-09-13T07:04:10.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678615+0000", 2024-09-13T07:04:10.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-13T07:04:10.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724987+0000", 2024-09-13T07:04:10.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046371969999999998 2024-09-13T07:04:10.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725075+0000", 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8091999999999996e-05 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725148+0000", 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2206999999999995e-05 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163338 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:10.304 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.575889592999999, 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046738452999999999, 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163338 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678619+0000", 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678622+0000", 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.649e-06 2024-09-13T07:04:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678628+0000", 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5029999999999997e-06 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678632+0000", 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4020000000000004e-06 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725182+0000", 2024-09-13T07:04:10.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046550389999999997 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725280+0000", 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7260000000000001e-05 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725358+0000", 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8720000000000005e-05 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.308 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163339 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2363392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.575874065000001, 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046920835000000001, 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163339 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:10.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678637+0000", 2024-09-13T07:04:10.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2479999999999999e-06 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678642+0000", 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0030000000000002e-06 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678646+0000", 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9870000000000001e-06 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725397+0000", 2024-09-13T07:04:10.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046750127000000002 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725482+0000", 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5928999999999997e-05 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725556+0000", 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3823000000000001e-05 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163342 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.575859426000001, 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047212991000000003, 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163342 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678652+0000", 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.486e-06 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678657+0000", 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6909999999999996e-06 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678660+0000", 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8009999999999999e-06 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725597+0000", 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046937372999999998 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725745+0000", 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000148314 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725863+0000", 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011768 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.316 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163345 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.574997863, 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046569193000000002, 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163345 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678662+0000", 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9991498 2024-09-13T07:04:10.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679516+0000", 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000854789 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679530+0000", 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3492999999999999e-05 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679537+0000", 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.323e-06 2024-09-13T07:04:10.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725892+0000", 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046354244000000003 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725975+0000", 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3356000000000006e-05 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726081+0000", 2024-09-13T07:04:10.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105992 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163349 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1298432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.574965800000001, 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046744247000000003, 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163349 2024-09-13T07:04:10.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679543+0000", 2024-09-13T07:04:10.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679546+0000", 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679551+0000", 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679554+0000", 2024-09-13T07:04:10.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8629999999999999e-06 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726117+0000", 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046563081999999999 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726213+0000", 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6207999999999998e-05 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726288+0000", 2024-09-13T07:04:10.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4741999999999997e-05 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163350 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1449984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.574953174000001, 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046935635000000003, 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.325 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163350 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679558+0000", 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.928e-06 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679561+0000", 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7659999999999999e-06 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679564+0000", 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4960000000000001e-06 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726327+0000", 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046762321000000003 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726415+0000", 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8487999999999997e-05 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726492+0000", 2024-09-13T07:04:10.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6995000000000004e-05 2024-09-13T07:04:10.329 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.329 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.329 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.329 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163353 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.329 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:10.329 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.574943064999999, 2024-09-13T07:04:10.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047130755000000003, 2024-09-13T07:04:10.329 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163353 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679568+0000", 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5969999999999999e-06 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679571+0000", 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679573+0000", 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726523+0000", 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046949882999999998 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726616+0000", 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3236e-05 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726697+0000", 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0834000000000006e-05 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163356 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2379776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:10.333 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.574933383000001, 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047304662999999997, 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163356 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679578+0000", 2024-09-13T07:04:10.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679582+0000", 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679589+0000", 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1149999999999996e-06 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726723+0000", 2024-09-13T07:04:10.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047133768999999999 2024-09-13T07:04:10.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726809+0000", 2024-09-13T07:04:10.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5402000000000007e-05 2024-09-13T07:04:10.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726881+0000", 2024-09-13T07:04:10.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2152999999999998e-05 2024-09-13T07:04:10.337 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.337 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:166663 3.b 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8129196~12979] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 40.856558868999997, 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.061564718999999997, 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 166663 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397947+0000", 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999967 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412144+0000", 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014196972 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412172+0000", 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7781999999999999e-05 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412182+0000", 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6069999999999997e-06 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412239+0000", 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.732e-05 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412365+0000", 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125356 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.413178+0000", 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081355299999999995 2024-09-13T07:04:10.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:10.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.443176+0000", 2024-09-13T07:04:10.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029997903999999999 2024-09-13T07:04:10.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:10.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459446+0000", 2024-09-13T07:04:10.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016269921999999999 2024-09-13T07:04:10.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:10.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459484+0000", 2024-09-13T07:04:10.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8488000000000001e-05 2024-09-13T07:04:10.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459515+0000", 2024-09-13T07:04:10.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0981e-05 2024-09-13T07:04:10.343 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.343 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.343 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.343 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.343 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.343 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:10.345 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:04:10.345 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:10.342+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:10.445 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:10.446+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:10.501 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:10.501 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:10.501 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:10.501 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:10.511 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:04:10.557 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:10.558+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:10.614 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:10.614 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:10.614 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:10.614 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:10.614 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:10.624 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:04:10.646 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:10.646+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:10.731 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:10.731 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:10.731 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:10.731 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:10.732 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.732 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:04:10.732 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:10.732 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 147.236955381, 2024-09-13T07:04:10.732 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:04:10.732 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.732 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.732 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.732 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:04:10.732 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.732 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:04:10.733 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.734 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.735 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:04:10.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:04:10.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:04:10.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:04:10.736 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.736 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.736 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.736 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:04:10.736 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:04:10.736 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.737 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.737 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.737 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.737 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.737 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.737 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:10.737 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.750471817, 2024-09-13T07:04:10.737 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:04:10.738 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.738 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.738 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.738 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.738 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.738 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:04:10.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.738 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.738 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.738 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.738 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:10.738 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.738 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:10.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:10.739 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.739 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.739 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.739 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.739 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:04:10.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:04:10.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:04:10.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:04:10.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.740 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.740 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:04:10.740 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:04:10.740 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.740 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:04:10.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:04:10.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:04:10.741 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:04:10.741 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.741 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.741 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.741 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.750401888, 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.742 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.743 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.743 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:04:10.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.743 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:10.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.743 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.743 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.743 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.743 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:10.743 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.744 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.745 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.746 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:10.747 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.75038601599999, 2024-09-13T07:04:10.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:04:10.747 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.747 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.747 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.747 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.747 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.747 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:04:10.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.747 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:04:10.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:04:10.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:04:10.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:04:10.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:04:10.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.750315559, 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.751 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.752 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:04:10.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.752 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:10.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:10.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:10.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:04:10.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:04:10.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.753 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.753 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:10.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:04:10.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:04:10.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:04:10.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:04:10.755 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.755 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.755 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.755 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.746963521, 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:04:10.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:04:10.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:04:10.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:04:10.759 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.759 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.759 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.74694722000001, 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:04:10.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:04:10.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:04:10.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:04:10.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:04:10.763 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.763 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.763 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.764 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.764 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:10.764 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.746927435, 2024-09-13T07:04:10.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:04:10.764 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.764 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.764 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.764 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.764 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.764 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:04:10.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.764 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:10.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:10.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.767 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.746911162, 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.768 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:04:10.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:04:10.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:04:10.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:04:10.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:04:10.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:04:10.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:04:10.771 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.771 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.74689333699999, 2024-09-13T07:04:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:04:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:04:10.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.773 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:10.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:10.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:04:10.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:04:10.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:04:10.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:04:10.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:04:10.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.74687724399999, 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.776 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.777 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:04:10.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.777 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:10.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:10.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:04:10.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:04:10.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:04:10.780 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.780 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.780 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.780 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.780 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:10.780 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.74685605799999, 2024-09-13T07:04:10.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:04:10.780 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.780 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.780 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.781 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.781 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.781 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:04:10.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.781 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:10.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.74684002500001, 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.784 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.785 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:04:10.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.785 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:10.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:10.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:04:10.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:04:10.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:10.788 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.74679078599999, 2024-09-13T07:04:10.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:04:10.789 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.789 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.789 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.789 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.789 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.789 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:04:10.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.789 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:10.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:10.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:04:10.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:04:10.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:04:10.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:04:10.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:04:10.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:04:10.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:04:10.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.793 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.793 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:10.793 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.74677272599999, 2024-09-13T07:04:10.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:04:10.793 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.793 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.793 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:04:10.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.793 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:04:10.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:04:10.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:04:10.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:04:10.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:04:10.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:04:10.796 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.796 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.796 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.74675621099999, 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.797 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:10.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:10.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:04:10.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:04:10.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:04:10.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:04:10.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:04:10.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:04:10.800 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.746737846, 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.801 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:04:10.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:04:10.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:04:10.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:04:10.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:04:10.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:04:10.805 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.805 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.805 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.805 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.805 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:10.805 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.746721748, 2024-09-13T07:04:10.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:04:10.805 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.805 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.805 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:10.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:04:10.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:04:10.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:04:10.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:04:10.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:04:10.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:04:10.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:04:10.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:04:10.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:04:10.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:04:10.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.809 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.809 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.810 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:10.810 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.74670581300001, 2024-09-13T07:04:10.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:04:10.810 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.810 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.810 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:04:10.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.810 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:04:10.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:04:10.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:04:10.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:04:10.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:04:10.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:04:10.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:04:10.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:04:10.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:04:10.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:04:10.813 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.813 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.813 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 132.74668931599999, 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.814 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:10.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:10.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:10.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:10.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:10.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:10.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:10.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:10.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:10.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:10.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:04:10.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:04:10.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:10.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:04:10.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:04:10.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:04:10.817 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.818 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.818 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.818 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:10.818 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:10.818 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:10.819 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:04:10.820 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:10.817+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:10.920 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:10.917+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:10.974 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:10.974 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:10.974 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:10.975 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:10.985 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:04:11.021 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:11.017+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:11.097 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:11.097 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:11.097 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:11.097 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:11.098 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:11.108 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:04:11.122 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:11.122+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:11.124 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:04:11.217 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:11.217 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:11.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:11.217 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:11.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.218 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:04:11.218 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:11.218 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 165.15914766899999, 2024-09-13T07:04:11.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:04:11.218 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.218 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:11.218 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:11.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:04:11.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:04:11.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:04:11.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:04:11.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:04:11.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:04:11.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:04:11.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:04:11.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:04:11.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:04:11.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:11.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:04:11.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:04:11.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 81.340931311999995, 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:04:11.222 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:11.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:04:11.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:11.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:04:11.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:04:11.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:04:11.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:04:11.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:04:11.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:04:11.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:04:11.226 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.226 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 81.340881924000001, 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.227 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:11.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:11.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.228 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.228 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 81.340859217000002, 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.231 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:04:11.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:11.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:04:11.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:04:11.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:04:11.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:04:11.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:04:11.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:04:11.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:04:11.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:04:11.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.235 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:11.236 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:11.236 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 81.340815023999994, 2024-09-13T07:04:11.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:04:11.236 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.236 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:11.236 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.236 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.236 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.236 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:04:11.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.236 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:04:11.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:04:11.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:04:11.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:04:11.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:04:11.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 81.309835934999995, 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:11.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:11.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:11.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:04:11.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:04:11.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:04:11.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:04:11.244 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.244 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.244 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.244 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137457 3.18 3:1a596dcc:::rbd_data.1194af11889b.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T07:04:11.244 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:11.244 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 66.564214125000007, 2024-09-13T07:04:11.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033734403000000003, 2024-09-13T07:04:11.245 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.245 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:11.245 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.245 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.245 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.245 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137457 2024-09-13T07:04:11.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.245 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:11.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:11.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651523+0000", 2024-09-13T07:04:11.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:11.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651531+0000", 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.328e-06 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651545+0000", 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4270000000000001e-05 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651555+0000", 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5210000000000008e-06 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651626+0000", 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1915999999999994e-05 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651761+0000", 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000134838 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.658577+0000", 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068151820000000004 2024-09-13T07:04:11.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685181+0000", 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026604200000000001 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685193+0000", 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2128e-05 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685226+0000", 2024-09-13T07:04:11.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3504999999999999e-05 2024-09-13T07:04:11.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685259+0000", 2024-09-13T07:04:11.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2866e-05 2024-09-13T07:04:11.250 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.250 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.250 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.250 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138565 3.0 3:002d0425:::rbd_data.11949f62878b.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:04:11.250 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:11.250 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.550843632999999, 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033394489999999999, 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138565 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664893+0000", 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664906+0000", 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2996000000000001e-05 2024-09-13T07:04:11.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664936+0000", 2024-09-13T07:04:11.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9609999999999999e-05 2024-09-13T07:04:11.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664949+0000", 2024-09-13T07:04:11.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383e-05 2024-09-13T07:04:11.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665046+0000", 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6459000000000006e-05 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665259+0000", 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000213141 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.672522+0000", 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072632110000000003 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.697982+0000", 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025459699999999998 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698018+0000", 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6375999999999999e-05 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698166+0000", 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014818700000000001 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698290+0000", 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012333999999999999 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138587 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.502377965999997, 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019224502000000001, 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.256 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.257 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.257 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138587 2024-09-13T07:04:11.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.257 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:11.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713360+0000", 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713368+0000", 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713383+0000", 2024-09-13T07:04:11.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5176e-05 2024-09-13T07:04:11.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713390+0000", 2024-09-13T07:04:11.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8669999999999996e-06 2024-09-13T07:04:11.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713449+0000", 2024-09-13T07:04:11.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8938999999999999e-05 2024-09-13T07:04:11.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713549+0000", 2024-09-13T07:04:11.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010009 2024-09-13T07:04:11.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:11.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.716087+0000", 2024-09-13T07:04:11.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025383820000000001 2024-09-13T07:04:11.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:11.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732351+0000", 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016263564000000001 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732450+0000", 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9529000000000002e-05 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732532+0000", 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1526000000000005e-05 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732585+0000", 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3819000000000001e-05 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138786 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.068162710999999, 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018847452000000001, 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:11.262 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.263 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138786 2024-09-13T07:04:11.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.263 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:11.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147575+0000", 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147583+0000", 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147601+0000", 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7961e-05 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147610+0000", 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147661+0000", 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0447e-05 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147754+0000", 2024-09-13T07:04:11.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2974999999999997e-05 2024-09-13T07:04:11.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:11.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.150580+0000", 2024-09-13T07:04:11.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002826566 2024-09-13T07:04:11.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:11.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166135+0000", 2024-09-13T07:04:11.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015554772 2024-09-13T07:04:11.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:11.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166225+0000", 2024-09-13T07:04:11.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0258000000000002e-05 2024-09-13T07:04:11.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:11.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166334+0000", 2024-09-13T07:04:11.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010816500000000001 2024-09-13T07:04:11.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166424+0000", 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0116000000000003e-05 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:139127 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.441111747000001, 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019295931999999998, 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:11.268 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.269 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.269 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139127 2024-09-13T07:04:11.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.269 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:11.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774626+0000", 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774635+0000", 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5419999999999994e-06 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774654+0000", 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8359000000000002e-05 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774663+0000", 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774718+0000", 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5315000000000001e-05 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774805+0000", 2024-09-13T07:04:11.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7284e-05 2024-09-13T07:04:11.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:11.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.779319+0000", 2024-09-13T07:04:11.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045138310000000003 2024-09-13T07:04:11.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:11.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793772+0000", 2024-09-13T07:04:11.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014452692 2024-09-13T07:04:11.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793821+0000", 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9563e-05 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793878+0000", 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6953999999999997e-05 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793923+0000", 2024-09-13T07:04:11.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4781999999999999e-05 2024-09-13T07:04:11.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.274 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.274 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:164691 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:11.274 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:11.274 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.157151872999997, 2024-09-13T07:04:11.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015985102000000001, 2024-09-13T07:04:11.274 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.274 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:11.274 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.274 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 164691 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:11.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:11.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058589+0000", 2024-09-13T07:04:11.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2510000000000001e-06 2024-09-13T07:04:11.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058594+0000", 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9089999999999999e-06 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058598+0000", 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2069999999999997e-06 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064175+0000", 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055769909999999999 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064210+0000", 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4966999999999999e-05 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074572+0000", 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010362219000000001 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165201 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:11.278 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:11.279 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.154623944000001, 2024-09-13T07:04:11.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013054266, 2024-09-13T07:04:11.279 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.279 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:11.279 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.279 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.279 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.279 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165201 2024-09-13T07:04:11.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.279 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061117+0000", 2024-09-13T07:04:11.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.12e-06 2024-09-13T07:04:11.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061125+0000", 2024-09-13T07:04:11.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8539999999999997e-06 2024-09-13T07:04:11.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061129+0000", 2024-09-13T07:04:11.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:04:11.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074092+0000", 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012963166999999999 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074136+0000", 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3578000000000002e-05 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074169+0000", 2024-09-13T07:04:11.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3581000000000003e-05 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165202 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7831552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.154606268999999, 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013116245, 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.283 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165202 2024-09-13T07:04:11.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.284 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:11.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:11.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061132+0000", 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061135+0000", 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3410000000000001e-06 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061142+0000", 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7079999999999997e-06 2024-09-13T07:04:11.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061147+0000", 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8930000000000004e-06 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074180+0000", 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013032346 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074217+0000", 2024-09-13T07:04:11.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:04:11.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074249+0000", 2024-09-13T07:04:11.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1711000000000002e-05 2024-09-13T07:04:11.287 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.287 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.287 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.287 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165203 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:11.287 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:11.287 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.154588154999999, 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013180211000000001, 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165203 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:11.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061150+0000", 2024-09-13T07:04:11.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:11.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061153+0000", 2024-09-13T07:04:11.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.108e-06 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061158+0000", 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0649999999999998e-06 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061162+0000", 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0080000000000004e-06 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074262+0000", 2024-09-13T07:04:11.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013100557000000001 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074299+0000", 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6981999999999999e-05 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074331+0000", 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1801000000000003e-05 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.292 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165204 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8093696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:11.292 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:11.292 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.154573444, 2024-09-13T07:04:11.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0132388, 2024-09-13T07:04:11.292 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.292 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:11.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.292 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165204 2024-09-13T07:04:11.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.292 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:11.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:11.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061167+0000", 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1409999999999999e-06 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061175+0000", 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5519999999999999e-06 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061179+0000", 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.172e-06 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074342+0000", 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013162821999999999 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074373+0000", 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1103999999999999e-05 2024-09-13T07:04:11.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074404+0000", 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1359999999999998e-05 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165205 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.154556485000001, 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013300635, 2024-09-13T07:04:11.296 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165205 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061184+0000", 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0779999999999998e-06 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061192+0000", 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7209999999999994e-06 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061196+0000", 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4469999999999999e-06 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074417+0000", 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013220750999999999 2024-09-13T07:04:11.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074452+0000", 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4814999999999998e-05 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074483+0000", 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1118000000000003e-05 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165206 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 42.154538363999997, 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013364454, 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165206 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:11.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:11.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061200+0000", 2024-09-13T07:04:11.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:11.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061203+0000", 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.559e-06 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061210+0000", 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061214+0000", 2024-09-13T07:04:11.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0999999999999997e-06 2024-09-13T07:04:11.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074494+0000", 2024-09-13T07:04:11.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013279094 2024-09-13T07:04:11.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074530+0000", 2024-09-13T07:04:11.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6538999999999997e-05 2024-09-13T07:04:11.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074565+0000", 2024-09-13T07:04:11.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4972999999999999e-05 2024-09-13T07:04:11.305 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.305 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.305 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.305 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169377 3.0 3:07b65a69:::rbd_data.1194e181a50b.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-13T07:04:11.305 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:11.305 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 38.858871968999999, 2024-09-13T07:04:11.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.032744151999999999, 2024-09-13T07:04:11.305 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.305 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169377 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:11.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356864+0000", 2024-09-13T07:04:11.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:11.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356880+0000", 2024-09-13T07:04:11.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6022999999999999e-05 2024-09-13T07:04:11.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356916+0000", 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5921999999999999e-05 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356931+0000", 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4311000000000001e-05 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357011+0000", 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9992000000000003e-05 2024-09-13T07:04:11.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357143+0000", 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000131822 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.361578+0000", 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00443525 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389466+0000", 2024-09-13T07:04:11.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027888342999999999 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389529+0000", 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2460999999999997e-05 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389574+0000", 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5405999999999999e-05 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389611+0000", 2024-09-13T07:04:11.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7122000000000001e-05 2024-09-13T07:04:11.311 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.311 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.311 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.311 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169731 3.0 3:031f8e93:::rbd_data.1194e181a50b.0000000000000006:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-13T07:04:11.311 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:11.311 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 37.934446170999998, 2024-09-13T07:04:11.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.021389756999999999, 2024-09-13T07:04:11.311 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:11.311 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:11.311 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169731 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:11.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281291+0000", 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281302+0000", 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0651e-05 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:11.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281324+0000", 2024-09-13T07:04:11.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1713999999999999e-05 2024-09-13T07:04:11.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:11.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281340+0000", 2024-09-13T07:04:11.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:04:11.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:11.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281450+0000", 2024-09-13T07:04:11.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110086 2024-09-13T07:04:11.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:11.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281593+0000", 2024-09-13T07:04:11.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014306 2024-09-13T07:04:11.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:11.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.286356+0000", 2024-09-13T07:04:11.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047626320000000002 2024-09-13T07:04:11.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:11.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302404+0000", 2024-09-13T07:04:11.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016048605000000001 2024-09-13T07:04:11.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:11.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302476+0000", 2024-09-13T07:04:11.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1483000000000007e-05 2024-09-13T07:04:11.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:11.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302578+0000", 2024-09-13T07:04:11.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101883 2024-09-13T07:04:11.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:11.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:11.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:11.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302683+0000", 2024-09-13T07:04:11.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104681 2024-09-13T07:04:11.317 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.317 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.317 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.317 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:11.317 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:11.317 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:11.318 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:04:11.319 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:04:11.320 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:11.320 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:11.320 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:11.320 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:11.318+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:11.321 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:11.342 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:11.366 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:11.387 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:11.387 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:11.387 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:11.387 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:11.387 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:11.387 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:11.387 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:11.387 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:11.388 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:11.392 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:11.392 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:11.419 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:11.417+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:11.427 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:11.427 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:11.438 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:04:11.520 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:11.517+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:11.543 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:11.544 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:11.553 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:04:11.621 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:11.617+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:11.656 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:11.656 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:11.666 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:04:11.722 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:11.721+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:11.784 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:11.784 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:11.794 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:04:11.823 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:11.821+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:11.902 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:11.903 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:11.912 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:04:11.924 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:11.922+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:12.025 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:12.026+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:12.062 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:12.062 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:12.072 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:04:12.127 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:12.125+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:12.178 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:12.179 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:12.188 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:04:12.228 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:12.225+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:12.290 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:12.290 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:12.300 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:04:12.328 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:04:12.342 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:12.341+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:12.429 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:12.425+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:12.450 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:12.451 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:12.461 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:04:12.530 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:12.530+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:12.563 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:12.563 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:12.573 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:04:12.631 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:12.629+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:12.679 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:12.679 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:12.689 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:04:12.732 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:12.730+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:12.792 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:12.792 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:12.802 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:04:12.849 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:12.850+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:12.921 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:12.921 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:12.931 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:04:12.933 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:12.934+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:13.037 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:13.033+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:13.038 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:13.038 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:13.048 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:04:13.138 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:13.138+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:13.157 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:13.158 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:13.167 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:04:13.239 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:13.238+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:13.318 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:13.318 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:13.329 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:04:13.377 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:13.378+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:13.393 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:04:13.440 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:13.437+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:13.451 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:13.451 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:13.462 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:04:13.527 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:13.541 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:13.542+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:13.547 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:13.568 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:13.570 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:13.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:04:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.07582043900001, 2024-09-13T07:04:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:04:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.571 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:04:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:13.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.573 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:13.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.58933687499999, 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.576 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:04:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:13.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:04:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:04:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.580 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.58926694600001, 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:13.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:04:13.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:04:13.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.589251074, 2024-09-13T07:04:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:04:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:04:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.585 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:04:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:04:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:04:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:04:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.58918061700001, 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:13.589 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:13.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:13.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:04:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:04:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:04:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:04:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.593 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.58582857900001, 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:04:13.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:13.597 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.58581227799999, 2024-09-13T07:04:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:04:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:04:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:13.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:04:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.601 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.58579249300001, 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:13.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.603 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.605 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.58577622000001, 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:04:13.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:04:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:04:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.585758395, 2024-09-13T07:04:13.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:04:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:04:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:04:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:04:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:04:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:04:13.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.585742302, 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:04:13.613 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:04:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:04:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:04:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:04:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:04:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:04:13.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:04:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:04:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.617 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.585721116, 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.621 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.58570508299999, 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:13.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:04:13.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.585655844, 2024-09-13T07:04:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.585637784, 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.629 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:04:13.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:04:13.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:04:13.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:04:13.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:04:13.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:04:13.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:04:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:04:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.585621269, 2024-09-13T07:04:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:04:13.633 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.58560290400001, 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.637 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:04:13.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:04:13.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:04:13.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:04:13.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:04:13.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:04:13.641 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.641 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.641 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.641 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.641 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:13.641 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.58558680600001, 2024-09-13T07:04:13.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:04:13.641 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.641 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.641 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.641 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:04:13.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:04:13.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.58557087099999, 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.645 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.648 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 135.585554374, 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:04:13.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:13.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.652 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.653 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.653 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:13.653 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:13.653 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:13.654 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:04:13.655 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:13.655 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:13.655 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:13.655 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:13.654+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:13.658 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:13.658 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:13.659 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:13.659 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:13.659 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:13.659 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:13.659 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:13.659 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:13.659 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:13.663 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:13.664 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:13.755 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:13.754+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:13.770 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:13.771 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:13.771 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:13.781 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:04:13.856 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:13.857+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:13.887 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:13.887 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:13.888 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:13.888 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:13.888 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:13.897 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:04:13.958 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:13.958+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:14.008 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:14.009 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:14.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:14.009 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:14.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.009 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:04:14.009 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:14.009 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 167.95082401400001, 2024-09-13T07:04:14.009 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:04:14.009 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.009 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:14.009 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.009 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:14.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:14.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.010 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.010 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.010 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:04:14.010 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:14.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:04:14.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:04:14.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.011 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:04:14.011 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:04:14.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.011 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.011 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:04:14.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:04:14.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:04:14.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:04:14.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.012 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.012 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.012 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:04:14.012 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:04:14.012 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.013 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.132607656999994, 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.014 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:14.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:14.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.015 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.015 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.015 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:04:14.015 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:14.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:04:14.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:04:14.017 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:04:14.018 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.018 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.018 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.018 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.018 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:14.018 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:14.018 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.132558269, 2024-09-13T07:04:14.018 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:04:14.018 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.019 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.019 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.019 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.019 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.019 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:04:14.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.019 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.019 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.019 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.019 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:14.019 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.019 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:14.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:04:14.020 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.020 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.020 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.020 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.020 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.021 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.022 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.023 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.023 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.023 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.023 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:14.023 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:14.023 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.132535562000001, 2024-09-13T07:04:14.023 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:04:14.023 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.023 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.023 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.023 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.023 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.023 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.024 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:04:14.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:04:14.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:04:14.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.025 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.025 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.025 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:04:14.025 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:04:14.025 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:04:14.026 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.132491368999993, 2024-09-13T07:04:14.027 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:04:14.028 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.028 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.028 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.028 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.028 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.028 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:04:14.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.028 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.028 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.028 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.028 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:14.028 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:04:14.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:04:14.030 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.031 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.101512279999994, 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.032 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.033 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.034 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.035 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137457 3.18 3:1a596dcc:::rbd_data.1194af11889b.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 69.355890470000006, 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033734403000000003, 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137457 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.036 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651523+0000", 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.037 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651531+0000", 2024-09-13T07:04:14.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.328e-06 2024-09-13T07:04:14.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651545+0000", 2024-09-13T07:04:14.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4270000000000001e-05 2024-09-13T07:04:14.038 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.038 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.038 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.038 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651555+0000", 2024-09-13T07:04:14.038 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5210000000000008e-06 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651626+0000", 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1915999999999994e-05 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651761+0000", 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000134838 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.658577+0000", 2024-09-13T07:04:14.039 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068151820000000004 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685181+0000", 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026604200000000001 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685193+0000", 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2128e-05 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685226+0000", 2024-09-13T07:04:14.040 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3504999999999999e-05 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685259+0000", 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2866e-05 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138565 3.0 3:002d0425:::rbd_data.11949f62878b.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.342519977999999, 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033394489999999999, 2024-09-13T07:04:14.041 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138565 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.042 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664893+0000", 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664906+0000", 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2996000000000001e-05 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.043 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664936+0000", 2024-09-13T07:04:14.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9609999999999999e-05 2024-09-13T07:04:14.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664949+0000", 2024-09-13T07:04:14.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383e-05 2024-09-13T07:04:14.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.044 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.044 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665046+0000", 2024-09-13T07:04:14.044 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6459000000000006e-05 2024-09-13T07:04:14.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.044 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665259+0000", 2024-09-13T07:04:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000213141 2024-09-13T07:04:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.672522+0000", 2024-09-13T07:04:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072632110000000003 2024-09-13T07:04:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.697982+0000", 2024-09-13T07:04:14.045 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025459699999999998 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698018+0000", 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6375999999999999e-05 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698166+0000", 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014818700000000001 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.046 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698290+0000", 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012333999999999999 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138587 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.294054310999996, 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019224502000000001, 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.047 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138587 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.048 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713360+0000", 2024-09-13T07:04:14.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:14.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713368+0000", 2024-09-13T07:04:14.049 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:04:14.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.049 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.049 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.049 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713383+0000", 2024-09-13T07:04:14.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5176e-05 2024-09-13T07:04:14.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713390+0000", 2024-09-13T07:04:14.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8669999999999996e-06 2024-09-13T07:04:14.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.050 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.050 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713449+0000", 2024-09-13T07:04:14.050 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8938999999999999e-05 2024-09-13T07:04:14.050 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.050 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713549+0000", 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010009 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.716087+0000", 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025383820000000001 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732351+0000", 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016263564000000001 2024-09-13T07:04:14.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:14.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732450+0000", 2024-09-13T07:04:14.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9529000000000002e-05 2024-09-13T07:04:14.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:14.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732532+0000", 2024-09-13T07:04:14.052 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1526000000000005e-05 2024-09-13T07:04:14.052 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.052 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.052 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.052 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732585+0000", 2024-09-13T07:04:14.053 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3819000000000001e-05 2024-09-13T07:04:14.053 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.053 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.053 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.053 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.053 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.056 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138786 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:04:14.056 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:14.056 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.859839055999998, 2024-09-13T07:04:14.056 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018847452000000001, 2024-09-13T07:04:14.056 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.056 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:14.056 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.056 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.056 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138786 2024-09-13T07:04:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.057 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147575+0000", 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147583+0000", 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147601+0000", 2024-09-13T07:04:14.058 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7961e-05 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147610+0000", 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147661+0000", 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0447e-05 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.059 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147754+0000", 2024-09-13T07:04:14.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2974999999999997e-05 2024-09-13T07:04:14.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:14.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.150580+0000", 2024-09-13T07:04:14.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002826566 2024-09-13T07:04:14.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.060 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:14.060 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166135+0000", 2024-09-13T07:04:14.060 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015554772 2024-09-13T07:04:14.060 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.060 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166225+0000", 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0258000000000002e-05 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166334+0000", 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010816500000000001 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166424+0000", 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0116000000000003e-05 2024-09-13T07:04:14.061 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.062 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.062 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.062 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.062 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.062 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:139127 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:04:14.062 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:14.062 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.232788092000007, 2024-09-13T07:04:14.062 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019295931999999998, 2024-09-13T07:04:14.062 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.062 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:14.062 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.062 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.062 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139127 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.063 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774626+0000", 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774635+0000", 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5419999999999994e-06 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774654+0000", 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8359000000000002e-05 2024-09-13T07:04:14.064 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774663+0000", 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774718+0000", 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5315000000000001e-05 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774805+0000", 2024-09-13T07:04:14.065 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7284e-05 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.779319+0000", 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045138310000000003 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793772+0000", 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014452692 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793821+0000", 2024-09-13T07:04:14.066 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9563e-05 2024-09-13T07:04:14.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:14.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793878+0000", 2024-09-13T07:04:14.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6953999999999997e-05 2024-09-13T07:04:14.067 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.067 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.067 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793923+0000", 2024-09-13T07:04:14.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4781999999999999e-05 2024-09-13T07:04:14.067 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.067 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.067 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:164691 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 44.948828218000003, 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015985102000000001, 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 164691 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.068 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:14.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:14.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058589+0000", 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2510000000000001e-06 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058594+0000", 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9089999999999999e-06 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058598+0000", 2024-09-13T07:04:14.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2069999999999997e-06 2024-09-13T07:04:14.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064175+0000", 2024-09-13T07:04:14.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055769909999999999 2024-09-13T07:04:14.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064210+0000", 2024-09-13T07:04:14.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4966999999999999e-05 2024-09-13T07:04:14.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074572+0000", 2024-09-13T07:04:14.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010362219000000001 2024-09-13T07:04:14.072 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.072 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.072 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.072 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165201 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:14.072 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:14.072 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 44.946300289, 2024-09-13T07:04:14.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013054266, 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165201 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061117+0000", 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.12e-06 2024-09-13T07:04:14.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061125+0000", 2024-09-13T07:04:14.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8539999999999997e-06 2024-09-13T07:04:14.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061129+0000", 2024-09-13T07:04:14.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:04:14.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074092+0000", 2024-09-13T07:04:14.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012963166999999999 2024-09-13T07:04:14.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074136+0000", 2024-09-13T07:04:14.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3578000000000002e-05 2024-09-13T07:04:14.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.076 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.076 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074169+0000", 2024-09-13T07:04:14.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3581000000000003e-05 2024-09-13T07:04:14.076 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.076 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.077 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.077 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165202 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7831552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:14.077 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:14.077 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 44.946282613999998, 2024-09-13T07:04:14.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013116245, 2024-09-13T07:04:14.077 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.077 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.077 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.077 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.077 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.077 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165202 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061132+0000", 2024-09-13T07:04:14.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061135+0000", 2024-09-13T07:04:14.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3410000000000001e-06 2024-09-13T07:04:14.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061142+0000", 2024-09-13T07:04:14.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7079999999999997e-06 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061147+0000", 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8930000000000004e-06 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074180+0000", 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013032346 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074217+0000", 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074249+0000", 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1711000000000002e-05 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165203 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:14.081 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 44.946264499999998, 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013180211000000001, 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165203 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061150+0000", 2024-09-13T07:04:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061153+0000", 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.108e-06 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061158+0000", 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0649999999999998e-06 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061162+0000", 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0080000000000004e-06 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074262+0000", 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013100557000000001 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074299+0000", 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6981999999999999e-05 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074331+0000", 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1801000000000003e-05 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.085 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165204 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8093696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 44.946249788999999, 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0132388, 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165204 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.086 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:14.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061167+0000", 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1409999999999999e-06 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061175+0000", 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5519999999999999e-06 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061179+0000", 2024-09-13T07:04:14.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.172e-06 2024-09-13T07:04:14.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074342+0000", 2024-09-13T07:04:14.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013162821999999999 2024-09-13T07:04:14.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074373+0000", 2024-09-13T07:04:14.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1103999999999999e-05 2024-09-13T07:04:14.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074404+0000", 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1359999999999998e-05 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165205 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 44.94623283, 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013300635, 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.090 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165205 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:14.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:14.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061184+0000", 2024-09-13T07:04:14.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0779999999999998e-06 2024-09-13T07:04:14.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061192+0000", 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7209999999999994e-06 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061196+0000", 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4469999999999999e-06 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074417+0000", 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013220750999999999 2024-09-13T07:04:14.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074452+0000", 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4814999999999998e-05 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074483+0000", 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1118000000000003e-05 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165206 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 44.946214709000003, 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013364454, 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165206 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061200+0000", 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061203+0000", 2024-09-13T07:04:14.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.559e-06 2024-09-13T07:04:14.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061210+0000", 2024-09-13T07:04:14.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-13T07:04:14.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061214+0000", 2024-09-13T07:04:14.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0999999999999997e-06 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074494+0000", 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013279094 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074530+0000", 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6538999999999997e-05 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074565+0000", 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4972999999999999e-05 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169377 3.0 3:07b65a69:::rbd_data.1194e181a50b.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 41.650548313999998, 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.032744151999999999, 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.099 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169377 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356864+0000", 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356880+0000", 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6022999999999999e-05 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356916+0000", 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5921999999999999e-05 2024-09-13T07:04:14.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356931+0000", 2024-09-13T07:04:14.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4311000000000001e-05 2024-09-13T07:04:14.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357011+0000", 2024-09-13T07:04:14.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9992000000000003e-05 2024-09-13T07:04:14.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357143+0000", 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000131822 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.361578+0000", 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00443525 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389466+0000", 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027888342999999999 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:14.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389529+0000", 2024-09-13T07:04:14.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2460999999999997e-05 2024-09-13T07:04:14.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:14.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389574+0000", 2024-09-13T07:04:14.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5405999999999999e-05 2024-09-13T07:04:14.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389611+0000", 2024-09-13T07:04:14.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7122000000000001e-05 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169731 3.0 3:031f8e93:::rbd_data.1194e181a50b.0000000000000006:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 40.726122515999997, 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.021389756999999999, 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.105 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169731 2024-09-13T07:04:14.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.106 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:14.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:14.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281291+0000", 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281302+0000", 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0651e-05 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281324+0000", 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1713999999999999e-05 2024-09-13T07:04:14.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281340+0000", 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281450+0000", 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110086 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281593+0000", 2024-09-13T07:04:14.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014306 2024-09-13T07:04:14.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:14.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.286356+0000", 2024-09-13T07:04:14.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047626320000000002 2024-09-13T07:04:14.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:14.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302404+0000", 2024-09-13T07:04:14.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016048605000000001 2024-09-13T07:04:14.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302476+0000", 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1483000000000007e-05 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302578+0000", 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101883 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302683+0000", 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104681 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.110 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.111 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.111 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.111 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.111 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:14.112 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:04:14.113 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:14.109+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:14.213 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:14.214+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:14.224 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:14.224 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:14.224 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:14.234 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:04:14.314 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:14.309+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:14.316 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 7, 2, 0, 1] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-13T07:04:14.316 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:04:14.316 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-13T07:04:14.316 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T07:04:14.316 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T07:04:14.317 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-13T07:04:14.342 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:04:14.412 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:14.412 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:14.412 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:14.412 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:14.412 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:14.414 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:14.413+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:14.423 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:04:14.444 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:14.444 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T07:04:14.444 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:14.516 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:14.513+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:14.541 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:14.541 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:14.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:14.541 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:14.541 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.541 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.541 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:14.541 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.418687897, 2024-09-13T07:04:14.541 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:04:14.541 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.541 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.542 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.542 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.542 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.542 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:04:14.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.542 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.542 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:14.542 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.542 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.542 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.543 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:04:14.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:04:14.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.544 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:04:14.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:04:14.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.544 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.544 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.544 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:04:14.544 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.411979225, 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.546 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.547 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:04:14.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:04:14.549 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:04:14.550 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.550 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.550 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.550 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.550 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.550 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.550 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:14.550 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.41196678199999, 2024-09-13T07:04:14.550 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:04:14.550 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.550 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.550 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:14.551 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.552 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.552 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:14.552 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:14.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.552 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.552 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:04:14.552 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:04:14.552 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.552 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:04:14.553 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:04:14.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:04:14.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.554 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.554 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.554 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:04:14.554 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:04:14.554 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.554 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.554 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.554 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.555 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.555 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.555 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:14.555 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.41194918399999, 2024-09-13T07:04:14.555 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:04:14.555 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.555 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.555 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.555 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.555 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.555 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:04:14.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.555 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:14.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:14.556 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.556 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.556 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.556 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.556 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:14.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.557 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:04:14.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:04:14.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.557 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.557 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:04:14.557 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:04:14.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.557 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:04:14.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.41193848100001, 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:04:14.559 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.560 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:14.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:14.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.561 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.561 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.561 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.561 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:04:14.561 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:04:14.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:04:14.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:04:14.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.562 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:04:14.562 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:04:14.562 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.562 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.562 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.563 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.564 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.564 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.564 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.564 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.564 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:14.564 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.41189454800001, 2024-09-13T07:04:14.564 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:04:14.564 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.564 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.564 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.564 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.564 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.565 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.566 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:04:14.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.411874762, 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:04:14.568 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.569 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.570 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.571 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.572 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.573 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:14.573 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.41185901199999, 2024-09-13T07:04:14.573 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:04:14.573 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.573 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.573 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.573 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.573 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.573 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:04:14.573 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.573 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.573 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.573 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.574 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.575 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.576 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.41184776, 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.577 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:14.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:14.578 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.578 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.578 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.578 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.578 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:04:14.579 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:04:14.580 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:04:14.581 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.581 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.581 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.581 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.581 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.581 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.581 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:14.581 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.41183716099999, 2024-09-13T07:04:14.581 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.582 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:04:14.583 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:04:14.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:04:14.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:04:14.584 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:04:14.584 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.584 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.584 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.584 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.585 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.411827668, 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.586 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:04:14.587 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:14.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:04:14.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:04:14.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.588 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:04:14.588 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:04:14.588 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.588 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.588 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:04:14.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:04:14.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:04:14.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:04:14.589 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.589 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.589 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.589 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:04:14.589 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.41181812400001, 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.590 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.591 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:04:14.592 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:04:14.593 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.411785521, 2024-09-13T07:04:14.594 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.595 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.596 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:04:14.597 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.598 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:14.599 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.411772498, 2024-09-13T07:04:14.599 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:04:14.599 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.599 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.599 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.599 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.599 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.599 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:04:14.599 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.599 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.599 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.599 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.599 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.600 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.601 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:04:14.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:04:14.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:04:14.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:04:14.602 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.602 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.602 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.602 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:04:14.602 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:04:14.602 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.408401236, 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.603 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:04:14.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.604 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:14.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.604 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.604 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:14.604 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.604 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.604 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.605 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:04:14.606 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 112.408381035, 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:14.607 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:14.608 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:14.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.609 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:04:14.609 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:04:14.609 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.609 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.609 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.610 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:04:14.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:04:14.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.611 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.611 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:04:14.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:04:14.611 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.611 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.611 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:166663 3.b e135/117 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head v 135'409, mlcod=132'407)", 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 45.125792070000003, 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045108144000000003, 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414172+0000", 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427200+0000", 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013028098 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427229+0000", 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9178000000000001e-05 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427239+0000", 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6870000000000006e-06 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427393+0000", 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015382899999999999 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427451+0000", 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8301000000000003e-05 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459232+0000", 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031780695999999997 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459282+0000", 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0701000000000001e-05 2024-09-13T07:04:14.615 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:176785 3.7 3:e640995f:::rbd_data.1194987e4741.0000000000000011:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:09.441952+0000", 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.0980140250000003, 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86669136400000002, 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 176785 2024-09-13T07:04:14.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441952+0000", 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441952+0000", 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441951+0000", 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441958+0000", 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3089999999999998e-06 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441970+0000", 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2404e-05 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441977+0000", 2024-09-13T07:04:14.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:04:14.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442051+0000", 2024-09-13T07:04:14.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3337000000000006e-05 2024-09-13T07:04:14.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:14.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442063+0000", 2024-09-13T07:04:14.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2931e-05 2024-09-13T07:04:14.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305282+0000", 2024-09-13T07:04:14.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86321888999999996 2024-09-13T07:04:14.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305447+0000", 2024-09-13T07:04:14.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016446500000000001 2024-09-13T07:04:14.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305579+0000", 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000132452 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.307846+0000", 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022664790000000001 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308562+0000", 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00071642899999999998 2024-09-13T07:04:14.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308606+0000", 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4073999999999998e-05 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308644+0000", 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7348999999999998e-05 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.622 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:176787 3.15 3:af329162:::rbd_data.1194987e4741.0000000000000013:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:09.442992+0000", 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.0969742819999997, 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86944086799999998, 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 176787 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442992+0000", 2024-09-13T07:04:14.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442992+0000", 2024-09-13T07:04:14.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442990+0000", 2024-09-13T07:04:14.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:14.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442999+0000", 2024-09-13T07:04:14.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8019999999999998e-06 2024-09-13T07:04:14.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443015+0000", 2024-09-13T07:04:14.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6002999999999999e-05 2024-09-13T07:04:14.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.625 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443024+0000", 2024-09-13T07:04:14.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3610000000000001e-06 2024-09-13T07:04:14.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443092+0000", 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7967999999999997e-05 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443105+0000", 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3969e-05 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.303912+0000", 2024-09-13T07:04:14.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86080616200000004 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.304190+0000", 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00027807599999999999 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.304362+0000", 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017219199999999999 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:14.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305588+0000", 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001226499 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312299+0000", 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0067103789999999998 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312377+0000", 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8291e-05 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312433+0000", 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5755999999999999e-05 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:176791 3.7 3:e2fcfe96:::rbd_data.1194987e4741.0000000000000018:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:09.444146+0000", 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 5.0958198220000002, 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86828808400000002, 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:14.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 176791 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444146+0000", 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444146+0000", 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:14.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444145+0000", 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444151+0000", 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0050000000000001e-06 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444168+0000", 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6399e-05 2024-09-13T07:04:14.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444175+0000", 2024-09-13T07:04:14.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5610000000000001e-06 2024-09-13T07:04:14.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444240+0000", 2024-09-13T07:04:14.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4893000000000004e-05 2024-09-13T07:04:14.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:14.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444250+0000", 2024-09-13T07:04:14.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0029e-05 2024-09-13T07:04:14.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:14.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.306520+0000", 2024-09-13T07:04:14.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86226998899999996 2024-09-13T07:04:14.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.633 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:14.633 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.306696+0000", 2024-09-13T07:04:14.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017564800000000001 2024-09-13T07:04:14.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.306820+0000", 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000123726 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308963+0000", 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021437990000000001 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312309+0000", 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033458590000000001 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:14.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312384+0000", 2024-09-13T07:04:14.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4184000000000007e-05 2024-09-13T07:04:14.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:14.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:14.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:14.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312435+0000", 2024-09-13T07:04:14.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1019000000000001e-05 2024-09-13T07:04:14.635 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.635 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.635 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.635 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:14.635 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:14.636 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:14.637 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:04:14.637 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:14.638+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:14.738 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:14.738+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:14.756 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:14.756 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:14.756 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:14.757 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:14.766 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:04:14.838 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:14.838+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:14.869 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:14.869 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:14.869 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:14.872 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:14.872 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:14.879 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:04:14.939 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:14.938+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:14.986 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:14.986 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:14.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:14.986 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:14.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.986 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:14.986 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:14.986 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382935946999993, 2024-09-13T07:04:14.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:04:14.986 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:14.986 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:14.986 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:14.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:14.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:14.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:14.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.991 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:14.991 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:14.991 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382904846000002, 2024-09-13T07:04:14.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:04:14.991 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:14.991 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:14.991 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:14.991 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:14.991 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.991 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:04:14.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.992 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:14.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:14.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:14.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:14.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:14.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:14.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:14.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:14.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:14.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:14.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:04:14.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:04:14.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:14.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:04:14.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:04:14.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:04:14.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:04:14.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:14.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:04:14.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:04:14.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:14.995 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382882531999996, 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.996 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:04:14.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:14.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:04:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:04:15.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:04:15.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:04:15.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.000 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.000 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.000 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382859500999999, 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:04:15.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:15.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:04:15.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:04:15.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:04:15.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:04:15.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:04:15.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:04:15.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:04:15.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.005 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.005 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.005 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:15.005 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382834951000007, 2024-09-13T07:04:15.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:04:15.005 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:15.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:04:15.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382799242000004, 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:04:15.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:04:15.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:04:15.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:04:15.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:04:15.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:04:15.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:04:15.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:04:15.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:04:15.014 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.014 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.014 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.014 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.014 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:15.015 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382774562999998, 2024-09-13T07:04:15.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:04:15.015 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.015 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.015 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.015 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.015 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.015 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:04:15.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.015 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:04:15.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:04:15.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:04:15.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:04:15.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:04:15.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:04:15.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382741288999995, 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:15.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:04:15.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:04:15.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:04:15.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:04:15.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:04:15.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:04:15.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:04:15.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:04:15.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:04:15.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:04:15.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:04:15.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:04:15.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:04:15.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382714500999995, 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:04:15.024 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:04:15.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:04:15.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:04:15.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:04:15.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:04:15.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:04:15.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.028 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382689947000003, 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:04:15.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.030 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:15.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:15.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:04:15.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:04:15.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:04:15.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:04:15.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:04:15.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:04:15.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:04:15.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:04:15.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:04:15.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:04:15.033 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.033 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.033 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382660177999995, 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.034 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:15.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:04:15.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:04:15.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:04:15.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:04:15.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:04:15.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:04:15.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.038 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382600335000006, 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:04:15.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:04:15.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:04:15.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:04:15.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:04:15.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:04:15.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:04:15.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382533484999996, 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:04:15.043 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:15.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:04:15.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:04:15.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:04:15.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:04:15.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:04:15.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:04:15.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:04:15.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:04:15.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382500983, 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.049 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:04:15.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.049 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:15.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:15.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:04:15.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:04:15.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.053 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.053 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:15.053 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382466362000002, 2024-09-13T07:04:15.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:04:15.053 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.053 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.053 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.053 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.053 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.053 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:04:15.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:04:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:04:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:04:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:04:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:04:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:04:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:04:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:04:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:04:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:04:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382416130999999, 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382367944999999, 2024-09-13T07:04:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:04:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:04:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:04:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:04:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:04:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:04:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:04:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:04:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:04:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382341307000004, 2024-09-13T07:04:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:04:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:04:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:04:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:04:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:04:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:04:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:04:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:04:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:04:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:04:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:04:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.382294471999998, 2024-09-13T07:04:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:04:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:04:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:04:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:04:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:04:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:04:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:04:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:04:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.378940055000001, 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:04:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:04:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:04:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:04:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:04:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:04:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:04:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:04:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:04:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:04:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:04:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:04:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:04:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.082 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:15.084 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:04:15.084 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:15.082+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:15.185 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:15.186+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:15.197 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:15.197 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:15.197 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:15.197 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:15.207 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:04:15.286 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:15.286+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:15.310 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:15.310 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:15.310 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:15.310 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:15.310 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:15.320 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:04:15.362 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:04:15.417 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:15.418+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:15.440 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:15.441 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:15.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:15.441 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:15.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.441 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:04:15.441 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:15.441 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 169.38001782500001, 2024-09-13T07:04:15.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:04:15.441 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.441 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:15.441 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.441 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.442 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.442 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:04:15.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.442 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:15.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:15.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:04:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:04:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:04:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:04:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:04:15.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:15.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:04:15.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:04:15.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:15.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:04:15.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:04:15.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.822233947, 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.447 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:04:15.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:04:15.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:04:15.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.451 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.82221097999999, 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:15.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:15.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:04:15.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:04:15.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:04:15.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:04:15.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:04:15.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:04:15.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:04:15.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:04:15.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:04:15.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:15.456 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.822193802, 2024-09-13T07:04:15.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:04:15.457 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.457 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.457 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:04:15.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.457 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:04:15.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:04:15.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:04:15.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:04:15.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:04:15.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:04:15.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:04:15.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:04:15.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:04:15.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:04:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:04:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:04:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:15.461 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.822179937, 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:04:15.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:04:15.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.822166792, 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:04:15.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:04:15.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:04:15.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:04:15.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:04:15.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:04:15.469 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.469 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.470 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.470 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:15.470 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.822153585, 2024-09-13T07:04:15.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:04:15.470 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.470 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.470 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.470 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.470 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.470 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:15.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:04:15.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:04:15.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:04:15.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:04:15.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:04:15.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.822139518, 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:04:15.474 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:04:15.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.478 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.822119504, 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:04:15.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:04:15.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:04:15.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:04:15.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.82209975400001, 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.483 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.484 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.484 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.484 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:04:15.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.484 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:15.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:15.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:04:15.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:04:15.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:04:15.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:04:15.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:04:15.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:04:15.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:04:15.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:04:15.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:04:15.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.490 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:15.491 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.82207494799999, 2024-09-13T07:04:15.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:04:15.491 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.491 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.491 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:04:15.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.491 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:04:15.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:04:15.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:04:15.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:04:15.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:04:15.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.822050683, 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.495 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:04:15.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:04:15.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:04:15.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:04:15.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:15.499 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.82200895, 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:04:15.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.504 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.504 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:15.504 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.821972661, 2024-09-13T07:04:15.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:04:15.504 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.504 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.504 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:04:15.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.504 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:15.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:15.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.378869328999997, 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:04:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:04:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:04:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:04:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:04:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.378852447, 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:04:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.513 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.513 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:04:15.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.513 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:15.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:04:15.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.377801890000001, 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:04:15.520 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.376127341, 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:04:15.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:15.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:04:15.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:04:15.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:15.525 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.855544950999999, 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:04:15.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:04:15.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:04:15.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:04:15.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:04:15.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:04:15.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172880 2.2 2:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e149)", 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.301248704999999, 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.97218948500000002, 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172880 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.530 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137687+0000", 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:15.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137702+0000", 2024-09-13T07:04:15.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5023e-05 2024-09-13T07:04:15.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137730+0000", 2024-09-13T07:04:15.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7719e-05 2024-09-13T07:04:15.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137748+0000", 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8189000000000002e-05 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137891+0000", 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014330699999999999 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137903+0000", 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1688999999999999e-05 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142887+0000", 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004984006 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142898+0000", 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.079e-05 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109703+0000", 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96680553899999999 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109879+0000", 2024-09-13T07:04:15.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017579800000000001 2024-09-13T07:04:15.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.535 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.535 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.535 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:15.536 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:04:15.537 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:15.533+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:15.637 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:15.633+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:15.664 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:04:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:15.692 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:15.692 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:15.701 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:04:15.737 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:15.738+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:15.809 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:15.809 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:15.809 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:15.809 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:15.809 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:15.816 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:15.819 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:04:15.838 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:15.838+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:15.838 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:15.866 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:15.886 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:15.910 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:15.933 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:15.934 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:15.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:15.934 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:15.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.934 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:04:15.934 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:15.934 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 169.87815606199999, 2024-09-13T07:04:15.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:04:15.934 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.934 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:15.934 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:04:15.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:04:15.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:04:15.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:04:15.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:04:15.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:04:15.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:04:15.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:15.938 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 137.72027482799999, 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:04:15.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:04:15.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:15.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 137.69450511700001, 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:04:15.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:04:15.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:04:15.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:04:15.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:04:15.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 137.673353951, 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.950 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:04:15.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:15.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:04:15.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:04:15.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:04:15.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:04:15.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:04:15.955 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 127.556309188, 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:15.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:15.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:15.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:15.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:04:15.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:04:15.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:15.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:04:15.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:04:15.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:15.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:04:15.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:04:15.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.271311943, 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.961 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:04:15.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:04:15.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:04:15.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:04:15.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:04:15.965 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.965 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.965 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.965 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:15.965 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 104.01960280500001, 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:04:15.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:04:15.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:04:15.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:04:15.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:04:15.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:04:15.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:04:15.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:04:15.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:04:15.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:04:15.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:04:15.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:04:15.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 104.019584771, 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.970 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:04:15.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:04:15.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:04:15.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:04:15.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:04:15.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 104.019568691, 2024-09-13T07:04:15.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:04:15.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.978 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 104.019553015, 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:04:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:04:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 104.019530093, 2024-09-13T07:04:15.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:04:15.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:04:15.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:04:15.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:04:15.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:04:15.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 104.019513996, 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:04:15.986 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 104.018912384, 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.990 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:04:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:04:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 104.018869608, 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:04:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:04:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.861851838000007, 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:04:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:04:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:04:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:04:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:04:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 62.009474908000001, 2024-09-13T07:04:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:04:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:04:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:04:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:04:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:04:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:04:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:04:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:04:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:04:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:164179 3.14 e135/67 3:2fc485bd:::rbd_data.11942e4f5fc7.0000000000000002:head v 135'401, mlcod=132'398)", 2024-09-13T07:04:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.986534454000001, 2024-09-13T07:04:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063492174999999998, 2024-09-13T07:04:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945792+0000", 2024-09-13T07:04:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954811+0000", 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090187940000000001 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954840+0000", 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8659000000000001e-05 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954852+0000", 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1966e-05 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954938+0000", 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6539000000000006e-05 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954975+0000", 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6303999999999997e-05 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009206+0000", 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.054231044999999999 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009286+0000", 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700999999999999e-05 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:165953 3.d 3:b3a20b45:::rbd_data.11942e4f5fc7.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8796117~39260] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.710086896, 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.095290105, 2024-09-13T07:04:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 165953 2024-09-13T07:04:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222240+0000", 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258929+0000", 2024-09-13T07:04:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.036688401000000002 2024-09-13T07:04:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258971+0000", 2024-09-13T07:04:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2212000000000002e-05 2024-09-13T07:04:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258989+0000", 2024-09-13T07:04:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7674000000000001e-05 2024-09-13T07:04:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259072+0000", 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3656e-05 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259297+0000", 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022450700000000001 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.260463+0000", 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011665340000000001 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.316214+0000", 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.055750268999999998 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317335+0000", 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011211719999999999 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317451+0000", 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011616 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317532+0000", 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0860999999999997e-05 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172852 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [call lock.unlock in=52b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.733629749000002, 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.90390594199999996, 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172852 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198696+0000", 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198710+0000", 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3548e-05 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198739+0000", 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9606000000000001e-05 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198752+0000", 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3033e-05 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198869+0000", 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116552 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199018+0000", 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014923700000000001 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199308+0000", 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028947199999999998 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.200731+0000", 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014229799999999999 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059443+0000", 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.85871189999999997 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059467+0000", 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.478e-05 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102380+0000", 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042912463999999997 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102605+0000", 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022489499999999999 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172853 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [watch reconnect cookie 140335574242864 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.881880890000001, 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.056267725999999997, 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172853 2024-09-13T07:04:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050445+0000", 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050461+0000", 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5783000000000001e-05 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050497+0000", 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5995999999999999e-05 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050516+0000", 2024-09-13T07:04:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9579999999999999e-05 2024-09-13T07:04:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050637+0000", 2024-09-13T07:04:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120536 2024-09-13T07:04:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050647+0000", 2024-09-13T07:04:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6989999999999998e-06 2024-09-13T07:04:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059533+0000", 2024-09-13T07:04:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088858749999999997 2024-09-13T07:04:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059545+0000", 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2473000000000001e-05 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102962+0000", 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043416656999999997 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.103612+0000", 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000650374 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.106715+0000", 2024-09-13T07:04:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031033229999999998 2024-09-13T07:04:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:16.028 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:16.029 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:16.029 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:16.030 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:16.029+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:16.031 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:16.031 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:04:16.031 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:16.031 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:16.032 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:16.032 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:16.032 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:16.032 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:16.032 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:16.032 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:16.032 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:16.131 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:16.129+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:16.187 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:16.187 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:16.198 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:04:16.232 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:16.233+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:16.299 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:16.299 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:16.309 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:04:16.332 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:16.330+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:16.426 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:16.426 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:16.433 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:16.429+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:16.437 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:04:16.535 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:16.534+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:16.595 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:16.596 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:16.605 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:04:16.635 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:16.633+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:16.705 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:16.705 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:16.715 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:04:16.736 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:16.733+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:16.824 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:16.825 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:16.834 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:04:16.836 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:16.833+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:16.937 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:16.937+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:16.947 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:16.947 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:16.957 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:04:17.038 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:17.038+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:17.064 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:17.065 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:17.074 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:04:17.139 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:17.138+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:17.181 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:17.181 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:17.190 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:04:17.240 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:17.237+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:17.298 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:17.298 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:17.308 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:04:17.341 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:17.342+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:17.376 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:04:17.438 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:17.438 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:17.441 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:17.437+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:17.452 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:04:17.542 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:17.542+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:17.568 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:17.568 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:17.578 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:04:17.642 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:17.642+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:17.700 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:17.701 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:17.711 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:04:17.744 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:17.742+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:17.813 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:17.813 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:17.823 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:04:17.844 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:17.841+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:17.926 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:17.926 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:17.936 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:04:17.945 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:17.946+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:18.033 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:04:18.039 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:18.039 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:18.049 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:04:18.081 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:18.082+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:18.147 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:18.146+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:18.158 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:18.158 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:18.168 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:04:18.187 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:18.210 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:18.233 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:18.248 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:18.245+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:18.252 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:18.270 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:18.285 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:18.286 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:18.292 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:18.296 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:04:18.315 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:18.315 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:18.315 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:18.315 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:18.315 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:18.315 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:18.315 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:18.315 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:18.315 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:18.318 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:18.318 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:18.349 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:18.350+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:18.383 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:04:18.450 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:18.445+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:18.453 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:18.454 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:18.454 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:18.454 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:18.464 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:04:18.551 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:18.550+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:18.568 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:18.568 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:18.568 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:18.568 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:18.568 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:18.578 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:04:18.652 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:18.650+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:18.690 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:04:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 172.63471704599999, 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:04:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:04:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:04:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:04:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:04:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:04:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:04:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:04:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:04:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 140.47683581199999, 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:04:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:04:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:04:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:04:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:04:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 140.45106610100001, 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:04:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:04:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:04:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:04:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:04:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:04:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 140.429914935, 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 130.312870172, 2024-09-13T07:04:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:04:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:04:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:04:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:04:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:04:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:04:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:04:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:04:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:04:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.027872927, 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:04:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:04:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:04:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:04:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:04:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:04:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:04:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:04:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.77616378899999, 2024-09-13T07:04:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:04:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:04:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:04:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:04:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:04:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:04:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:04:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:04:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.776145755, 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:04:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:04:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:04:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:04:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:04:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:04:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.77612967499999, 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:04:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.776113999, 2024-09-13T07:04:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:04:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:04:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:04:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:04:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:04:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:04:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:04:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:04:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.776091077, 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:04:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:04:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:04:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:04:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.77607498, 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:04:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:04:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:04:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:04:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:04:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:04:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.77547336799999, 2024-09-13T07:04:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:04:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:04:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.77543059200001, 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:04:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:04:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:04:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:04:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:04:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:04:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:04:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:04:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:04:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:04:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 96.618412821999996, 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.766035892000005, 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:04:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:04:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:04:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:04:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:04:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:04:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:04:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:04:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:04:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:164179 3.14 e135/67 3:2fc485bd:::rbd_data.11942e4f5fc7.0000000000000002:head v 135'401, mlcod=132'398)", 2024-09-13T07:04:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 49.743095437999997, 2024-09-13T07:04:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063492174999999998, 2024-09-13T07:04:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945792+0000", 2024-09-13T07:04:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954811+0000", 2024-09-13T07:04:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090187940000000001 2024-09-13T07:04:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954840+0000", 2024-09-13T07:04:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8659000000000001e-05 2024-09-13T07:04:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954852+0000", 2024-09-13T07:04:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1966e-05 2024-09-13T07:04:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954938+0000", 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6539000000000006e-05 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954975+0000", 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6303999999999997e-05 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009206+0000", 2024-09-13T07:04:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.054231044999999999 2024-09-13T07:04:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009286+0000", 2024-09-13T07:04:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700999999999999e-05 2024-09-13T07:04:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:165953 3.d 3:b3a20b45:::rbd_data.11942e4f5fc7.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8796117~39260] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 49.466647879999996, 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.095290105, 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 165953 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222240+0000", 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258929+0000", 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.036688401000000002 2024-09-13T07:04:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258971+0000", 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2212000000000002e-05 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258989+0000", 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7674000000000001e-05 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259072+0000", 2024-09-13T07:04:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3656e-05 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259297+0000", 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022450700000000001 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.260463+0000", 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011665340000000001 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.316214+0000", 2024-09-13T07:04:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.055750268999999998 2024-09-13T07:04:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317335+0000", 2024-09-13T07:04:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011211719999999999 2024-09-13T07:04:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317451+0000", 2024-09-13T07:04:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011616 2024-09-13T07:04:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317532+0000", 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0860999999999997e-05 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172852 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [call lock.unlock in=52b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.490190732999999, 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.90390594199999996, 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172852 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198696+0000", 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198710+0000", 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3548e-05 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198739+0000", 2024-09-13T07:04:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9606000000000001e-05 2024-09-13T07:04:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198752+0000", 2024-09-13T07:04:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3033e-05 2024-09-13T07:04:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198869+0000", 2024-09-13T07:04:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116552 2024-09-13T07:04:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199018+0000", 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014923700000000001 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199308+0000", 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028947199999999998 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.200731+0000", 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014229799999999999 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059443+0000", 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.85871189999999997 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059467+0000", 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.478e-05 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102380+0000", 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042912463999999997 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102605+0000", 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022489499999999999 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172853 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [watch reconnect cookie 140335574242864 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.638441874000002, 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.056267725999999997, 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172853 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050445+0000", 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050461+0000", 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5783000000000001e-05 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050497+0000", 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5995999999999999e-05 2024-09-13T07:04:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050516+0000", 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9579999999999999e-05 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050637+0000", 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120536 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050647+0000", 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6989999999999998e-06 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059533+0000", 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088858749999999997 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059545+0000", 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2473000000000001e-05 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102962+0000", 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043416656999999997 2024-09-13T07:04:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.103612+0000", 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000650374 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.106715+0000", 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031033229999999998 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:18.785 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:18.787 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:18.785+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:18.787 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:04:18.888 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:18.885+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:18.903 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:18.903 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:18.903 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:18.913 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:04:18.989 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:18.990+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:19.022 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:19.022 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:19.022 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:19.022 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:19.022 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:19.032 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:04:19.090 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:19.090+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:19.138 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:19.138 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:19.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137474 3.6 3:65c31370:::rbd_data.1194af11889b.000000000000013b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 186185~11715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 74.393533887999993, 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0054357140000001, 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137474 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:19.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:19.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.140 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.140 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.140 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.140 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743934+0000", 2024-09-13T07:04:19.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743949+0000", 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5705999999999998e-05 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743960+0000", 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0084e-05 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743971+0000", 2024-09-13T07:04:19.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0942e-05 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744045+0000", 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4626e-05 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744062+0000", 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7339000000000001e-05 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746366+0000", 2024-09-13T07:04:19.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.002303457 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746520+0000", 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015385900000000001 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746734+0000", 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000214253 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749128+0000", 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002394129 2024-09-13T07:04:19.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749267+0000", 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013878299999999999 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749328+0000", 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0918000000000002e-05 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749371+0000", 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3188000000000001e-05 2024-09-13T07:04:19.144 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.145 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.145 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.145 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163305 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2387968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.145 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:19.145 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.459315406999998, 2024-09-13T07:04:19.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044745869000000001, 2024-09-13T07:04:19.145 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.145 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.145 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.145 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.145 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163305 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678153+0000", 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678156+0000", 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6350000000000002e-06 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678161+0000", 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678166+0000", 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.498e-06 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722751+0000", 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044585134999999998 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722836+0000", 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4672999999999994e-05 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722900+0000", 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4042999999999997e-05 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163307 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.459294943000003, 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045060944999999998, 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.149 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163307 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678177+0000", 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678183+0000", 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0599999999999996e-06 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678188+0000", 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722941+0000", 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044752844999999999 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723073+0000", 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013128599999999999 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723235+0000", 2024-09-13T07:04:19.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000162671 2024-09-13T07:04:19.153 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.153 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.153 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.153 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163310 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 544768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.153 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:19.153 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.459275990999998, 2024-09-13T07:04:19.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045290534, 2024-09-13T07:04:19.153 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163310 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.154 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678195+0000", 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.785e-06 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678201+0000", 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678205+0000", 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6999999999999999e-06 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723299+0000", 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045093964 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723384+0000", 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4562999999999997e-05 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723484+0000", 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100045 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163313 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1093632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:19.157 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.459260006000001, 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045562975999999998, 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163313 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678212+0000", 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8650000000000001e-06 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678217+0000", 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2660000000000001e-06 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678221+0000", 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-06 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723552+0000", 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045330489000000002 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.160 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723660+0000", 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010770800000000001 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723772+0000", 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112608 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163317 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1642496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.161 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.459231197999998, 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045770165000000002, 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163317 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:19.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678237+0000", 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678241+0000", 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2669999999999998e-06 2024-09-13T07:04:19.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678247+0000", 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7309999999999996e-06 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678252+0000", 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723830+0000", 2024-09-13T07:04:19.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045577234000000001 2024-09-13T07:04:19.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723936+0000", 2024-09-13T07:04:19.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106277 2024-09-13T07:04:19.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724008+0000", 2024-09-13T07:04:19.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2422999999999994e-05 2024-09-13T07:04:19.165 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.165 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.165 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163320 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.459210773999999, 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045968883000000002, 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163320 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.166 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:19.167 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678261+0000", 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8090000000000001e-06 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678267+0000", 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8860000000000001e-06 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678271+0000", 2024-09-13T07:04:19.168 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4549999999999997e-06 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724061+0000", 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045789656999999997 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724151+0000", 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9734999999999994e-05 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724227+0000", 2024-09-13T07:04:19.169 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6848999999999996e-05 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163324 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.458926441000003, 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045988788000000003, 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.170 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163324 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678542+0000", 2024-09-13T07:04:19.171 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678547+0000", 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6829999999999999e-06 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678556+0000", 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1929999999999999e-06 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.172 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678562+0000", 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.083e-06 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724267+0000", 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045705427 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724417+0000", 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014984699999999999 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.173 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724532+0000", 2024-09-13T07:04:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011430900000000001 2024-09-13T07:04:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163327 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 733184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.458901406999999, 2024-09-13T07:04:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046185902000000001, 2024-09-13T07:04:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163327 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.175 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678567+0000", 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678571+0000", 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3799999999999998e-06 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678576+0000", 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7760000000000003e-06 2024-09-13T07:04:19.176 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678581+0000", 2024-09-13T07:04:19.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8640000000000004e-06 2024-09-13T07:04:19.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724561+0000", 2024-09-13T07:04:19.177 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045979178000000002 2024-09-13T07:04:19.177 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.177 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.177 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.177 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724649+0000", 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8677000000000005e-05 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724754+0000", 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104527 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163331 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:19.178 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.458883253000003, 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046361773000000002, 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163331 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.179 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678585+0000", 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678588+0000", 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0469999999999998e-06 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.180 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678595+0000", 2024-09-13T07:04:19.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0399999999999998e-06 2024-09-13T07:04:19.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678599+0000", 2024-09-13T07:04:19.181 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:04:19.181 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.181 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.181 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.181 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724780+0000", 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046181168000000002 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724871+0000", 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0562000000000005e-05 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724948+0000", 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6985999999999998e-05 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.182 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163334 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1662976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.458866049999997, 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046544502000000001, 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163334 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.183 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.184 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678605+0000", 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5380000000000001e-06 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678611+0000", 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7389999999999996e-06 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678615+0000", 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.185 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724987+0000", 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046371969999999998 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725075+0000", 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8091999999999996e-05 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725148+0000", 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2206999999999995e-05 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.186 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163338 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.458849319999999, 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046738452999999999, 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163338 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.187 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678619+0000", 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.188 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678622+0000", 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.649e-06 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678628+0000", 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5029999999999997e-06 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678632+0000", 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4020000000000004e-06 2024-09-13T07:04:19.189 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725182+0000", 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046550389999999997 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725280+0000", 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7260000000000001e-05 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725358+0000", 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8720000000000005e-05 2024-09-13T07:04:19.190 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163339 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2363392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.458833792, 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046920835000000001, 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163339 2024-09-13T07:04:19.191 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:19.192 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678637+0000", 2024-09-13T07:04:19.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2479999999999999e-06 2024-09-13T07:04:19.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.193 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678642+0000", 2024-09-13T07:04:19.193 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0030000000000002e-06 2024-09-13T07:04:19.193 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.193 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.193 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678646+0000", 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9870000000000001e-06 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725397+0000", 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046750127000000002 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725482+0000", 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5928999999999997e-05 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.194 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725556+0000", 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3823000000000001e-05 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163342 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.458819153, 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047212991000000003, 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.195 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163342 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.196 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678652+0000", 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.486e-06 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678657+0000", 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6909999999999996e-06 2024-09-13T07:04:19.197 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678660+0000", 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8009999999999999e-06 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725597+0000", 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046937372999999998 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725745+0000", 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000148314 2024-09-13T07:04:19.198 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725863+0000", 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011768 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163345 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.457957589999999, 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046569193000000002, 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.199 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163345 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.200 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678662+0000", 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9991498 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679516+0000", 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000854789 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679530+0000", 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3492999999999999e-05 2024-09-13T07:04:19.201 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679537+0000", 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.323e-06 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725892+0000", 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046354244000000003 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725975+0000", 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3356000000000006e-05 2024-09-13T07:04:19.202 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726081+0000", 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105992 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163349 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1298432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.457925527, 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046744247000000003, 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.203 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163349 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.204 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679543+0000", 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679546+0000", 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.205 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679551+0000", 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679554+0000", 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8629999999999999e-06 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726117+0000", 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046563081999999999 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.206 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726213+0000", 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6207999999999998e-05 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726288+0000", 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4741999999999997e-05 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163350 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1449984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.457912901, 2024-09-13T07:04:19.207 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046935635000000003, 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163350 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.208 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679558+0000", 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.928e-06 2024-09-13T07:04:19.209 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679561+0000", 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7659999999999999e-06 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679564+0000", 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4960000000000001e-06 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726327+0000", 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046762321000000003 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.210 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726415+0000", 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8487999999999997e-05 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726492+0000", 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6995000000000004e-05 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163353 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:19.211 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.457902791999999, 2024-09-13T07:04:19.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047130755000000003, 2024-09-13T07:04:19.212 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.212 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.212 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.212 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.212 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.212 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163353 2024-09-13T07:04:19.212 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.212 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.212 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.212 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.212 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:19.212 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679568+0000", 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5969999999999999e-06 2024-09-13T07:04:19.213 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679571+0000", 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679573+0000", 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726523+0000", 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046949882999999998 2024-09-13T07:04:19.214 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726616+0000", 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3236e-05 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726697+0000", 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0834000000000006e-05 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.215 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163356 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2379776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 50.457893110000001, 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047304662999999997, 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163356 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.216 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679578+0000", 2024-09-13T07:04:19.217 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679582+0000", 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679589+0000", 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1149999999999996e-06 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726723+0000", 2024-09-13T07:04:19.218 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047133768999999999 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726809+0000", 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5402000000000007e-05 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726881+0000", 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2152999999999998e-05 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.219 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:166663 3.b 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8129196~12979] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 49.739518596000003, 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.061564718999999997, 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 166663 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:19.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397947+0000", 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999967 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412144+0000", 2024-09-13T07:04:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014196972 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412172+0000", 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7781999999999999e-05 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412182+0000", 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6069999999999997e-06 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412239+0000", 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.732e-05 2024-09-13T07:04:19.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412365+0000", 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125356 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.413178+0000", 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081355299999999995 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.443176+0000", 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029997903999999999 2024-09-13T07:04:19.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459446+0000", 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016269921999999999 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459484+0000", 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8488000000000001e-05 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459515+0000", 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0981e-05 2024-09-13T07:04:19.224 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.225 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.225 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.225 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.225 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.225 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:19.226 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:04:19.227 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:19.226+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:19.327 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:19.326+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:19.377 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:19.378 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:19.378 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:19.378 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:19.388 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:04:19.433 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:19.434+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:19.455 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 7, 2, 0, 1] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-13T07:04:19.455 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:04:19.456 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [3, 4, 5, 6, 7, 2, 0, 1] 2024-09-13T07:04:19.456 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-09-13T07:04:19.495 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:19.495 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:19.495 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:19.495 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:19.495 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:19.506 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:04:19.528 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:19.526+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:19.611 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:04:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.118100067, 2024-09-13T07:04:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:04:19.613 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:04:19.614 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.615 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:04:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:04:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:04:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:04:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.616 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.631616503, 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:19.617 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:04:19.618 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:04:19.619 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.620 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.631546574, 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.621 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.622 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.623 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.624 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.63153070199999, 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.625 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.626 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.627 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.628 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.631460245, 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.629 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.630 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.631 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:04:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:04:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:04:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:04:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:04:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:04:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.628108207, 2024-09-13T07:04:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:04:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.633 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:19.634 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:04:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:04:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:04:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:04:19.635 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:04:19.636 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.62809190600001, 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.637 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:04:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.638 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.639 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.640 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.628072121, 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.641 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.642 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:04:19.643 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:04:19.644 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.628055848, 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.645 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:04:19.646 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.647 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.62803802299999, 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.649 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:04:19.650 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:04:19.651 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.62802192999999, 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:04:19.652 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.653 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.654 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.655 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.62800074399999, 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.656 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.657 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:04:19.658 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:04:19.659 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.62798471100001, 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.660 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.661 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:04:19.662 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.663 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.62793547199999, 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.664 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.665 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.666 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.667 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.62791741199999, 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:19.668 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.669 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.670 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:19.671 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.62790089699999, 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.672 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.673 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:04:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:04:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:04:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:04:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:04:19.674 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.627882532, 2024-09-13T07:04:19.675 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.676 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:04:19.677 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:04:19.678 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.627866434, 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.679 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.680 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.681 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.682 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.627850499, 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.683 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.684 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:04:19.685 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:04:19.686 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 141.62783400199999, 2024-09-13T07:04:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:04:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:19.687 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.688 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:04:19.689 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:04:19.690 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:04:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:19.691 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:19.691 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:19.692 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:04:19.693 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:19.689+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:19.765 INFO:tasks.ceph.mon.a.smithi118.stderr:2024-09-13T07:04:19.766+0000 7fd1ec319700 -1 mon.a@0(leader).osd e184 definitely_dead 0 2024-09-13T07:04:19.793 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:19.789+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:19.841 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:19.841 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:19.841 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:19.841 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:19.850 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:04:19.894 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:19.894+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:19.952 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:19.952 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:19.952 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:19.952 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:19.953 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:19.962 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:04:19.994 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:19.993+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:20.066 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:20.067 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:20.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:20.067 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:20.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.067 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:04:20.067 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:20.067 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 174.01057640600001, 2024-09-13T07:04:20.067 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:04:20.067 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.067 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:20.067 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.067 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.068 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:04:20.069 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:04:20.070 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:04:20.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.071 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.071 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:04:20.071 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:04:20.071 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.071 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.071 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.071 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.071 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.071 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:20.071 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:20.071 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 90.192360049000001, 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.072 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:20.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:04:20.073 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:20.073 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.073 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.073 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.073 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.074 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.075 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 90.192310660999993, 2024-09-13T07:04:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:04:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.077 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.078 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.079 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 90.192287953999994, 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:20.080 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:20.081 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:04:20.082 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.083 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:20.084 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 90.192243761, 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.085 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:20.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:04:20.086 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:20.086 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.086 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.086 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.086 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.087 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:04:20.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:04:20.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:04:20.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:04:20.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.088 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.088 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.088 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:04:20.088 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:04:20.088 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.088 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 90.161264672000001, 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:20.089 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:20.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.090 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:20.090 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:20.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.090 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.090 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.091 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:04:20.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:04:20.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:04:20.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:04:20.092 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.092 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.092 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:20.092 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:04:20.092 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137457 3.18 3:1a596dcc:::rbd_data.1194af11889b.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 75.415642861999999, 2024-09-13T07:04:20.093 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033734403000000003, 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137457 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.094 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651523+0000", 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651531+0000", 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.328e-06 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.095 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651545+0000", 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4270000000000001e-05 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651555+0000", 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5210000000000008e-06 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651626+0000", 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1915999999999994e-05 2024-09-13T07:04:20.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651761+0000", 2024-09-13T07:04:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000134838 2024-09-13T07:04:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.658577+0000", 2024-09-13T07:04:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068151820000000004 2024-09-13T07:04:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685181+0000", 2024-09-13T07:04:20.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026604200000000001 2024-09-13T07:04:20.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:20.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685193+0000", 2024-09-13T07:04:20.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2128e-05 2024-09-13T07:04:20.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.098 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:20.098 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685226+0000", 2024-09-13T07:04:20.098 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3504999999999999e-05 2024-09-13T07:04:20.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.098 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685259+0000", 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2866e-05 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138565 3.0 3:002d0425:::rbd_data.11949f62878b.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.402272370000006, 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033394489999999999, 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.099 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138565 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.100 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:20.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664893+0000", 2024-09-13T07:04:20.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:20.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.101 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.101 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.101 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664906+0000", 2024-09-13T07:04:20.101 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2996000000000001e-05 2024-09-13T07:04:20.101 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664936+0000", 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9609999999999999e-05 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664949+0000", 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383e-05 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665046+0000", 2024-09-13T07:04:20.102 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6459000000000006e-05 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665259+0000", 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000213141 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.672522+0000", 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072632110000000003 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.697982+0000", 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025459699999999998 2024-09-13T07:04:20.103 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698018+0000", 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6375999999999999e-05 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698166+0000", 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014818700000000001 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698290+0000", 2024-09-13T07:04:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012333999999999999 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138587 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 71.353806703000004, 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019224502000000001, 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.105 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138587 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.106 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713360+0000", 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713368+0000", 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713383+0000", 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5176e-05 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.107 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713390+0000", 2024-09-13T07:04:20.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8669999999999996e-06 2024-09-13T07:04:20.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713449+0000", 2024-09-13T07:04:20.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8938999999999999e-05 2024-09-13T07:04:20.108 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.108 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.108 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.108 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713549+0000", 2024-09-13T07:04:20.108 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010009 2024-09-13T07:04:20.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:20.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.716087+0000", 2024-09-13T07:04:20.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025383820000000001 2024-09-13T07:04:20.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:20.109 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732351+0000", 2024-09-13T07:04:20.109 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016263564000000001 2024-09-13T07:04:20.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.109 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.109 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:20.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732450+0000", 2024-09-13T07:04:20.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9529000000000002e-05 2024-09-13T07:04:20.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:20.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732532+0000", 2024-09-13T07:04:20.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1526000000000005e-05 2024-09-13T07:04:20.110 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.110 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.110 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.110 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732585+0000", 2024-09-13T07:04:20.110 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3819000000000001e-05 2024-09-13T07:04:20.110 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.111 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.111 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.111 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.111 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.111 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138786 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:04:20.111 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:20.111 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.919591448000006, 2024-09-13T07:04:20.111 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018847452000000001, 2024-09-13T07:04:20.111 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.111 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:20.111 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.111 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.111 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.112 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138786 2024-09-13T07:04:20.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.112 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:20.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.112 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.112 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.112 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:20.112 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147575+0000", 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147583+0000", 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147601+0000", 2024-09-13T07:04:20.113 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7961e-05 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147610+0000", 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147661+0000", 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0447e-05 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.114 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147754+0000", 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2974999999999997e-05 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.150580+0000", 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002826566 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166135+0000", 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015554772 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:20.115 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166225+0000", 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0258000000000002e-05 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166334+0000", 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010816500000000001 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166424+0000", 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0116000000000003e-05 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.116 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.117 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.117 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:139127 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:04:20.117 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:20.117 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 70.292540484, 2024-09-13T07:04:20.117 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019295931999999998, 2024-09-13T07:04:20.117 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.117 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:20.117 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.117 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.117 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.117 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139127 2024-09-13T07:04:20.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.118 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774626+0000", 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774635+0000", 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5419999999999994e-06 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774654+0000", 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8359000000000002e-05 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.119 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774663+0000", 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774718+0000", 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5315000000000001e-05 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774805+0000", 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7284e-05 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.779319+0000", 2024-09-13T07:04:20.120 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045138310000000003 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793772+0000", 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014452692 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793821+0000", 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9563e-05 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793878+0000", 2024-09-13T07:04:20.121 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6953999999999997e-05 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793923+0000", 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4781999999999999e-05 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:164691 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.008580610000003, 2024-09-13T07:04:20.122 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015985102000000001, 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 164691 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.123 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058589+0000", 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2510000000000001e-06 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.124 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058594+0000", 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9089999999999999e-06 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058598+0000", 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2069999999999997e-06 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064175+0000", 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055769909999999999 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.125 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064210+0000", 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4966999999999999e-05 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074572+0000", 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010362219000000001 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.126 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165201 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.006052681, 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013054266, 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165201 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061115+0000", 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061117+0000", 2024-09-13T07:04:20.128 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.12e-06 2024-09-13T07:04:20.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061125+0000", 2024-09-13T07:04:20.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8539999999999997e-06 2024-09-13T07:04:20.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.129 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061129+0000", 2024-09-13T07:04:20.129 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:04:20.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.129 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.129 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074092+0000", 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012963166999999999 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074136+0000", 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3578000000000002e-05 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074169+0000", 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3581000000000003e-05 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.130 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165202 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7831552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.006035005999998, 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013116245, 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165202 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.131 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061132+0000", 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.132 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061135+0000", 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3410000000000001e-06 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061142+0000", 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7079999999999997e-06 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061147+0000", 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8930000000000004e-06 2024-09-13T07:04:20.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074180+0000", 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013032346 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074217+0000", 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074249+0000", 2024-09-13T07:04:20.134 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1711000000000002e-05 2024-09-13T07:04:20.135 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.135 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.135 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.135 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.135 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165203 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:20.135 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:20.135 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.006016891999998, 2024-09-13T07:04:20.135 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013180211000000001, 2024-09-13T07:04:20.135 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.135 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:20.135 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.135 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165203 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.136 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061150+0000", 2024-09-13T07:04:20.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:20.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061153+0000", 2024-09-13T07:04:20.137 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.108e-06 2024-09-13T07:04:20.137 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.137 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.137 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.137 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061158+0000", 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0649999999999998e-06 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061162+0000", 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0080000000000004e-06 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074262+0000", 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013100557000000001 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.138 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074299+0000", 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6981999999999999e-05 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074331+0000", 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1801000000000003e-05 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165204 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8093696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:20.139 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:20.140 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.006002180999999, 2024-09-13T07:04:20.140 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0132388, 2024-09-13T07:04:20.140 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.140 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:20.140 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.140 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165204 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.141 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061167+0000", 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1409999999999999e-06 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061175+0000", 2024-09-13T07:04:20.142 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5519999999999999e-06 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061179+0000", 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.172e-06 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074342+0000", 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013162821999999999 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.143 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074373+0000", 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1103999999999999e-05 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074404+0000", 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1359999999999998e-05 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165205 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:20.144 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.005985222, 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013300635, 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165205 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061184+0000", 2024-09-13T07:04:20.146 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0779999999999998e-06 2024-09-13T07:04:20.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061192+0000", 2024-09-13T07:04:20.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7209999999999994e-06 2024-09-13T07:04:20.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.147 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.147 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.147 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061196+0000", 2024-09-13T07:04:20.147 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4469999999999999e-06 2024-09-13T07:04:20.147 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074417+0000", 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013220750999999999 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074452+0000", 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4814999999999998e-05 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074483+0000", 2024-09-13T07:04:20.148 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1118000000000003e-05 2024-09-13T07:04:20.149 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.149 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.149 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.149 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.149 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.149 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165206 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:20.149 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:20.149 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 51.005967101000003, 2024-09-13T07:04:20.149 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013364454, 2024-09-13T07:04:20.149 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.149 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:20.150 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.150 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.150 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.150 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165206 2024-09-13T07:04:20.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.150 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.150 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.150 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.150 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:20.150 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.150 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:20.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061200+0000", 2024-09-13T07:04:20.151 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:20.151 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.151 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.151 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.151 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061203+0000", 2024-09-13T07:04:20.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.559e-06 2024-09-13T07:04:20.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061210+0000", 2024-09-13T07:04:20.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-13T07:04:20.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.152 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.152 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061214+0000", 2024-09-13T07:04:20.152 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0999999999999997e-06 2024-09-13T07:04:20.152 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.152 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074494+0000", 2024-09-13T07:04:20.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013279094 2024-09-13T07:04:20.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074530+0000", 2024-09-13T07:04:20.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6538999999999997e-05 2024-09-13T07:04:20.153 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.153 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.153 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.153 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074565+0000", 2024-09-13T07:04:20.153 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4972999999999999e-05 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169377 3.0 3:07b65a69:::rbd_data.1194e181a50b.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 47.710300705999998, 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.032744151999999999, 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.154 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169377 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.155 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356864+0000", 2024-09-13T07:04:20.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:20.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356880+0000", 2024-09-13T07:04:20.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6022999999999999e-05 2024-09-13T07:04:20.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.156 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.156 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.156 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356916+0000", 2024-09-13T07:04:20.156 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5921999999999999e-05 2024-09-13T07:04:20.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356931+0000", 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4311000000000001e-05 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357011+0000", 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9992000000000003e-05 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357143+0000", 2024-09-13T07:04:20.157 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000131822 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.361578+0000", 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00443525 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389466+0000", 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027888342999999999 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:20.158 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389529+0000", 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2460999999999997e-05 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389574+0000", 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5405999999999999e-05 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389611+0000", 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7122000000000001e-05 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.159 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.160 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.160 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169731 3.0 3:031f8e93:::rbd_data.1194e181a50b.0000000000000006:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-13T07:04:20.160 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:20.160 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 46.785874907999997, 2024-09-13T07:04:20.160 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.021389756999999999, 2024-09-13T07:04:20.160 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:20.160 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:20.160 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:20.160 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:20.160 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:20.160 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169731 2024-09-13T07:04:20.160 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:20.161 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281291+0000", 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281302+0000", 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0651e-05 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281324+0000", 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1713999999999999e-05 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:20.162 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281340+0000", 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281450+0000", 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110086 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281593+0000", 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014306 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:20.163 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.286356+0000", 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047626320000000002 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302404+0000", 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016048605000000001 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302476+0000", 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1483000000000007e-05 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.164 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:20.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302578+0000", 2024-09-13T07:04:20.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101883 2024-09-13T07:04:20.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:20.165 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:20.165 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:20.165 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302683+0000", 2024-09-13T07:04:20.165 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104681 2024-09-13T07:04:20.165 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.165 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.165 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.165 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:20.165 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:20.165 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:20.167 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:04:20.167 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:20.166+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:20.268 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:20.266+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:20.320 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:04:20.323 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:20.323 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:20.323 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:20.323 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:20.334 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:04:20.377 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:20.378+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:20.399 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:04:20.414 INFO:teuthology.orchestra.run.smithi118.stderr:marked out osd.7. 2024-09-13T07:04:20.470 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:20.470+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:20.499 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:20.504 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:20.504 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:20.505 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:20.505 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:20.505 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:20.516 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:04:20.526 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:20.547 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:20.566 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:20.571 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:20.569+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:20.588 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:20.610 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:20.613 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:20.613 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:20.623 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:04:20.631 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:20.632 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:20.632 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:20.632 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:20.632 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:20.632 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:20.632 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:20.632 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:20.632 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:20.634 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:20.635 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:20.672 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:20.670+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:20.739 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:20.739 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:20.750 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:04:20.773 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:20.770+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:20.854 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:20.854 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:20.864 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:04:20.873 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:20.874+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:20.971 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:20.971 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:20.974 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:20.973+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:20.981 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:04:21.075 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:21.073+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:21.093 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:21.093 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:21.103 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:04:21.176 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:21.173+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:21.210 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:21.210 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:21.220 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:04:21.277 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:21.278+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:21.323 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:21.323 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:21.333 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:04:21.378 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:21.378+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:21.419 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:04:21.448 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:21.448 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:21.458 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:04:21.478 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:21.477+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:21.562 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:21.563 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:21.572 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:04:21.579 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:21.577+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:21.677 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:21.677 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:21.679 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:21.677+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:21.686 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:04:21.780 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:21.778+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:21.803 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:21.803 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:21.813 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:04:21.881 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:21.882+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:21.921 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:21.921 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:21.931 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:04:21.982 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:21.981+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:22.081 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:22.081 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:22.083 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:22.081+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:22.090 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:04:22.184 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:22.182+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:22.196 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:22.197 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:22.206 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:04:22.285 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:22.286+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:22.315 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:22.315 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:22.325 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:04:22.386 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:22.381+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:22.486 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:22.485+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:22.493 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:22.493 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:22.504 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:04:22.587 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:22.586+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:22.619 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:22.619 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:22.630 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:04:22.635 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:04:22.688 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:22.686+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:22.788 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:22.785+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:22.812 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:22.812 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:22.825 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:04:22.836 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:22.859 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:22.890 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:22.890+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:22.936 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:22.936 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:22.946 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:04:22.991 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:22.989+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:23.087 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:23.091 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:23.090+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:23.108 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:23.108 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:23.108 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:23.108 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:23.108 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:23.121 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:04:23.135 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:23.160 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:23.180 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:23.181 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:23.181 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:23.181 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:23.181 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:23.181 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:23.181 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:23.181 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:23.181 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:23.184 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:23.184 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:23.192 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:23.189+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:23.227 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:23.227 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:23.227 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:23.227 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:23.227 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:23.237 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:04:23.293 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:23.289+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:23.342 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.219596029, 2024-09-13T07:04:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:04:23.342 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:04:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:04:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:04:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:04:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:04:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:04:23.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.346 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.212887357, 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:04:23.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:04:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:04:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:04:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:04:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.350 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.212874914, 2024-09-13T07:04:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:04:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.351 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:04:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:23.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:23.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:04:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:04:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:04:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:04:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:04:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:04:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:04:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:04:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:04:23.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:04:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:04:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:04:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:23.356 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.212857316, 2024-09-13T07:04:23.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:04:23.357 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.357 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.357 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.357 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.357 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.357 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:04:23.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.357 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:23.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:04:23.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:04:23.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:04:23.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:04:23.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:04:23.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:04:23.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:04:23.360 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.361 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.361 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.361 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.361 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:23.361 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.212846613, 2024-09-13T07:04:23.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:04:23.361 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.361 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.361 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.361 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.361 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:23.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:04:23.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:04:23.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:04:23.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:04:23.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:04:23.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:04:23.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:04:23.365 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.365 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.365 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.365 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.365 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:23.366 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.21280268, 2024-09-13T07:04:23.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:04:23.366 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.366 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.366 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.366 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:04:23.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.366 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.367 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:04:23.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:04:23.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:04:23.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:04:23.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:04:23.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:04:23.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:04:23.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.212782894, 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:04:23.370 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:23.371 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:04:23.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:04:23.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.212767144, 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.374 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:23.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:04:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:04:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:04:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:04:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:04:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:04:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.212755892, 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:04:23.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.212745293, 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:04:23.382 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:23.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:04:23.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:04:23.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:04:23.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:04:23.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:04:23.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:04:23.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.2127358, 2024-09-13T07:04:23.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.212726256, 2024-09-13T07:04:23.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:23.394 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.212693653, 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.21268062999999, 2024-09-13T07:04:23.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:23.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:23.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:23.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:04:23.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:04:23.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:04:23.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.402 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.20930936800001, 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:04:23.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:23.406 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 121.20928916699999, 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:23.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:04:23.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:04:23.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:04:23.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:04:23.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:04:23.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:04:23.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:04:23.410 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.411 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.411 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.411 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:166663 3.b e135/117 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head v 135'409, mlcod=132'407)", 2024-09-13T07:04:23.411 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:23.411 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 53.926700201999999, 2024-09-13T07:04:23.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045108144000000003, 2024-09-13T07:04:23.411 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.411 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:23.411 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:23.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:23.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414172+0000", 2024-09-13T07:04:23.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:23.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427200+0000", 2024-09-13T07:04:23.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013028098 2024-09-13T07:04:23.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427229+0000", 2024-09-13T07:04:23.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9178000000000001e-05 2024-09-13T07:04:23.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427239+0000", 2024-09-13T07:04:23.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6870000000000006e-06 2024-09-13T07:04:23.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427393+0000", 2024-09-13T07:04:23.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015382899999999999 2024-09-13T07:04:23.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427451+0000", 2024-09-13T07:04:23.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8301000000000003e-05 2024-09-13T07:04:23.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459232+0000", 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031780695999999997 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459282+0000", 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0701000000000001e-05 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.415 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:176785 3.7 3:e640995f:::rbd_data.1194987e4741.0000000000000011:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:09.441952+0000", 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.898922156999999, 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86669136400000002, 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 176785 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441952+0000", 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441952+0000", 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441951+0000", 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441958+0000", 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3089999999999998e-06 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441970+0000", 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2404e-05 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441977+0000", 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442051+0000", 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3337000000000006e-05 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442063+0000", 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2931e-05 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305282+0000", 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86321888999999996 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305447+0000", 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016446500000000001 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305579+0000", 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000132452 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.307846+0000", 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022664790000000001 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308562+0000", 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00071642899999999998 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308606+0000", 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4073999999999998e-05 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308644+0000", 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7348999999999998e-05 2024-09-13T07:04:23.421 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:176787 3.15 3:af329162:::rbd_data.1194987e4741.0000000000000013:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:09.442992+0000", 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.897882414, 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86944086799999998, 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 176787 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.422 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442992+0000", 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442992+0000", 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442990+0000", 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442999+0000", 2024-09-13T07:04:23.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8019999999999998e-06 2024-09-13T07:04:23.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443015+0000", 2024-09-13T07:04:23.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6002999999999999e-05 2024-09-13T07:04:23.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443024+0000", 2024-09-13T07:04:23.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3610000000000001e-06 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443092+0000", 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7967999999999997e-05 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443105+0000", 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3969e-05 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.303912+0000", 2024-09-13T07:04:23.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86080616200000004 2024-09-13T07:04:23.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.304190+0000", 2024-09-13T07:04:23.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00027807599999999999 2024-09-13T07:04:23.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:04:23.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.304362+0000", 2024-09-13T07:04:23.426 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017219199999999999 2024-09-13T07:04:23.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305588+0000", 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001226499 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312299+0000", 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0067103789999999998 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312377+0000", 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8291e-05 2024-09-13T07:04:23.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312433+0000", 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5755999999999999e-05 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:176791 3.7 3:e2fcfe96:::rbd_data.1194987e4741.0000000000000018:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:09.444146+0000", 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 13.896727953999999, 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86828808400000002, 2024-09-13T07:04:23.428 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:23.429 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:23.429 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:23.429 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:23.429 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.429 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 176791 2024-09-13T07:04:23.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.429 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:23.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:23.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444146+0000", 2024-09-13T07:04:23.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444146+0000", 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444145+0000", 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444151+0000", 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0050000000000001e-06 2024-09-13T07:04:23.430 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444168+0000", 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6399e-05 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444175+0000", 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5610000000000001e-06 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444240+0000", 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4893000000000004e-05 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.431 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444250+0000", 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0029e-05 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.306520+0000", 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86226998899999996 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.306696+0000", 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017564800000000001 2024-09-13T07:04:23.432 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.306820+0000", 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000123726 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308963+0000", 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021437990000000001 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312309+0000", 2024-09-13T07:04:23.433 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033458590000000001 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312384+0000", 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4184000000000007e-05 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312435+0000", 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1019000000000001e-05 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:23.434 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:23.435 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:04:23.436 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:23.433+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:23.437 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:04:23.536 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:23.534+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:23.545 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:178837 3.f 3.50f7bc8f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-13T07:04:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:04:23.494329+0000", 2024-09-13T07:04:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.052236483, 2024-09-13T07:04:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.052275997999999997, 2024-09-13T07:04:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for peered", 2024-09-13T07:04:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 178837 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:04:23.494329+0000", 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:04:23.494329+0000", 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:04:23.494328+0000", 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:04:23.494333+0000", 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2440000000000001e-06 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:04:23.494347+0000", 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3022e-05 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:04:23.494358+0000", 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1759999999999999e-05 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:04:23.494463+0000", 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010439500000000001 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:04:23.494494+0000", 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0902000000000003e-05 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-13T07:04:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-13T07:04:23.549 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:23.555 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:04:23.637 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:23.638+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:23.664 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:23.665 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:23.675 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:04:23.738 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:23.738+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:23.780 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.179296875000006, 2024-09-13T07:04:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:04:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.782 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.782 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:04:23.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.782 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:23.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:23.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:04:23.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:04:23.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:04:23.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:04:23.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:04:23.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:04:23.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:04:23.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.179265774000001, 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.786 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:23.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:04:23.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:04:23.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:04:23.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:04:23.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:04:23.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:04:23.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:04:23.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:04:23.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:04:23.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.179243459999995, 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.791 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:04:23.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.792 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:23.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:23.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:04:23.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:04:23.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:04:23.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:04:23.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:04:23.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:04:23.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:04:23.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:04:23.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:04:23.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.795 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.179220428999997, 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:04:23.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.800 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.179195879000005, 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:04:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:04:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:04:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:04:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:04:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:04:23.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:04:23.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.179160170000003, 2024-09-13T07:04:23.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:04:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:04:23.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.179135490999997, 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:04:23.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:04:23.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:04:23.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:04:23.814 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.814 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.814 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.179102216999993, 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:04:23.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:04:23.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:04:23.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:04:23.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:23.819 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.179075428999994, 2024-09-13T07:04:23.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:04:23.820 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.820 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.820 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.820 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.820 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.820 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:04:23.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.820 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:23.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:23.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:04:23.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:23.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.179050875000001, 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:04:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:04:23.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:04:23.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.179021105999993, 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:04:23.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:04:23.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:04:23.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.833 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.833 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.833 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:23.833 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.178961263000005, 2024-09-13T07:04:23.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:04:23.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.834 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:04:23.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.834 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:23.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:23.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:04:23.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:23.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:04:23.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:04:23.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:04:23.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:04:23.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:04:23.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:04:23.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:04:23.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:04:23.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:04:23.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:04:23.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:04:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:04:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:04:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:04:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.178894412999995, 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:23.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.178861910999998, 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:23.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.178827290000001, 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.848 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:23.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:23.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:04:23.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:04:23.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.178777058999998, 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:04:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.853 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.853 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.853 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.853 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.853 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:04:23.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.853 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:23.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:23.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.857 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.178728872999997, 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.178702235000003, 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.862 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:04:23.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:04:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:04:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.178655399999997, 2024-09-13T07:04:23.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:04:23.868 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.868 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.868 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:04:23.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.868 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:04:23.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:04:23.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:04:23.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.175300983, 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:23.872 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:23.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:04:23.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:23.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:04:23.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:04:23.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:23.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:23.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:23.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:04:23.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:04:23.876 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.876 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.876 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.876 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:23.876 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:23.876 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:23.878 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:04:23.878 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:23.878+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:23.979 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:23.978+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:24.001 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:24.001 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:24.012 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:04:24.079 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:24.077+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:24.115 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:04:24.136 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:24.137 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:24.137 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:24.137 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:24.137 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:24.149 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:04:24.194 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:24.193+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:24.250 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:24.251 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:24.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:24.251 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:24.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.251 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:04:24.251 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:24.251 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 178.190555741, 2024-09-13T07:04:24.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:04:24.251 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.251 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:24.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.252 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:04:24.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.252 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:24.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:24.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:04:24.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:04:24.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:04:24.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:04:24.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:04:24.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:04:24.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:04:24.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:04:24.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:04:24.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:04:24.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:04:24.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:04:24.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:24.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:04:24.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:04:24.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:24.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:04:24.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:04:24.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:24.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:04:24.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:04:24.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:24.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:04:24.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:04:24.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:04:24.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:04:24.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.257 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.632771863, 2024-09-13T07:04:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:04:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:04:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:04:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:04:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:04:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:04:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:04:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:04:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:04:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:04:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:04:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.632748896, 2024-09-13T07:04:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:04:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:04:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:04:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:04:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:04:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:04:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:04:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.632731718, 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:04:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:04:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:04:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.632717853, 2024-09-13T07:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:04:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:04:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:04:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:04:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:04:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:04:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:04:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.63270470800001, 2024-09-13T07:04:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:04:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:04:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:04:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:04:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:04:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:04:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:04:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:04:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:04:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:04:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.632691501, 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:04:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:04:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:04:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:04:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:04:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:04:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:04:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:04:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:04:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:04:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.632677434, 2024-09-13T07:04:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:04:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:04:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:04:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.63265742, 2024-09-13T07:04:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.63263766999999, 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:04:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:04:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:04:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:04:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:04:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:04:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:04:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:04:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:04:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:04:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.632612864, 2024-09-13T07:04:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:04:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:04:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:04:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:04:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:04:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:04:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:04:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:04:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.632588599, 2024-09-13T07:04:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:04:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.632546866, 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.63251057700001, 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:04:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:04:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:04:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:04:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.189407245, 2024-09-13T07:04:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:04:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:04:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:04:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:04:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:04:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:04:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:04:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.189390363, 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:04:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:04:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:04:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:04:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:04:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:04:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:04:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:04:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.188339806, 2024-09-13T07:04:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:04:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:04:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:04:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:04:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:04:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:04:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:04:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:04:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:04:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.186665257, 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:04:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:04:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:04:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:04:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:04:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.666082867, 2024-09-13T07:04:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:04:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:04:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:04:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:04:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:04:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:04:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:04:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:04:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:04:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:04:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:04:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172880 2.2 2:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e149)", 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 40.111786621, 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.97218948500000002, 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172880 2024-09-13T07:04:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137687+0000", 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137702+0000", 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5023e-05 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137730+0000", 2024-09-13T07:04:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7719e-05 2024-09-13T07:04:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137748+0000", 2024-09-13T07:04:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8189000000000002e-05 2024-09-13T07:04:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137891+0000", 2024-09-13T07:04:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014330699999999999 2024-09-13T07:04:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137903+0000", 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1688999999999999e-05 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142887+0000", 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004984006 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142898+0000", 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.079e-05 2024-09-13T07:04:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109703+0000", 2024-09-13T07:04:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96680553899999999 2024-09-13T07:04:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109879+0000", 2024-09-13T07:04:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017579800000000001 2024-09-13T07:04:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.345 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.345 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:24.346 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:04:24.347 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:24.346+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:24.447 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:24.445+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:24.460 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:24.460 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:24.460 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:24.470 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:04:24.548 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:24.546+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:24.622 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:24.623 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:24.632 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:04:24.649 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:24.650+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:24.743 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:04:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 178.68681932699999, 2024-09-13T07:04:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:04:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:04:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:04:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:04:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:04:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:04:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 146.52893809299999, 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:04:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 146.50316838200001, 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:04:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:04:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:04:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:04:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:04:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:04:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:04:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 146.482017216, 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:04:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:04:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:04:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:04:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 136.36497245300001, 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 118.07997520799999, 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:04:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.82826607, 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.82824803600001, 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:04:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:04:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:04:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:04:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:04:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:04:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.828231956, 2024-09-13T07:04:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:04:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:04:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:04:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:04:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:04:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.82821628000001, 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:04:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.82819335799999, 2024-09-13T07:04:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:04:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:04:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.82817726099999, 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:04:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:04:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:04:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:04:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.827575649, 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:04:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:04:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.827532873, 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:04:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:04:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:04:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:04:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:04:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:04:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:04:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:04:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.670515103, 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:04:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:04:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.818138172999994, 2024-09-13T07:04:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:04:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:04:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:04:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:04:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:04:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:04:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:04:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:04:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:04:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:04:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:04:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:164179 3.14 e135/67 3:2fc485bd:::rbd_data.11942e4f5fc7.0000000000000002:head v 135'401, mlcod=132'398)", 2024-09-13T07:04:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.795197719000001, 2024-09-13T07:04:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063492174999999998, 2024-09-13T07:04:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945792+0000", 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954811+0000", 2024-09-13T07:04:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090187940000000001 2024-09-13T07:04:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954840+0000", 2024-09-13T07:04:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8659000000000001e-05 2024-09-13T07:04:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954852+0000", 2024-09-13T07:04:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1966e-05 2024-09-13T07:04:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954938+0000", 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6539000000000006e-05 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954975+0000", 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6303999999999997e-05 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009206+0000", 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.054231044999999999 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009286+0000", 2024-09-13T07:04:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700999999999999e-05 2024-09-13T07:04:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:165953 3.d 3:b3a20b45:::rbd_data.11942e4f5fc7.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8796117~39260] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.518750161, 2024-09-13T07:04:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.095290105, 2024-09-13T07:04:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 165953 2024-09-13T07:04:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222240+0000", 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258929+0000", 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.036688401000000002 2024-09-13T07:04:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258971+0000", 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2212000000000002e-05 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258989+0000", 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7674000000000001e-05 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259072+0000", 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3656e-05 2024-09-13T07:04:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259297+0000", 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022450700000000001 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.260463+0000", 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011665340000000001 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.316214+0000", 2024-09-13T07:04:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.055750268999999998 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317335+0000", 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011211719999999999 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317451+0000", 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011616 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317532+0000", 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0860999999999997e-05 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172852 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [call lock.unlock in=52b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.542293014000002, 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.90390594199999996, 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172852 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198696+0000", 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198710+0000", 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3548e-05 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198739+0000", 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9606000000000001e-05 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198752+0000", 2024-09-13T07:04:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3033e-05 2024-09-13T07:04:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198869+0000", 2024-09-13T07:04:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116552 2024-09-13T07:04:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199018+0000", 2024-09-13T07:04:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014923700000000001 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199308+0000", 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028947199999999998 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.200731+0000", 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014229799999999999 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059443+0000", 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.85871189999999997 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059467+0000", 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.478e-05 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102380+0000", 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042912463999999997 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102605+0000", 2024-09-13T07:04:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022489499999999999 2024-09-13T07:04:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172853 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [watch reconnect cookie 140335574242864 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-13T07:04:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 40.690544154999998, 2024-09-13T07:04:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.056267725999999997, 2024-09-13T07:04:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172853 2024-09-13T07:04:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050445+0000", 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050461+0000", 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5783000000000001e-05 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050497+0000", 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5995999999999999e-05 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050516+0000", 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9579999999999999e-05 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050637+0000", 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120536 2024-09-13T07:04:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050647+0000", 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6989999999999998e-06 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059533+0000", 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088858749999999997 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059545+0000", 2024-09-13T07:04:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2473000000000001e-05 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102962+0000", 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043416656999999997 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.103612+0000", 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000650374 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.106715+0000", 2024-09-13T07:04:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031033229999999998 2024-09-13T07:04:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:24.838 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:24.839 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:24.837+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:24.840 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:04:24.940 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:24.938+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:24.958 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:24.958 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:24.958 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:24.969 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:04:25.041 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:25.042+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:25.079 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:25.079 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:25.079 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:25.079 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:25.079 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:25.089 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:04:25.141 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:25.142+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:25.185 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:04:25.242 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:25.242 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:25.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137474 3.6 3:65c31370:::rbd_data.1194af11889b.000000000000013b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 186185~11715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.497345963000001, 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0054357140000001, 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137474 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743934+0000", 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:25.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743949+0000", 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5705999999999998e-05 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743960+0000", 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0084e-05 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743971+0000", 2024-09-13T07:04:25.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0942e-05 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744045+0000", 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4626e-05 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744062+0000", 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7339000000000001e-05 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746366+0000", 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.002303457 2024-09-13T07:04:25.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746520+0000", 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015385900000000001 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746734+0000", 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000214253 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749128+0000", 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002394129 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749267+0000", 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013878299999999999 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749328+0000", 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0918000000000002e-05 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749371+0000", 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3188000000000001e-05 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.248 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163305 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2387968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.563127481999999, 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044745869000000001, 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163305 2024-09-13T07:04:25.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678153+0000", 2024-09-13T07:04:25.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678156+0000", 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6350000000000002e-06 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678161+0000", 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678166+0000", 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.498e-06 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722751+0000", 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044585134999999998 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722836+0000", 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4672999999999994e-05 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722900+0000", 2024-09-13T07:04:25.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4042999999999997e-05 2024-09-13T07:04:25.253 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.253 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.253 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.253 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163307 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.253 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:25.253 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.563107017999997, 2024-09-13T07:04:25.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045060944999999998, 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163307 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:25.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:25.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678177+0000", 2024-09-13T07:04:25.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:04:25.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678183+0000", 2024-09-13T07:04:25.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0599999999999996e-06 2024-09-13T07:04:25.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678188+0000", 2024-09-13T07:04:25.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:04:25.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722941+0000", 2024-09-13T07:04:25.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044752844999999999 2024-09-13T07:04:25.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723073+0000", 2024-09-13T07:04:25.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013128599999999999 2024-09-13T07:04:25.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723235+0000", 2024-09-13T07:04:25.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000162671 2024-09-13T07:04:25.259 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.259 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.259 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.259 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163310 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 544768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.259 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:25.259 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.563088065999999, 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045290534, 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163310 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678195+0000", 2024-09-13T07:04:25.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.785e-06 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678201+0000", 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678205+0000", 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6999999999999999e-06 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723299+0000", 2024-09-13T07:04:25.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045093964 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723384+0000", 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4562999999999997e-05 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723484+0000", 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100045 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163313 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1093632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.563072081000001, 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045562975999999998, 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163313 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678212+0000", 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8650000000000001e-06 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678217+0000", 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2660000000000001e-06 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678221+0000", 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-06 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723552+0000", 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045330489000000002 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723660+0000", 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010770800000000001 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723772+0000", 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112608 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163317 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1642496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.563043272999998, 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045770165000000002, 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163317 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678237+0000", 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678241+0000", 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2669999999999998e-06 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678247+0000", 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7309999999999996e-06 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678252+0000", 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-13T07:04:25.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723830+0000", 2024-09-13T07:04:25.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045577234000000001 2024-09-13T07:04:25.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723936+0000", 2024-09-13T07:04:25.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106277 2024-09-13T07:04:25.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724008+0000", 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2422999999999994e-05 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163320 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.563022848999999, 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045968883000000002, 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.272 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163320 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:25.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678261+0000", 2024-09-13T07:04:25.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8090000000000001e-06 2024-09-13T07:04:25.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678267+0000", 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8860000000000001e-06 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678271+0000", 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4549999999999997e-06 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724061+0000", 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045789656999999997 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724151+0000", 2024-09-13T07:04:25.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9734999999999994e-05 2024-09-13T07:04:25.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724227+0000", 2024-09-13T07:04:25.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6848999999999996e-05 2024-09-13T07:04:25.277 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.277 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.277 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.277 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163324 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.277 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:25.277 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.562738516000003, 2024-09-13T07:04:25.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045988788000000003, 2024-09-13T07:04:25.278 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.278 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.278 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.278 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.278 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.278 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163324 2024-09-13T07:04:25.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.280 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:25.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678542+0000", 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678547+0000", 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6829999999999999e-06 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678556+0000", 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1929999999999999e-06 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678562+0000", 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.083e-06 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724267+0000", 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045705427 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724417+0000", 2024-09-13T07:04:25.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014984699999999999 2024-09-13T07:04:25.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724532+0000", 2024-09-13T07:04:25.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011430900000000001 2024-09-13T07:04:25.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.283 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163327 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 733184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.562713481999999, 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046185902000000001, 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163327 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:25.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:25.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678567+0000", 2024-09-13T07:04:25.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678571+0000", 2024-09-13T07:04:25.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3799999999999998e-06 2024-09-13T07:04:25.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678576+0000", 2024-09-13T07:04:25.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7760000000000003e-06 2024-09-13T07:04:25.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678581+0000", 2024-09-13T07:04:25.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8640000000000004e-06 2024-09-13T07:04:25.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724561+0000", 2024-09-13T07:04:25.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045979178000000002 2024-09-13T07:04:25.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724649+0000", 2024-09-13T07:04:25.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8677000000000005e-05 2024-09-13T07:04:25.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724754+0000", 2024-09-13T07:04:25.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104527 2024-09-13T07:04:25.290 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.290 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.290 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.290 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163331 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.290 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.562695327999997, 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046361773000000002, 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163331 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678585+0000", 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678588+0000", 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0469999999999998e-06 2024-09-13T07:04:25.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678595+0000", 2024-09-13T07:04:25.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0399999999999998e-06 2024-09-13T07:04:25.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678599+0000", 2024-09-13T07:04:25.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:04:25.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724780+0000", 2024-09-13T07:04:25.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046181168000000002 2024-09-13T07:04:25.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724871+0000", 2024-09-13T07:04:25.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0562000000000005e-05 2024-09-13T07:04:25.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724948+0000", 2024-09-13T07:04:25.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6985999999999998e-05 2024-09-13T07:04:25.295 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.295 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.295 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.295 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163334 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1662976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.295 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:25.295 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.562678124999998, 2024-09-13T07:04:25.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046544502000000001, 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163334 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:25.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:25.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678605+0000", 2024-09-13T07:04:25.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5380000000000001e-06 2024-09-13T07:04:25.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678611+0000", 2024-09-13T07:04:25.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7389999999999996e-06 2024-09-13T07:04:25.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678615+0000", 2024-09-13T07:04:25.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-13T07:04:25.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724987+0000", 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046371969999999998 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725075+0000", 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8091999999999996e-05 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725148+0000", 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2206999999999995e-05 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.299 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163338 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.562661394999999, 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046738452999999999, 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163338 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.300 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:25.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:25.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678619+0000", 2024-09-13T07:04:25.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678622+0000", 2024-09-13T07:04:25.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.649e-06 2024-09-13T07:04:25.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678628+0000", 2024-09-13T07:04:25.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5029999999999997e-06 2024-09-13T07:04:25.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678632+0000", 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4020000000000004e-06 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725182+0000", 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046550389999999997 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725280+0000", 2024-09-13T07:04:25.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7260000000000001e-05 2024-09-13T07:04:25.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725358+0000", 2024-09-13T07:04:25.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8720000000000005e-05 2024-09-13T07:04:25.304 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.304 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.304 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.304 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163339 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2363392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.304 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:25.304 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.562645867000001, 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046920835000000001, 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163339 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:25.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:25.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678637+0000", 2024-09-13T07:04:25.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2479999999999999e-06 2024-09-13T07:04:25.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678642+0000", 2024-09-13T07:04:25.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0030000000000002e-06 2024-09-13T07:04:25.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678646+0000", 2024-09-13T07:04:25.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9870000000000001e-06 2024-09-13T07:04:25.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725397+0000", 2024-09-13T07:04:25.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046750127000000002 2024-09-13T07:04:25.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725482+0000", 2024-09-13T07:04:25.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5928999999999997e-05 2024-09-13T07:04:25.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725556+0000", 2024-09-13T07:04:25.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3823000000000001e-05 2024-09-13T07:04:25.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.309 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.309 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163342 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.310 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:25.310 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.562631228000001, 2024-09-13T07:04:25.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047212991000000003, 2024-09-13T07:04:25.310 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.310 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.310 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.310 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.310 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.310 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163342 2024-09-13T07:04:25.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.310 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:25.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:25.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:25.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678652+0000", 2024-09-13T07:04:25.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.486e-06 2024-09-13T07:04:25.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678657+0000", 2024-09-13T07:04:25.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6909999999999996e-06 2024-09-13T07:04:25.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678660+0000", 2024-09-13T07:04:25.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8009999999999999e-06 2024-09-13T07:04:25.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725597+0000", 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046937372999999998 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725745+0000", 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000148314 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725863+0000", 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011768 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.315 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163345 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.315 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:25.315 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.561769665, 2024-09-13T07:04:25.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046569193000000002, 2024-09-13T07:04:25.315 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.315 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.315 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.315 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.315 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.315 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163345 2024-09-13T07:04:25.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678662+0000", 2024-09-13T07:04:25.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9991498 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679516+0000", 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000854789 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679530+0000", 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3492999999999999e-05 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.317 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679537+0000", 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.323e-06 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725892+0000", 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046354244000000003 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725975+0000", 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3356000000000006e-05 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726081+0000", 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105992 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163349 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1298432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.561737602000001, 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046744247000000003, 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.319 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.320 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.320 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163349 2024-09-13T07:04:25.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.320 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:25.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:25.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679543+0000", 2024-09-13T07:04:25.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679546+0000", 2024-09-13T07:04:25.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:04:25.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679551+0000", 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679554+0000", 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8629999999999999e-06 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726117+0000", 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046563081999999999 2024-09-13T07:04:25.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726213+0000", 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6207999999999998e-05 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726288+0000", 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4741999999999997e-05 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163350 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1449984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.561724976000001, 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046935635000000003, 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163350 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:25.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:25.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:25.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679558+0000", 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.928e-06 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679561+0000", 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7659999999999999e-06 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679564+0000", 2024-09-13T07:04:25.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4960000000000001e-06 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726327+0000", 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046762321000000003 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726415+0000", 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8487999999999997e-05 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726492+0000", 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6995000000000004e-05 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163353 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.561714866999999, 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047130755000000003, 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.328 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163353 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679568+0000", 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5969999999999999e-06 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679571+0000", 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679573+0000", 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726523+0000", 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046949882999999998 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726616+0000", 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3236e-05 2024-09-13T07:04:25.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726697+0000", 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0834000000000006e-05 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163356 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2379776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.561705185000001, 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047304662999999997, 2024-09-13T07:04:25.332 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163356 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:25.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:25.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:25.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679578+0000", 2024-09-13T07:04:25.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:04:25.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679582+0000", 2024-09-13T07:04:25.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:04:25.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679589+0000", 2024-09-13T07:04:25.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1149999999999996e-06 2024-09-13T07:04:25.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726723+0000", 2024-09-13T07:04:25.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047133768999999999 2024-09-13T07:04:25.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726809+0000", 2024-09-13T07:04:25.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5402000000000007e-05 2024-09-13T07:04:25.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726881+0000", 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2152999999999998e-05 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:166663 3.b 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8129196~12979] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.843330670999997, 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.061564718999999997, 2024-09-13T07:04:25.337 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:25.338 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:25.338 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:25.338 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:25.338 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:25.338 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 166663 2024-09-13T07:04:25.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.338 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:25.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:25.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:25.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397947+0000", 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999967 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412144+0000", 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014196972 2024-09-13T07:04:25.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412172+0000", 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7781999999999999e-05 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412182+0000", 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6069999999999997e-06 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412239+0000", 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.732e-05 2024-09-13T07:04:25.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:25.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412365+0000", 2024-09-13T07:04:25.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125356 2024-09-13T07:04:25.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:25.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.413178+0000", 2024-09-13T07:04:25.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081355299999999995 2024-09-13T07:04:25.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.443176+0000", 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029997903999999999 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459446+0000", 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016269921999999999 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459484+0000", 2024-09-13T07:04:25.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8488000000000001e-05 2024-09-13T07:04:25.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:25.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:25.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:25.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459515+0000", 2024-09-13T07:04:25.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0981e-05 2024-09-13T07:04:25.343 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.343 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.343 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.343 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:25.343 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:25.343 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:25.344 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:25.242+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:25.345 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:04:25.345 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:25.346 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:25.346+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:25.349 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:25.370 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:25.389 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:25.411 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:25.426 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 2, 0, 1] out_osds: [7] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-13T07:04:25.426 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:04:25.427 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T07:04:25.427 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.000000 2024-09-13T07:04:25.427 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0 2024-09-13T07:04:25.434 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:25.446 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:25.446+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:25.450 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:25.450 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:25.460 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:04:25.462 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:25.462 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:25.462 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:25.462 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:25.462 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:25.462 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:25.462 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:25.463 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:25.463 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:25.464 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:25.465 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:25.548 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:25.546+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:25.560 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:25.560 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:25.570 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:04:25.649 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:25.645+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:25.669 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:25.669 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:25.679 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:04:25.750 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:25.750+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:25.786 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:25.786 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:25.796 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:04:25.851 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:25.849+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:25.901 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:25.901 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:25.913 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:04:25.951 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:25.950+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:26.020 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:26.020 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:26.030 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:04:26.052 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:26.049+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:26.124 INFO:teuthology.orchestra.run.smithi118.stderr:set osd.0 primary-affinity to 0 (802) 2024-09-13T07:04:26.129 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:04:26.145 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:26.145 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:26.152 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:26.150+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:26.156 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:04:26.254 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:26.253+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:26.302 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:26.302 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:26.312 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:04:26.355 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:26.354+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:26.414 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:26.414 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:26.424 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:04:26.465 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:26.466+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:26.541 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:26.541 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:26.551 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:04:26.557 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:26.554+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:26.658 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:26.658+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:26.660 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:26.660 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:26.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:04:26.758 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:26.757+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:26.774 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:26.774 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:26.784 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:04:26.859 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:26.857+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:26.896 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:26.896 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:26.906 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:04:26.959 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:26.958+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:27.058 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:27.058 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:27.060 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:27.058+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:27.068 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:04:27.134 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:04:27.162 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:27.157+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:27.177 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:27.178 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:27.188 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:04:27.263 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:27.262+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:27.296 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:27.297 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:27.305 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:04:27.364 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:27.362+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:27.413 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:27.413 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:27.423 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:04:27.465 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:04:27.466 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:27.466+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:27.526 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:27.527 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:27.537 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:04:27.566 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:27.561+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:27.616 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:27.636 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:27.645 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:27.645 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:27.646 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:27.646 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:27.652 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:27.655 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:04:27.666 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:27.666+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:27.672 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:27.693 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:27.717 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:27.746 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:27.746 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:27.746 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:27.747 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:27.747 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:27.747 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:27.747 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:27.747 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:27.747 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:27.749 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:27.750 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:27.757 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:27.757 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:27.760 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:27.760 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:27.760 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:27.767 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:04:27.768 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:27.765+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:27.868 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:27.869+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:27.874 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137474 3.6 3:65c31370:::rbd_data.1194af11889b.000000000000013b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 186185~11715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-13T07:04:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:27.874 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 83.129217987999994, 2024-09-13T07:04:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0054357140000001, 2024-09-13T07:04:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137474 2024-09-13T07:04:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:27.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743934+0000", 2024-09-13T07:04:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743949+0000", 2024-09-13T07:04:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5705999999999998e-05 2024-09-13T07:04:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743960+0000", 2024-09-13T07:04:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0084e-05 2024-09-13T07:04:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743971+0000", 2024-09-13T07:04:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0942e-05 2024-09-13T07:04:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.878 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744045+0000", 2024-09-13T07:04:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4626e-05 2024-09-13T07:04:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744062+0000", 2024-09-13T07:04:27.879 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7339000000000001e-05 2024-09-13T07:04:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746366+0000", 2024-09-13T07:04:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.002303457 2024-09-13T07:04:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.880 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746520+0000", 2024-09-13T07:04:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015385900000000001 2024-09-13T07:04:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746734+0000", 2024-09-13T07:04:27.881 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000214253 2024-09-13T07:04:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749128+0000", 2024-09-13T07:04:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002394129 2024-09-13T07:04:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:27.882 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749267+0000", 2024-09-13T07:04:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013878299999999999 2024-09-13T07:04:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749328+0000", 2024-09-13T07:04:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0918000000000002e-05 2024-09-13T07:04:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749371+0000", 2024-09-13T07:04:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3188000000000001e-05 2024-09-13T07:04:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.883 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163305 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2387968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.194999506999999, 2024-09-13T07:04:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044745869000000001, 2024-09-13T07:04:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.884 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163305 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.885 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678153+0000", 2024-09-13T07:04:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678156+0000", 2024-09-13T07:04:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6350000000000002e-06 2024-09-13T07:04:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678161+0000", 2024-09-13T07:04:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:04:27.886 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678166+0000", 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.498e-06 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722751+0000", 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044585134999999998 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722836+0000", 2024-09-13T07:04:27.887 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4672999999999994e-05 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722900+0000", 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4042999999999997e-05 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163307 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.194979042999996, 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045060944999999998, 2024-09-13T07:04:27.888 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163307 2024-09-13T07:04:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.889 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678177+0000", 2024-09-13T07:04:27.890 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:04:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678183+0000", 2024-09-13T07:04:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0599999999999996e-06 2024-09-13T07:04:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678188+0000", 2024-09-13T07:04:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:04:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.891 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722941+0000", 2024-09-13T07:04:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044752844999999999 2024-09-13T07:04:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723073+0000", 2024-09-13T07:04:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013128599999999999 2024-09-13T07:04:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723235+0000", 2024-09-13T07:04:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000162671 2024-09-13T07:04:27.892 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163310 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 544768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.194960090999999, 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045290534, 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.893 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163310 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:27.894 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678195+0000", 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.785e-06 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678201+0000", 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.895 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678205+0000", 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6999999999999999e-06 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723299+0000", 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045093964 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723384+0000", 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4562999999999997e-05 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.896 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723484+0000", 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100045 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163313 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1093632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.194944106000001, 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045562975999999998, 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.897 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163313 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:27.898 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678212+0000", 2024-09-13T07:04:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8650000000000001e-06 2024-09-13T07:04:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.899 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678217+0000", 2024-09-13T07:04:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2660000000000001e-06 2024-09-13T07:04:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678221+0000", 2024-09-13T07:04:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-06 2024-09-13T07:04:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.900 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723552+0000", 2024-09-13T07:04:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045330489000000002 2024-09-13T07:04:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723660+0000", 2024-09-13T07:04:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010770800000000001 2024-09-13T07:04:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.901 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723772+0000", 2024-09-13T07:04:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112608 2024-09-13T07:04:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163317 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1642496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.902 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.194915297999998, 2024-09-13T07:04:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045770165000000002, 2024-09-13T07:04:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.903 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163317 2024-09-13T07:04:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.904 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678237+0000", 2024-09-13T07:04:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.905 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678241+0000", 2024-09-13T07:04:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2669999999999998e-06 2024-09-13T07:04:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678247+0000", 2024-09-13T07:04:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7309999999999996e-06 2024-09-13T07:04:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678252+0000", 2024-09-13T07:04:27.906 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-13T07:04:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723830+0000", 2024-09-13T07:04:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045577234000000001 2024-09-13T07:04:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.907 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723936+0000", 2024-09-13T07:04:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106277 2024-09-13T07:04:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724008+0000", 2024-09-13T07:04:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2422999999999994e-05 2024-09-13T07:04:27.908 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163320 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.194894873999999, 2024-09-13T07:04:27.909 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045968883000000002, 2024-09-13T07:04:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163320 2024-09-13T07:04:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.910 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:27.911 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.912 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678261+0000", 2024-09-13T07:04:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8090000000000001e-06 2024-09-13T07:04:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678267+0000", 2024-09-13T07:04:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8860000000000001e-06 2024-09-13T07:04:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.913 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678271+0000", 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4549999999999997e-06 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724061+0000", 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045789656999999997 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724151+0000", 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9734999999999994e-05 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.914 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724227+0000", 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6848999999999996e-05 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163324 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.194610541000003, 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045988788000000003, 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.915 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163324 2024-09-13T07:04:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.916 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678542+0000", 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678547+0000", 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6829999999999999e-06 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.917 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678556+0000", 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1929999999999999e-06 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678562+0000", 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.083e-06 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724267+0000", 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045705427 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.918 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724417+0000", 2024-09-13T07:04:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014984699999999999 2024-09-13T07:04:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724532+0000", 2024-09-13T07:04:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011430900000000001 2024-09-13T07:04:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.919 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163327 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 733184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.194585506999999, 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046185902000000001, 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163327 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.920 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.921 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678567+0000", 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678571+0000", 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3799999999999998e-06 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678576+0000", 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7760000000000003e-06 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678581+0000", 2024-09-13T07:04:27.922 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8640000000000004e-06 2024-09-13T07:04:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724561+0000", 2024-09-13T07:04:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045979178000000002 2024-09-13T07:04:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724649+0000", 2024-09-13T07:04:27.923 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8677000000000005e-05 2024-09-13T07:04:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724754+0000", 2024-09-13T07:04:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104527 2024-09-13T07:04:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163331 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:27.924 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.194567352999997, 2024-09-13T07:04:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046361773000000002, 2024-09-13T07:04:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163331 2024-09-13T07:04:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:27.925 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678585+0000", 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678588+0000", 2024-09-13T07:04:27.926 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0469999999999998e-06 2024-09-13T07:04:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678595+0000", 2024-09-13T07:04:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0399999999999998e-06 2024-09-13T07:04:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678599+0000", 2024-09-13T07:04:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:04:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.927 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724780+0000", 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046181168000000002 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724871+0000", 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0562000000000005e-05 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724948+0000", 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6985999999999998e-05 2024-09-13T07:04:27.928 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163334 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1662976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.194550149999998, 2024-09-13T07:04:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046544502000000001, 2024-09-13T07:04:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.929 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163334 2024-09-13T07:04:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.930 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.931 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678605+0000", 2024-09-13T07:04:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5380000000000001e-06 2024-09-13T07:04:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678611+0000", 2024-09-13T07:04:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7389999999999996e-06 2024-09-13T07:04:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.932 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678615+0000", 2024-09-13T07:04:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-13T07:04:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724987+0000", 2024-09-13T07:04:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046371969999999998 2024-09-13T07:04:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.933 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725075+0000", 2024-09-13T07:04:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8091999999999996e-05 2024-09-13T07:04:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725148+0000", 2024-09-13T07:04:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2206999999999995e-05 2024-09-13T07:04:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.934 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163338 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.194533419999999, 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046738452999999999, 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163338 2024-09-13T07:04:27.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.936 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678619+0000", 2024-09-13T07:04:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678622+0000", 2024-09-13T07:04:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.649e-06 2024-09-13T07:04:27.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678628+0000", 2024-09-13T07:04:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5029999999999997e-06 2024-09-13T07:04:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678632+0000", 2024-09-13T07:04:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4020000000000004e-06 2024-09-13T07:04:27.938 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725182+0000", 2024-09-13T07:04:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046550389999999997 2024-09-13T07:04:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725280+0000", 2024-09-13T07:04:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7260000000000001e-05 2024-09-13T07:04:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.939 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725358+0000", 2024-09-13T07:04:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8720000000000005e-05 2024-09-13T07:04:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163339 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2363392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:27.940 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.194517892, 2024-09-13T07:04:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046920835000000001, 2024-09-13T07:04:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163339 2024-09-13T07:04:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:27.941 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:27.942 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678637+0000", 2024-09-13T07:04:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2479999999999999e-06 2024-09-13T07:04:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678642+0000", 2024-09-13T07:04:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0030000000000002e-06 2024-09-13T07:04:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.943 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678646+0000", 2024-09-13T07:04:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9870000000000001e-06 2024-09-13T07:04:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.944 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725397+0000", 2024-09-13T07:04:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046750127000000002 2024-09-13T07:04:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.945 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725482+0000", 2024-09-13T07:04:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5928999999999997e-05 2024-09-13T07:04:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725556+0000", 2024-09-13T07:04:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3823000000000001e-05 2024-09-13T07:04:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.946 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163342 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.194503253000001, 2024-09-13T07:04:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047212991000000003, 2024-09-13T07:04:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.947 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163342 2024-09-13T07:04:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.948 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678652+0000", 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.486e-06 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.949 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678657+0000", 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6909999999999996e-06 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678660+0000", 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8009999999999999e-06 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725597+0000", 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046937372999999998 2024-09-13T07:04:27.950 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725745+0000", 2024-09-13T07:04:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000148314 2024-09-13T07:04:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725863+0000", 2024-09-13T07:04:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011768 2024-09-13T07:04:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.951 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163345 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.19364169, 2024-09-13T07:04:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046569193000000002, 2024-09-13T07:04:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163345 2024-09-13T07:04:27.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.953 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678662+0000", 2024-09-13T07:04:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9991498 2024-09-13T07:04:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679516+0000", 2024-09-13T07:04:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000854789 2024-09-13T07:04:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.954 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679530+0000", 2024-09-13T07:04:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3492999999999999e-05 2024-09-13T07:04:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679537+0000", 2024-09-13T07:04:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.323e-06 2024-09-13T07:04:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.955 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725892+0000", 2024-09-13T07:04:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046354244000000003 2024-09-13T07:04:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725975+0000", 2024-09-13T07:04:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3356000000000006e-05 2024-09-13T07:04:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726081+0000", 2024-09-13T07:04:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105992 2024-09-13T07:04:27.956 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163349 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1298432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.193609627000001, 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046744247000000003, 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163349 2024-09-13T07:04:27.957 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.958 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679543+0000", 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679546+0000", 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679551+0000", 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.959 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679554+0000", 2024-09-13T07:04:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8629999999999999e-06 2024-09-13T07:04:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726117+0000", 2024-09-13T07:04:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046563081999999999 2024-09-13T07:04:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726213+0000", 2024-09-13T07:04:27.960 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6207999999999998e-05 2024-09-13T07:04:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.961 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726288+0000", 2024-09-13T07:04:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4741999999999997e-05 2024-09-13T07:04:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163350 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1449984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.193597001000001, 2024-09-13T07:04:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046935635000000003, 2024-09-13T07:04:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.965 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163350 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.966 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679558+0000", 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.928e-06 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.967 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679561+0000", 2024-09-13T07:04:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7659999999999999e-06 2024-09-13T07:04:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679564+0000", 2024-09-13T07:04:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4960000000000001e-06 2024-09-13T07:04:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726327+0000", 2024-09-13T07:04:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046762321000000003 2024-09-13T07:04:27.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726415+0000", 2024-09-13T07:04:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8487999999999997e-05 2024-09-13T07:04:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.969 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726492+0000", 2024-09-13T07:04:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6995000000000004e-05 2024-09-13T07:04:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163353 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.193586891999999, 2024-09-13T07:04:27.970 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047130755000000003, 2024-09-13T07:04:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163353 2024-09-13T07:04:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:27.971 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.972 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679568+0000", 2024-09-13T07:04:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5969999999999999e-06 2024-09-13T07:04:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679571+0000", 2024-09-13T07:04:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:04:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679573+0000", 2024-09-13T07:04:27.973 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:04:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726523+0000", 2024-09-13T07:04:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046949882999999998 2024-09-13T07:04:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.974 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726616+0000", 2024-09-13T07:04:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3236e-05 2024-09-13T07:04:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726697+0000", 2024-09-13T07:04:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0834000000000006e-05 2024-09-13T07:04:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.975 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163356 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2379776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.193577210000001, 2024-09-13T07:04:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047304662999999997, 2024-09-13T07:04:27.976 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163356 2024-09-13T07:04:27.977 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:27.978 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679578+0000", 2024-09-13T07:04:27.979 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:04:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679582+0000", 2024-09-13T07:04:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:04:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.980 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679589+0000", 2024-09-13T07:04:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1149999999999996e-06 2024-09-13T07:04:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726723+0000", 2024-09-13T07:04:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047133768999999999 2024-09-13T07:04:27.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726809+0000", 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5402000000000007e-05 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726881+0000", 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2152999999999998e-05 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.982 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:166663 3.b 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8129196~12979] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 58.475202695999997, 2024-09-13T07:04:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.061564718999999997, 2024-09-13T07:04:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 166663 2024-09-13T07:04:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:27.983 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.984 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397947+0000", 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999967 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412144+0000", 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014196972 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412172+0000", 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7781999999999999e-05 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.985 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412182+0000", 2024-09-13T07:04:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6069999999999997e-06 2024-09-13T07:04:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412239+0000", 2024-09-13T07:04:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.732e-05 2024-09-13T07:04:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:27.986 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412365+0000", 2024-09-13T07:04:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125356 2024-09-13T07:04:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.413178+0000", 2024-09-13T07:04:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081355299999999995 2024-09-13T07:04:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.443176+0000", 2024-09-13T07:04:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029997903999999999 2024-09-13T07:04:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.987 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459446+0000", 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016269921999999999 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459484+0000", 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8488000000000001e-05 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459515+0000", 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0981e-05 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.988 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:27.989 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:27.989 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:27.990 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:04:27.991 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:27.989+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:28.091 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:28.090+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:28.102 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:28.102 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:28.102 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:28.102 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:28.112 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:04:28.192 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:28.190+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:28.220 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:28.221 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:28.221 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:28.221 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:28.221 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:28.231 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:04:28.293 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:28.289+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:28.343 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:28.343 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:28.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:28.343 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:28.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.344 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:04:28.344 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:28.344 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 164.84859019999999, 2024-09-13T07:04:28.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:04:28.344 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.344 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.344 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.344 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.344 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.344 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:04:28.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.344 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:28.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:28.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:04:28.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:04:28.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:04:28.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:04:28.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:04:28.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:04:28.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:04:28.348 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.348 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.349 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.349 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.349 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:28.349 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.36210663599999, 2024-09-13T07:04:28.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:04:28.349 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.349 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.349 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.349 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.350 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.350 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:04:28.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.350 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:28.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:28.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:28.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:04:28.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:04:28.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:04:28.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:04:28.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:04:28.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:04:28.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.352 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.352 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.353 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.36203670699999, 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.354 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:04:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:04:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:04:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:04:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:04:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:04:28.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:04:28.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:04:28.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:04:28.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:04:28.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:04:28.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:04:28.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:04:28.358 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.358 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.358 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.358 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.358 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:28.358 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.36202083500001, 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.362 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.363 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:28.363 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.36195037799999, 2024-09-13T07:04:28.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:04:28.363 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.363 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.363 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.363 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.363 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.363 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:04:28.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.363 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:04:28.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:04:28.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:04:28.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:04:28.365 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.365 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.365 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.365 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:04:28.365 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:04:28.366 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.35859833999999, 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.367 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.368 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.369 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.370 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.370 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:04:28.372 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:04:28.372 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.372 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:04:28.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:04:28.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.373 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.373 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:04:28.373 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:04:28.373 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.373 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.373 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.373 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.373 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.358582039, 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.374 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:28.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:28.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.375 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.375 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:04:28.375 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.375 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.375 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:04:28.376 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.377 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.35856225399999, 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:28.378 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:28.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.379 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.379 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.379 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:28.379 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.379 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:04:28.380 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.381 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.35854598099999, 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.382 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.383 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:04:28.384 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.385 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.35852815600001, 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:28.386 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.387 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:04:28.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:04:28.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:04:28.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:04:28.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.388 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.388 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:04:28.388 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:04:28.388 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.388 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.389 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:28.390 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.35851206300001, 2024-09-13T07:04:28.390 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:04:28.390 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.390 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.390 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.390 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.390 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.390 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:04:28.390 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.390 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.390 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.390 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.390 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.391 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.392 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.393 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.35849087700001, 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.394 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:04:28.395 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.396 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:28.397 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.358474844, 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.398 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:04:28.399 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:04:28.400 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.35842560500001, 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:04:28.401 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.402 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.402 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.402 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.402 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.402 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:04:28.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.402 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.402 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.402 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:28.402 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.402 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.402 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:04:28.403 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:04:28.404 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.35840754500001, 2024-09-13T07:04:28.405 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.406 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.407 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:04:28.408 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.409 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.409 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:04:28.409 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:04:28.409 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.409 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.409 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.409 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.409 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.409 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.409 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:28.409 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.35839103000001, 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.410 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:28.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:04:28.411 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.411 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.411 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.411 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.411 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.412 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.413 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:28.414 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.35837266499999, 2024-09-13T07:04:28.414 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:04:28.414 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.414 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.414 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.414 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.414 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.414 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:04:28.414 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.414 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.414 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.414 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.414 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.415 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.416 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:04:28.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:04:28.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.417 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.417 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:04:28.417 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:04:28.417 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.417 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.417 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.417 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.417 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.35835656699999, 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.418 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.419 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.420 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.421 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.35834063199999, 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.422 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:04:28.423 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:04:28.424 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 150.358324135, 2024-09-13T07:04:28.425 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:04:28.426 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.426 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.426 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.426 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.426 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.426 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:04:28.426 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.426 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.426 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.426 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.426 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:28.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:28.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.427 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.427 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.427 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:28.427 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.427 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:04:28.428 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.429 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.430 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.430 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:28.431 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:04:28.432 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:28.429+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:28.532 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:28.530+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:28.545 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:28.546 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:28.546 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:28.546 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:28.556 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:04:28.633 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:28.633+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:28.663 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:28.663 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:28.663 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:28.663 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:28.663 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:28.672 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:04:28.734 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:28.734+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:28.783 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:28.783 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:28.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:28.783 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:28.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.783 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:04:28.783 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:28.783 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 182.72685663499999, 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:04:28.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:04:28.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:04:28.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:04:28.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:04:28.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:04:28.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.787 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.908640278000007, 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.788 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:28.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:28.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:04:28.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:04:28.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:04:28.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:04:28.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:04:28.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:04:28.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.908590889999999, 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:28.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:28.796 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.908568183, 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.908523990000006, 2024-09-13T07:04:28.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:04:28.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:04:28.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 98.877544900999993, 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:04:28.804 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:04:28.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137457 3.18 3:1a596dcc:::rbd_data.1194af11889b.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.131923091000004, 2024-09-13T07:04:28.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033734403000000003, 2024-09-13T07:04:28.809 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.809 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:28.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.809 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.809 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137457 2024-09-13T07:04:28.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.809 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:28.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:28.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651523+0000", 2024-09-13T07:04:28.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:28.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651531+0000", 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.328e-06 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651545+0000", 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4270000000000001e-05 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651555+0000", 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5210000000000008e-06 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651626+0000", 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1915999999999994e-05 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651761+0000", 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000134838 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.658577+0000", 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068151820000000004 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:28.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685181+0000", 2024-09-13T07:04:28.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026604200000000001 2024-09-13T07:04:28.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:28.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685193+0000", 2024-09-13T07:04:28.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2128e-05 2024-09-13T07:04:28.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:28.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685226+0000", 2024-09-13T07:04:28.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3504999999999999e-05 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685259+0000", 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2866e-05 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138565 3.0 3:002d0425:::rbd_data.11949f62878b.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.118552598999997, 2024-09-13T07:04:28.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033394489999999999, 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138565 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664893+0000", 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664906+0000", 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2996000000000001e-05 2024-09-13T07:04:28.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664936+0000", 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9609999999999999e-05 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664949+0000", 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383e-05 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665046+0000", 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6459000000000006e-05 2024-09-13T07:04:28.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665259+0000", 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000213141 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.672522+0000", 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072632110000000003 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.697982+0000", 2024-09-13T07:04:28.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025459699999999998 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698018+0000", 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6375999999999999e-05 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698166+0000", 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014818700000000001 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698290+0000", 2024-09-13T07:04:28.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012333999999999999 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138587 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 80.070086931999995, 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019224502000000001, 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138587 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713360+0000", 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713368+0000", 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713383+0000", 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5176e-05 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713390+0000", 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8669999999999996e-06 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713449+0000", 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8938999999999999e-05 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713549+0000", 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010009 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.716087+0000", 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025383820000000001 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732351+0000", 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016263564000000001 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732450+0000", 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9529000000000002e-05 2024-09-13T07:04:28.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732532+0000", 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1526000000000005e-05 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732585+0000", 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3819000000000001e-05 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138786 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 79.635871676999997, 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018847452000000001, 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138786 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:28.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:28.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147575+0000", 2024-09-13T07:04:28.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:28.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147583+0000", 2024-09-13T07:04:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:04:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147601+0000", 2024-09-13T07:04:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7961e-05 2024-09-13T07:04:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147610+0000", 2024-09-13T07:04:28.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147661+0000", 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0447e-05 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147754+0000", 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2974999999999997e-05 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:28.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.150580+0000", 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002826566 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166135+0000", 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015554772 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166225+0000", 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0258000000000002e-05 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:28.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166334+0000", 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010816500000000001 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166424+0000", 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0116000000000003e-05 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:139127 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:28.831 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 79.008820713000006, 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019295931999999998, 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139127 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774626+0000", 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774635+0000", 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5419999999999994e-06 2024-09-13T07:04:28.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774654+0000", 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8359000000000002e-05 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774663+0000", 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774718+0000", 2024-09-13T07:04:28.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5315000000000001e-05 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774805+0000", 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7284e-05 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.779319+0000", 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045138310000000003 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793772+0000", 2024-09-13T07:04:28.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014452692 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793821+0000", 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9563e-05 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793878+0000", 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6953999999999997e-05 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793923+0000", 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4781999999999999e-05 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:164691 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.724860839000002, 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015985102000000001, 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.837 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 164691 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058589+0000", 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2510000000000001e-06 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058594+0000", 2024-09-13T07:04:28.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9089999999999999e-06 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058598+0000", 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2069999999999997e-06 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064175+0000", 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055769909999999999 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064210+0000", 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4966999999999999e-05 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074572+0000", 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010362219000000001 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165202 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7831552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:28.841 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.722315235000003, 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013116245, 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165202 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061132+0000", 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061135+0000", 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3410000000000001e-06 2024-09-13T07:04:28.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061142+0000", 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7079999999999997e-06 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061147+0000", 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8930000000000004e-06 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074180+0000", 2024-09-13T07:04:28.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013032346 2024-09-13T07:04:28.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074217+0000", 2024-09-13T07:04:28.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:04:28.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074249+0000", 2024-09-13T07:04:28.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1711000000000002e-05 2024-09-13T07:04:28.845 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.845 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.845 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165203 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.722297120999997, 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013180211000000001, 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165203 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.846 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061150+0000", 2024-09-13T07:04:28.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061153+0000", 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.108e-06 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061158+0000", 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0649999999999998e-06 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061162+0000", 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0080000000000004e-06 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074262+0000", 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013100557000000001 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074299+0000", 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6981999999999999e-05 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.849 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074331+0000", 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1801000000000003e-05 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165204 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8093696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.722282409999998, 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0132388, 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.850 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165204 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061167+0000", 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1409999999999999e-06 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061175+0000", 2024-09-13T07:04:28.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5519999999999999e-06 2024-09-13T07:04:28.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061179+0000", 2024-09-13T07:04:28.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.172e-06 2024-09-13T07:04:28.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074342+0000", 2024-09-13T07:04:28.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013162821999999999 2024-09-13T07:04:28.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074373+0000", 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1103999999999999e-05 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074404+0000", 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1359999999999998e-05 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165205 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:28.854 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.722265450999998, 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013300635, 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165205 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:28.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:28.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061184+0000", 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0779999999999998e-06 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061192+0000", 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7209999999999994e-06 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061196+0000", 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4469999999999999e-06 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074417+0000", 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013220750999999999 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074452+0000", 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4814999999999998e-05 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074483+0000", 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1118000000000003e-05 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.858 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165206 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.722247330000002, 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013364454, 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165206 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.859 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061200+0000", 2024-09-13T07:04:28.860 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061203+0000", 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.559e-06 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061210+0000", 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.861 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061214+0000", 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0999999999999997e-06 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074494+0000", 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013279094 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074530+0000", 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6538999999999997e-05 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.862 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074565+0000", 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4972999999999999e-05 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169377 3.0 3:07b65a69:::rbd_data.1194e181a50b.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 56.426580934999997, 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.032744151999999999, 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.863 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169377 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356864+0000", 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356880+0000", 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6022999999999999e-05 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356916+0000", 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5921999999999999e-05 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.865 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356931+0000", 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4311000000000001e-05 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357011+0000", 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9992000000000003e-05 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357143+0000", 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000131822 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.866 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.361578+0000", 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00443525 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389466+0000", 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027888342999999999 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389529+0000", 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2460999999999997e-05 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:28.867 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389574+0000", 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5405999999999999e-05 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389611+0000", 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7122000000000001e-05 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169731 3.0 3:031f8e93:::rbd_data.1194e181a50b.0000000000000006:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 55.502155137000003, 2024-09-13T07:04:28.868 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.021389756999999999, 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169731 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.869 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281291+0000", 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281302+0000", 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0651e-05 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.870 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281324+0000", 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1713999999999999e-05 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281340+0000", 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281450+0000", 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110086 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281593+0000", 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014306 2024-09-13T07:04:28.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.286356+0000", 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047626320000000002 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302404+0000", 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016048605000000001 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302476+0000", 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1483000000000007e-05 2024-09-13T07:04:28.872 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302578+0000", 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101883 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302683+0000", 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104681 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.873 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:178874 3.16 3:68eb4b70:::journal.1194897e7c0a:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e189)", 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:24.159236+0000", 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 4.624212354, 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.95629728899999999, 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 178874 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159236+0000", 2024-09-13T07:04:28.874 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159236+0000", 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159234+0000", 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159241+0000", 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.277e-06 2024-09-13T07:04:28.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159294+0000", 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2166000000000002e-05 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159302+0000", 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159393+0000", 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1423999999999997e-05 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.876 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159408+0000", 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4605e-05 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:25.115368+0000", 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.955959534 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:25.115533+0000", 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016526900000000001 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.877 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.878 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.878 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:28.878 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:28.878 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:28.879 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:04:28.880 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:28.878+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:28.980 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:28.981+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:28.994 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:28.994 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:28.995 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:28.995 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:29.004 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:04:29.081 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:29.078+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:29.110 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:29.110 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:29.110 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:29.110 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:29.110 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:29.119 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:04:29.148 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T07:04:29.181 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:29.182+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:29.239 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:29.239 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:29.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:29.239 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:29.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.239 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.239 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:29.239 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.116321216, 2024-09-13T07:04:29.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.243 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.109612544, 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:29.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:04:29.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:04:29.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:04:29.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:04:29.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:04:29.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:04:29.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:04:29.247 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.109600101, 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.248 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.250 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.109582503, 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.252 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.253 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.253 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.253 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.253 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:04:29.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.253 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:29.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:04:29.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:04:29.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:04:29.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:04:29.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:04:29.256 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.256 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.256 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.1095718, 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:04:29.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:04:29.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:04:29.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:04:29.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:04:29.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.109527867, 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.261 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:04:29.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:04:29.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.109508081, 2024-09-13T07:04:29.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:04:29.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:04:29.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:04:29.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:04:29.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:04:29.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.270 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.270 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:29.270 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.109492331, 2024-09-13T07:04:29.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:04:29.270 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.270 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.270 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.270 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.270 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.270 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:04:29.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.270 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:29.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:29.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:04:29.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:04:29.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.272 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:04:29.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:04:29.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.272 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.10948107900001, 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.274 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:29.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:04:29.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:29.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:04:29.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:04:29.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:04:29.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:04:29.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:04:29.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:04:29.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:04:29.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:04:29.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:04:29.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:04:29.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:04:29.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:04:29.278 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.10947048, 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.279 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:29.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.282 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.283 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.283 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:29.283 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.10946098700001, 2024-09-13T07:04:29.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:04:29.283 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.283 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.283 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.283 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.283 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.283 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:04:29.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:04:29.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:04:29.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:04:29.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:04:29.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:04:29.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:04:29.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.109451443, 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:04:29.287 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.288 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.288 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.288 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.288 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.288 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:04:29.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.288 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:29.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:04:29.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.291 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.292 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:29.292 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.10941884, 2024-09-13T07:04:29.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:04:29.292 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.292 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.292 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.292 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:04:29.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.292 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:04:29.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:04:29.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:04:29.296 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.296 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.296 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.296 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.296 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:29.296 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.109405817, 2024-09-13T07:04:29.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:04:29.296 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.296 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.296 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:04:29.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:04:29.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:04:29.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:04:29.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:04:29.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.10603455499999, 2024-09-13T07:04:29.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:29.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:04:29.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:29.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:04:29.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:04:29.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:04:29.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:04:29.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:04:29.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:04:29.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:04:29.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:04:29.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:04:29.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:04:29.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:04:29.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:04:29.305 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.305 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.305 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.305 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:29.305 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:29.305 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 127.106014354, 2024-09-13T07:04:29.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:04:29.305 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.305 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:29.305 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.306 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.306 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.306 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:04:29.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.306 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:29.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:04:29.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:04:29.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:04:29.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:04:29.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.309 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.309 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.312 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:166663 3.b e135/117 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head v 135'409, mlcod=132'407)", 2024-09-13T07:04:29.312 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:29.312 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 59.823425389000001, 2024-09-13T07:04:29.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045108144000000003, 2024-09-13T07:04:29.312 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.312 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:29.312 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414172+0000", 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427200+0000", 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013028098 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427229+0000", 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9178000000000001e-05 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427239+0000", 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6870000000000006e-06 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427393+0000", 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015382899999999999 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427451+0000", 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8301000000000003e-05 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459232+0000", 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031780695999999997 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459282+0000", 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0701000000000001e-05 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.316 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:176785 3.7 3:e640995f:::rbd_data.1194987e4741.0000000000000011:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-13T07:04:29.317 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:09.441952+0000", 2024-09-13T07:04:29.317 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.795647343999999, 2024-09-13T07:04:29.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86669136400000002, 2024-09-13T07:04:29.317 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.317 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:29.317 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.317 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.317 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.317 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 176785 2024-09-13T07:04:29.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.317 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.317 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441952+0000", 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441952+0000", 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441951+0000", 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441958+0000", 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3089999999999998e-06 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441970+0000", 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2404e-05 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441977+0000", 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442051+0000", 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3337000000000006e-05 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442063+0000", 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2931e-05 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305282+0000", 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86321888999999996 2024-09-13T07:04:29.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305447+0000", 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016446500000000001 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305579+0000", 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000132452 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.307846+0000", 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022664790000000001 2024-09-13T07:04:29.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:29.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308562+0000", 2024-09-13T07:04:29.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00071642899999999998 2024-09-13T07:04:29.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:29.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308606+0000", 2024-09-13T07:04:29.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4073999999999998e-05 2024-09-13T07:04:29.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308644+0000", 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7348999999999998e-05 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:176787 3.15 3:af329162:::rbd_data.1194987e4741.0000000000000013:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:09.442992+0000", 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.794607600999999, 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86944086799999998, 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.323 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 176787 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442992+0000", 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442992+0000", 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442990+0000", 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442999+0000", 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8019999999999998e-06 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443015+0000", 2024-09-13T07:04:29.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6002999999999999e-05 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443024+0000", 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3610000000000001e-06 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443092+0000", 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7967999999999997e-05 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443105+0000", 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3969e-05 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.303912+0000", 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86080616200000004 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.304190+0000", 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00027807599999999999 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:04:29.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.304362+0000", 2024-09-13T07:04:29.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017219199999999999 2024-09-13T07:04:29.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:29.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305588+0000", 2024-09-13T07:04:29.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001226499 2024-09-13T07:04:29.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:29.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312299+0000", 2024-09-13T07:04:29.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0067103789999999998 2024-09-13T07:04:29.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:29.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312377+0000", 2024-09-13T07:04:29.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8291e-05 2024-09-13T07:04:29.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312433+0000", 2024-09-13T07:04:29.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5755999999999999e-05 2024-09-13T07:04:29.329 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.329 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.329 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:176791 3.7 3:e2fcfe96:::rbd_data.1194987e4741.0000000000000018:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:09.444146+0000", 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 19.793453141000001, 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86828808400000002, 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 176791 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:29.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444146+0000", 2024-09-13T07:04:29.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:29.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444146+0000", 2024-09-13T07:04:29.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:29.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:29.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444145+0000", 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444151+0000", 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0050000000000001e-06 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444168+0000", 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6399e-05 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444175+0000", 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5610000000000001e-06 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444240+0000", 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4893000000000004e-05 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444250+0000", 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0029e-05 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.306520+0000", 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86226998899999996 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.306696+0000", 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017564800000000001 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.306820+0000", 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000123726 2024-09-13T07:04:29.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308963+0000", 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021437990000000001 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312309+0000", 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033458590000000001 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312384+0000", 2024-09-13T07:04:29.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4184000000000007e-05 2024-09-13T07:04:29.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:29.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:29.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:29.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312435+0000", 2024-09-13T07:04:29.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1019000000000001e-05 2024-09-13T07:04:29.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.336 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.336 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:29.336 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:29.336 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:29.338 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:04:29.338 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:29.338+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:29.439 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:29.438+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:29.454 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:29.454 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:29.454 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:29.454 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:29.464 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:04:29.540 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:29.537+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:29.567 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:29.567 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:29.577 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:04:29.640 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:29.638+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:29.687 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:29.688 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:29.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:29.688 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:29.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.688 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.688 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:29.688 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.08617168000001, 2024-09-13T07:04:29.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:04:29.688 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.688 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.688 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:29.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.693 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.693 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.693 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:29.693 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.086140579, 2024-09-13T07:04:29.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:04:29.693 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.693 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:29.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:04:29.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:04:29.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:04:29.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:04:29.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:04:29.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:04:29.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:04:29.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:04:29.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:04:29.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.697 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.698 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.698 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.698 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:29.698 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.086118265, 2024-09-13T07:04:29.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:04:29.698 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.698 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.699 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:04:29.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.699 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:29.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.702 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.703 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.703 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.703 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:29.703 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.086095234, 2024-09-13T07:04:29.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:04:29.703 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.703 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.703 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.703 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.703 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.704 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:04:29.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.704 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:29.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:29.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:04:29.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:04:29.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:04:29.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:04:29.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:04:29.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:04:29.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:04:29.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:04:29.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:04:29.708 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.708 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.708 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.708 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.08607068400001, 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:29.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:29.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:29.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:04:29.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.086034975, 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:04:29.713 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:04:29.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:04:29.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.717 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.086010296, 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:29.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:04:29.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:04:29.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.721 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.08597702199999, 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:04:29.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:04:29.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:04:29.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:04:29.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:04:29.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:04:29.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:04:29.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.726 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.08595023399999, 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:04:29.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:04:29.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:04:29.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:04:29.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:04:29.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:04:29.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:04:29.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.08592568, 2024-09-13T07:04:29.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:29.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:29.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:04:29.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:04:29.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:04:29.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.736 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.736 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.736 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:29.736 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.08589591099999, 2024-09-13T07:04:29.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:04:29.736 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.736 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:04:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:04:29.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:04:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:04:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:04:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:04:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.08583606800001, 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:04:29.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:04:29.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:04:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:04:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:04:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:04:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.085769218, 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:04:29.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:04:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:04:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:04:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:04:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.085736716, 2024-09-13T07:04:29.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:04:29.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:04:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:04:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:04:29.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:04:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:04:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:04:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.755 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.085702095, 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:29.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:04:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:04:29.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:04:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:04:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.085651864, 2024-09-13T07:04:29.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:04:29.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:04:29.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:04:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:04:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:04:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:04:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:04:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.085603678, 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:04:29.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:04:29.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.08557704, 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:04:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:04:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:04:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:04:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:04:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:04:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:04:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:04:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:04:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:04:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.774 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.085530205, 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:29.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:04:29.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:04:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:04:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:04:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:04:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:04:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:04:29.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.082175788, 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:04:29.779 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:04:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:04:29.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:04:29.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:04:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:04:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:04:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:04:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:04:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:29.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:04:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:04:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:29.784 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:29.784 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:29.785 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T07:04:29.786 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:04:29.787 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:29.781+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:29.886 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:29.885+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:29.940 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:29.944 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:29.945 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:29.945 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:29.945 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:29.954 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:04:29.961 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:29.982 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:29.987 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:29.986+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:30.000 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:30.020 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:30.047 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:30.066 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:30.066 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:30.076 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:04:30.079 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:30.079 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:30.079 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:30.079 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:30.079 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:30.079 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:30.079 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:30.080 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:30.080 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:30.082 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:30.083 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:30.087 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:30.085+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:30.149 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is set 2024-09-13T07:04:30.188 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:30.188 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:30.188 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:30.186+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:30.197 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:04:30.289 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:30.290+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:30.350 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:30.350 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:30.360 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:04:30.390 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:30.385+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:30.466 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:30.467 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:30.476 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:04:30.491 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:30.489+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:30.586 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:30.586 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:30.591 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:30.589+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:30.595 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:04:30.693 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:30.693+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:30.715 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:30.716 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:30.726 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:04:30.794 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:30.794+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:30.831 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:30.832 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:30.841 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:04:30.895 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:30.893+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:30.949 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:30.950 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:30.959 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:04:31.005 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:31.006+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:31.068 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:31.069 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:31.078 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:04:31.097 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:31.093+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:31.138 INFO:tasks.thrashosds.thrasher:in_osds: [3, 4, 5, 6, 2, 0, 1] out_osds: [7] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-13T07:04:31.138 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T07:04:31.138 INFO:tasks.thrashosds.thrasher:Reviving osd 5 2024-09-13T07:04:31.138 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T07:04:31.139 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T07:04:31.141 INFO:tasks.ceph.osd.5:Started 2024-09-13T07:04:31.141 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:04:31.195 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:31.195 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:31.198 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:31.198+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:31.204 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:04:31.243 INFO:teuthology.orchestra.run.smithi164.stderr:no valid command found; 10 closest matches: 2024-09-13T07:04:31.244 INFO:teuthology.orchestra.run.smithi164.stderr:0 2024-09-13T07:04:31.244 INFO:teuthology.orchestra.run.smithi164.stderr:1 2024-09-13T07:04:31.244 INFO:teuthology.orchestra.run.smithi164.stderr:2 2024-09-13T07:04:31.244 INFO:teuthology.orchestra.run.smithi164.stderr:abort 2024-09-13T07:04:31.244 INFO:teuthology.orchestra.run.smithi164.stderr:assert 2024-09-13T07:04:31.244 INFO:teuthology.orchestra.run.smithi164.stderr:config diff 2024-09-13T07:04:31.244 INFO:teuthology.orchestra.run.smithi164.stderr:config diff get 2024-09-13T07:04:31.244 INFO:teuthology.orchestra.run.smithi164.stderr:config get 2024-09-13T07:04:31.244 INFO:teuthology.orchestra.run.smithi164.stderr:config help [] 2024-09-13T07:04:31.244 INFO:teuthology.orchestra.run.smithi164.stderr:config set ... 2024-09-13T07:04:31.244 INFO:teuthology.orchestra.run.smithi164.stderr:admin_socket: invalid command 2024-09-13T07:04:31.246 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T07:04:31.246 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-13T07:04:31.299 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:31.298+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:31.312 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:31.312 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:31.324 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:04:31.400 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:31.397+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:31.427 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:31.427 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:31.437 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:04:31.501 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:31.497+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:31.546 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:31.546 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:31.555 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:04:31.602 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:31.601+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:31.661 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:31.661 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:31.671 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:04:31.702 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:31.702+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:31.768 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:04:31.765+0000 7f60a2f3d880 -1 Falling back to public interface 2024-09-13T07:04:31.778 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:31.778 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:31.788 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:04:31.803 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:31.801+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:31.895 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:31.895 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:31.904 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:04:31.905 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-09-13T07:04:31.902+0000 7fdd8576c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22269) UID: 0 2024-09-13T07:04:32.005 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:32.006+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:32.010 INFO:teuthology.orchestra.run.smithi118.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:32.010 INFO:teuthology.orchestra.run.smithi118.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:32.020 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:04:32.084 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T07:04:32.107 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:32.106+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:32.137 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:32.137 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:32.148 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:04:32.162 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T07:04:32.209 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:32.210+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:32.234 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0: {} 2024-09-13T07:04:32.254 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1: {} 2024-09-13T07:04:32.254 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-13T07:04:32.255 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T07:04:32.265 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:04:32.282 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2: {} 2024-09-13T07:04:32.300 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3: {} 2024-09-13T07:04:32.308 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-13T07:04:32.305+0000 7f9dfb773700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22307) UID: 0 2024-09-13T07:04:32.323 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4: {} 2024-09-13T07:04:32.349 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6: {} 2024-09-13T07:04:32.371 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7: {} 2024-09-13T07:04:32.371 INFO:teuthology.orchestra.run.smithi118.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T07:04:32.371 INFO:teuthology.orchestra.run.smithi118.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T07:04:32.371 INFO:teuthology.orchestra.run.smithi118.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T07:04:32.371 INFO:teuthology.orchestra.run.smithi118.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T07:04:32.371 INFO:teuthology.orchestra.run.smithi118.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T07:04:32.371 INFO:teuthology.orchestra.run.smithi118.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T07:04:32.371 INFO:teuthology.orchestra.run.smithi118.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T07:04:32.371 INFO:teuthology.orchestra.run.smithi118.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T07:04:32.372 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.769634913, 2024-09-13T07:04:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:04:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:04:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:32.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:04:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:04:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:04:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:04:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:04:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:04:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.769603812, 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:04:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:32.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:04:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:04:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:04:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:04:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:04:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:04:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:04:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:04:32.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:04:32.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:04:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.76958149799999, 2024-09-13T07:04:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:04:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:04:32.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:04:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:04:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:04:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:04:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:04:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:04:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:04:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:04:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:04:32.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:04:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:04:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:32.388 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.769558467, 2024-09-13T07:04:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:04:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:04:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:04:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:04:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:04:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:04:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:04:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:04:32.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:04:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:04:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:04:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:04:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.769533917, 2024-09-13T07:04:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:04:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:04:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:04:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:04:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:04:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:04:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:04:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:04:32.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:04:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:04:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:04:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:04:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.769498208, 2024-09-13T07:04:32.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:04:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:04:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:04:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:04:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:04:32.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:04:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:04:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.769473529, 2024-09-13T07:04:32.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:04:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:04:32.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:04:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:04:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:04:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:04:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:04:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:04:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:04:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:04:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.76944025500001, 2024-09-13T07:04:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:04:32.408 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:04:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:04:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:04:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:04:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:04:32.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:04:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:04:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:04:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:04:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:04:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:04:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:04:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.76941346700001, 2024-09-13T07:04:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:04:32.413 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:04:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:04:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:04:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:04:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:04:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:04:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:04:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:04:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:04:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:04:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:04:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:04:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:04:32.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:04:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:04:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:04:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:32.418 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.769388913, 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:04:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:04:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:04:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:04:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:04:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:04:32.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.76935914400001, 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:04:32.423 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:04:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:32.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:04:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:04:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:04:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:04:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:04:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:04:32.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:04:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:04:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:04:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:04:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:04:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:04:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:04:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:04:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.428 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.769299301, 2024-09-13T07:04:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:04:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:04:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:04:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:04:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:04:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:04:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:04:32.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:04:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:04:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:04:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:04:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:04:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:04:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:04:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:04:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:04:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.76923245099999, 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:04:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:04:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:04:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:04:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:04:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:04:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:04:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:04:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:04:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:04:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:04:32.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:04:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:04:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:04:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:04:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.769199949, 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.439 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:04:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:04:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:04:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:04:32.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:04:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:04:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:04:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:04:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:04:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:04:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:04:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:04:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:04:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:04:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.769165328, 2024-09-13T07:04:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:04:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.444 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:04:32.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:04:32.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:04:32.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:04:32.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:04:32.448 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.448 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.448 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.449 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.449 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:32.449 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.769115097, 2024-09-13T07:04:32.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:04:32.449 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.449 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.449 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:04:32.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.449 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:32.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:32.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:04:32.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:04:32.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:04:32.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:04:32.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:04:32.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:04:32.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:04:32.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:04:32.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:04:32.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:04:32.453 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.453 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.453 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.769066911, 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.454 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:32.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:32.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:32.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:04:32.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:04:32.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:04:32.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:04:32.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:04:32.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.459 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.459 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:32.459 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.769040273, 2024-09-13T07:04:32.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:04:32.459 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.459 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.459 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.459 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.459 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.459 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:04:32.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.459 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:04:32.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:04:32.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:04:32.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:04:32.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:04:32.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:04:32.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:04:32.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:04:32.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:04:32.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:04:32.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.463 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.464 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:32.464 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.768993438, 2024-09-13T07:04:32.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:04:32.464 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.464 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.464 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.464 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.464 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.464 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:04:32.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.464 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:32.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:32.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:04:32.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:04:32.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:04:32.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:04:32.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:04:32.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:32.468 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.765639021, 2024-09-13T07:04:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:04:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:04:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:04:32.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:04:32.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:04:32.473 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.473 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.473 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.473 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.473 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.473 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:32.474 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T07:04:32.475 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T07:04:32.475 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:32.474+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:32.476 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:04:32.576 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:32.573+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:32.590 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:32.590 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:32.590 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:32.590 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:32.600 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:04:32.677 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:32.678+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:32.702 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:32.702 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:32.703 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:32.703 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:32.703 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:32.712 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:04:32.778 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:32.777+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:32.823 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:04:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 186.76305917600001, 2024-09-13T07:04:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:04:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:04:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:04:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:04:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:04:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:04:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:04:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:04:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.205275298, 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:04:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:04:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:04:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:04:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:04:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.205252331, 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:04:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:04:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:04:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:04:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:04:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:04:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:04:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:04:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:04:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:04:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.205235153, 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:04:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:04:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:04:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:04:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:04:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.205221288, 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:04:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.20520814299999, 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:04:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:04:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:04:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:04:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:04:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:04:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:04:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:04:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.205194936, 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:04:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:04:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:04:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:04:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:04:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:04:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.205180869, 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:04:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:04:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:04:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.205160855, 2024-09-13T07:04:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:04:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:04:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:04:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:04:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:04:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:04:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:04:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:04:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:04:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.205141105, 2024-09-13T07:04:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:04:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:04:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:04:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:04:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:04:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:04:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:04:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.205116299, 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:04:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:04:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:04:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:04:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:04:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:04:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:04:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:04:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:04:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:04:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:04:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:04:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:04:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.205092034, 2024-09-13T07:04:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:04:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.205050301, 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:04:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:04:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:04:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.20501401200001, 2024-09-13T07:04:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:04:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:04:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:04:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 110.76191068, 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:04:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:04:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:04:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 110.761893798, 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:04:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:04:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:04:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:04:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:04:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:04:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:04:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:04:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 110.760843241, 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:04:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 110.759168692, 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:04:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:04:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:04:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:04:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:04:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:04:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:04:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:04:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:04:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:04:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.238586302, 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:04:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:04:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:04:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172880 2.2 2:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e149)", 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.684290056000002, 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.97218948500000002, 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:32.910 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172880 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:32.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137687+0000", 2024-09-13T07:04:32.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:32.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:32.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137702+0000", 2024-09-13T07:04:32.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5023e-05 2024-09-13T07:04:32.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:32.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137730+0000", 2024-09-13T07:04:32.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7719e-05 2024-09-13T07:04:32.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137748+0000", 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8189000000000002e-05 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137891+0000", 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014330699999999999 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137903+0000", 2024-09-13T07:04:32.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1688999999999999e-05 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142887+0000", 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004984006 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142898+0000", 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.079e-05 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:32.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109703+0000", 2024-09-13T07:04:32.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96680553899999999 2024-09-13T07:04:32.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:32.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:32.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:32.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109879+0000", 2024-09-13T07:04:32.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017579800000000001 2024-09-13T07:04:32.915 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.915 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.915 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.915 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:32.915 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:32.915 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:32.917 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:04:32.917 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:32.917+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:33.017 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:33.018+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:33.065 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:33.065 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:33.065 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:33.065 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:33.074 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:04:33.118 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:33.118+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:33.175 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is set 2024-09-13T07:04:33.182 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:33.182 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:33.182 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:33.182 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:33.182 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:33.194 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:04:33.238 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-13T07:04:33.237+0000 7f0147f0e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22305) UID: 0 2024-09-13T07:04:33.301 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:04:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 187.24562345699999, 2024-09-13T07:04:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:04:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:04:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:04:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:04:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:04:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:04:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:04:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:04:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:04:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:04:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:04:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 155.08774222299999, 2024-09-13T07:04:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:04:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:04:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:04:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:04:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:04:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:04:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:04:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:04:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:04:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:04:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:04:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:04:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:04:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:04:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:04:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:04:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 155.06197251200001, 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:04:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:04:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:04:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:04:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:04:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:04:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:04:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:04:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:04:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:04:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:04:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:04:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:04:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:04:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:04:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:04:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:04:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 155.040821346, 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:04:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:04:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:04:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:04:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:04:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:04:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:04:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:04:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:04:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:04:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:04:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:04:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:04:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:04:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:04:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:04:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:04:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:04:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 144.92377658300001, 2024-09-13T07:04:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:04:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:04:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:04:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:04:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:04:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:04:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:04:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:04:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:04:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:04:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:04:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:04:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:04:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:04:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:04:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:04:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:04:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.63877933800001, 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:04:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:04:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:04:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:04:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:04:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:04:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.3870702, 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:04:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:04:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:04:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:04:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:04:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:04:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.387052166, 2024-09-13T07:04:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:04:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:04:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:04:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:04:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:04:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:04:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:04:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:04:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:04:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:33.346 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.38703608599999, 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:33.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:04:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:04:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:04:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:04:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:04:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:04:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:04:33.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:04:33.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.38702041000001, 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:04:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:04:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:04:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:04:33.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.38699748800001, 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:04:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:33.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:04:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:04:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.38698139100001, 2024-09-13T07:04:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:04:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.359 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:04:33.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.38637977899999, 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:04:33.363 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:04:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:04:33.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:04:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:04:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:04:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:04:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:33.367 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.38633700299999, 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:33.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:04:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:04:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:04:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:04:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:04:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:04:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:04:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:04:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:04:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:04:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.229319233, 2024-09-13T07:04:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:04:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:04:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:04:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:04:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:04:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:04:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:04:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.376942303000007, 2024-09-13T07:04:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:04:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:04:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:04:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:04:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:04:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:04:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:04:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:04:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:164179 3.14 e135/67 3:2fc485bd:::rbd_data.11942e4f5fc7.0000000000000002:head v 135'401, mlcod=132'398)", 2024-09-13T07:04:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.354001848999999, 2024-09-13T07:04:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063492174999999998, 2024-09-13T07:04:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945792+0000", 2024-09-13T07:04:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954811+0000", 2024-09-13T07:04:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090187940000000001 2024-09-13T07:04:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954840+0000", 2024-09-13T07:04:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8659000000000001e-05 2024-09-13T07:04:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954852+0000", 2024-09-13T07:04:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1966e-05 2024-09-13T07:04:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954938+0000", 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6539000000000006e-05 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954975+0000", 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6303999999999997e-05 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009206+0000", 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.054231044999999999 2024-09-13T07:04:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009286+0000", 2024-09-13T07:04:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700999999999999e-05 2024-09-13T07:04:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:165953 3.d 3:b3a20b45:::rbd_data.11942e4f5fc7.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8796117~39260] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.077554290999998, 2024-09-13T07:04:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.095290105, 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 165953 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222240+0000", 2024-09-13T07:04:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258929+0000", 2024-09-13T07:04:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.036688401000000002 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258971+0000", 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2212000000000002e-05 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258989+0000", 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7674000000000001e-05 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259072+0000", 2024-09-13T07:04:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3656e-05 2024-09-13T07:04:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259297+0000", 2024-09-13T07:04:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022450700000000001 2024-09-13T07:04:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.260463+0000", 2024-09-13T07:04:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011665340000000001 2024-09-13T07:04:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.316214+0000", 2024-09-13T07:04:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.055750268999999998 2024-09-13T07:04:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317335+0000", 2024-09-13T07:04:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011211719999999999 2024-09-13T07:04:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317451+0000", 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011616 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317532+0000", 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0860999999999997e-05 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172852 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [call lock.unlock in=52b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-13T07:04:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.101097144000001, 2024-09-13T07:04:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.90390594199999996, 2024-09-13T07:04:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172852 2024-09-13T07:04:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198696+0000", 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198710+0000", 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3548e-05 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198739+0000", 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9606000000000001e-05 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198752+0000", 2024-09-13T07:04:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3033e-05 2024-09-13T07:04:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198869+0000", 2024-09-13T07:04:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116552 2024-09-13T07:04:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199018+0000", 2024-09-13T07:04:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014923700000000001 2024-09-13T07:04:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199308+0000", 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028947199999999998 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.200731+0000", 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014229799999999999 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059443+0000", 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.85871189999999997 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059467+0000", 2024-09-13T07:04:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.478e-05 2024-09-13T07:04:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102380+0000", 2024-09-13T07:04:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042912463999999997 2024-09-13T07:04:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102605+0000", 2024-09-13T07:04:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022489499999999999 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172853 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [watch reconnect cookie 140335574242864 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 49.249348285000004, 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.056267725999999997, 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172853 2024-09-13T07:04:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050445+0000", 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050461+0000", 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5783000000000001e-05 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050497+0000", 2024-09-13T07:04:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5995999999999999e-05 2024-09-13T07:04:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050516+0000", 2024-09-13T07:04:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9579999999999999e-05 2024-09-13T07:04:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050637+0000", 2024-09-13T07:04:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120536 2024-09-13T07:04:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050647+0000", 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6989999999999998e-06 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059533+0000", 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088858749999999997 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059545+0000", 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2473000000000001e-05 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102962+0000", 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043416656999999997 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.103612+0000", 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000650374 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.106715+0000", 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031033229999999998 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:33.404 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:33.405 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:33.401+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:33.405 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T07:04:33.506 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:33.505+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:33.519 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:33.519 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:33.519 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:33.520 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:33.529 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T07:04:33.607 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:33.606+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:33.632 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:33.633 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:33.633 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:33.633 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:33.633 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:33.642 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T07:04:33.708 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:33.706+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:33.746 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:33.746 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:33.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:33.746 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:33.746 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.746 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137474 3.6 3:65c31370:::rbd_data.1194af11889b.000000000000013b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 186185~11715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-13T07:04:33.746 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:33.746 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 89.002368540999996, 2024-09-13T07:04:33.746 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0054357140000001, 2024-09-13T07:04:33.746 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.746 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:33.747 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.747 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.747 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.747 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137474 2024-09-13T07:04:33.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.747 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.747 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:33.747 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.747 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.747 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.747 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743935+0000", 2024-09-13T07:04:33.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743934+0000", 2024-09-13T07:04:33.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:33.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.748 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.748 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.748 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743949+0000", 2024-09-13T07:04:33.748 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5705999999999998e-05 2024-09-13T07:04:33.748 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743960+0000", 2024-09-13T07:04:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0084e-05 2024-09-13T07:04:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.743971+0000", 2024-09-13T07:04:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0942e-05 2024-09-13T07:04:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744045+0000", 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4626e-05 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.744062+0000", 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7339000000000001e-05 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746366+0000", 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.002303457 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.750 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746520+0000", 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015385900000000001 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.746734+0000", 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000214253 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749128+0000", 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002394129 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.751 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:33.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749267+0000", 2024-09-13T07:04:33.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013878299999999999 2024-09-13T07:04:33.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:33.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749328+0000", 2024-09-13T07:04:33.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0918000000000002e-05 2024-09-13T07:04:33.752 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.752 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.752 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.752 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:05.749371+0000", 2024-09-13T07:04:33.752 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3188000000000001e-05 2024-09-13T07:04:33.753 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.753 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.753 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.753 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.753 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.753 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163305 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2387968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.753 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:33.753 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.068150059999994, 2024-09-13T07:04:33.753 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044745869000000001, 2024-09-13T07:04:33.753 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.753 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.753 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163305 2024-09-13T07:04:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.754 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678154+0000", 2024-09-13T07:04:33.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678153+0000", 2024-09-13T07:04:33.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.755 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.755 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.755 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678156+0000", 2024-09-13T07:04:33.755 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6350000000000002e-06 2024-09-13T07:04:33.755 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678161+0000", 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2410000000000003e-06 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678166+0000", 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.498e-06 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722751+0000", 2024-09-13T07:04:33.756 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044585134999999998 2024-09-13T07:04:33.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722836+0000", 2024-09-13T07:04:33.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4672999999999994e-05 2024-09-13T07:04:33.757 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.757 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.757 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722900+0000", 2024-09-13T07:04:33.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4042999999999997e-05 2024-09-13T07:04:33.757 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.757 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.757 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163307 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.068129596000006, 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045060944999999998, 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163307 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678174+0000", 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678177+0000", 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9189999999999999e-06 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678183+0000", 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0599999999999996e-06 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678188+0000", 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6189999999999999e-06 2024-09-13T07:04:33.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.722941+0000", 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044752844999999999 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723073+0000", 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013128599999999999 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723235+0000", 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000162671 2024-09-13T07:04:33.761 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.762 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.762 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.762 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163310 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 544768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.762 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:33.762 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.068110644000001, 2024-09-13T07:04:33.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045290534, 2024-09-13T07:04:33.762 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.762 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163310 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678193+0000", 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678195+0000", 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.785e-06 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678201+0000", 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0529999999999997e-06 2024-09-13T07:04:33.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678205+0000", 2024-09-13T07:04:33.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6999999999999999e-06 2024-09-13T07:04:33.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723299+0000", 2024-09-13T07:04:33.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045093964 2024-09-13T07:04:33.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723384+0000", 2024-09-13T07:04:33.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4562999999999997e-05 2024-09-13T07:04:33.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723484+0000", 2024-09-13T07:04:33.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000100045 2024-09-13T07:04:33.766 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.766 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.766 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.767 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163313 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1093632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.767 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:33.767 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.068094658999996, 2024-09-13T07:04:33.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045562975999999998, 2024-09-13T07:04:33.767 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.767 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.767 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.767 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.767 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.767 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163313 2024-09-13T07:04:33.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.767 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678209+0000", 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678212+0000", 2024-09-13T07:04:33.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8650000000000001e-06 2024-09-13T07:04:33.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678217+0000", 2024-09-13T07:04:33.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2660000000000001e-06 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678221+0000", 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4159999999999997e-06 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723552+0000", 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045330489000000002 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723660+0000", 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010770800000000001 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723772+0000", 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000112608 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163317 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1642496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:33.771 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.068065851, 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045770165000000002, 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163317 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678238+0000", 2024-09-13T07:04:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678237+0000", 2024-09-13T07:04:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678241+0000", 2024-09-13T07:04:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2669999999999998e-06 2024-09-13T07:04:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678247+0000", 2024-09-13T07:04:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7309999999999996e-06 2024-09-13T07:04:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678252+0000", 2024-09-13T07:04:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8250000000000004e-06 2024-09-13T07:04:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723830+0000", 2024-09-13T07:04:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045577234000000001 2024-09-13T07:04:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.723936+0000", 2024-09-13T07:04:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000106277 2024-09-13T07:04:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724008+0000", 2024-09-13T07:04:33.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2422999999999994e-05 2024-09-13T07:04:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163320 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.068045427000001, 2024-09-13T07:04:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045968883000000002, 2024-09-13T07:04:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.776 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163320 2024-09-13T07:04:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:33.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678258+0000", 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678261+0000", 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8090000000000001e-06 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678267+0000", 2024-09-13T07:04:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8860000000000001e-06 2024-09-13T07:04:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678271+0000", 2024-09-13T07:04:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4549999999999997e-06 2024-09-13T07:04:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724061+0000", 2024-09-13T07:04:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045789656999999997 2024-09-13T07:04:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724151+0000", 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.9734999999999994e-05 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724227+0000", 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6848999999999996e-05 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.780 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163324 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.067761094000005, 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045988788000000003, 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163324 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:33.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678543+0000", 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678542+0000", 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678547+0000", 2024-09-13T07:04:33.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6829999999999999e-06 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678556+0000", 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1929999999999999e-06 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678562+0000", 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.083e-06 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724267+0000", 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045705427 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724417+0000", 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014984699999999999 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724532+0000", 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011430900000000001 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.784 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163327 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 733184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.067736060000001, 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046185902000000001, 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163327 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.785 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678568+0000", 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678567+0000", 2024-09-13T07:04:33.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678571+0000", 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3799999999999998e-06 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678576+0000", 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7760000000000003e-06 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678581+0000", 2024-09-13T07:04:33.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8640000000000004e-06 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724561+0000", 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045979178000000002 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724649+0000", 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8677000000000005e-05 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724754+0000", 2024-09-13T07:04:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104527 2024-09-13T07:04:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163331 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1282048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.067717905999999, 2024-09-13T07:04:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046361773000000002, 2024-09-13T07:04:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.789 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163331 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678586+0000", 2024-09-13T07:04:33.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678585+0000", 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678588+0000", 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0469999999999998e-06 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678595+0000", 2024-09-13T07:04:33.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0399999999999998e-06 2024-09-13T07:04:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678599+0000", 2024-09-13T07:04:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5519999999999998e-06 2024-09-13T07:04:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724780+0000", 2024-09-13T07:04:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046181168000000002 2024-09-13T07:04:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.792 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724871+0000", 2024-09-13T07:04:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0562000000000005e-05 2024-09-13T07:04:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724948+0000", 2024-09-13T07:04:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6985999999999998e-05 2024-09-13T07:04:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.793 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163334 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1662976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.067700703, 2024-09-13T07:04:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046544502000000001, 2024-09-13T07:04:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163334 2024-09-13T07:04:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678603+0000", 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678605+0000", 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5380000000000001e-06 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678611+0000", 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7389999999999996e-06 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678615+0000", 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2830000000000003e-06 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.724987+0000", 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046371969999999998 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725075+0000", 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8091999999999996e-05 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725148+0000", 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2206999999999995e-05 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.797 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163338 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.067683973000001, 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046738452999999999, 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163338 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678620+0000", 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678619+0000", 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678622+0000", 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.649e-06 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678628+0000", 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5029999999999997e-06 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678632+0000", 2024-09-13T07:04:33.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4020000000000004e-06 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725182+0000", 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046550389999999997 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725280+0000", 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.7260000000000001e-05 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725358+0000", 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8720000000000005e-05 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163339 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2363392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.067668444999995, 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046920835000000001, 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.802 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163339 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678635+0000", 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678637+0000", 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2479999999999999e-06 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678642+0000", 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0030000000000002e-06 2024-09-13T07:04:33.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678646+0000", 2024-09-13T07:04:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9870000000000001e-06 2024-09-13T07:04:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725397+0000", 2024-09-13T07:04:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046750127000000002 2024-09-13T07:04:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725482+0000", 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5928999999999997e-05 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725556+0000", 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3823000000000001e-05 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163342 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:33.806 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.067653805999996, 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047212991000000003, 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163342 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678650+0000", 2024-09-13T07:04:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678652+0000", 2024-09-13T07:04:33.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.486e-06 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678657+0000", 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6909999999999996e-06 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678660+0000", 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8009999999999999e-06 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725597+0000", 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046937372999999998 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725745+0000", 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000148314 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725863+0000", 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011768 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.810 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163345 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.066792242999995, 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046569193000000002, 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163345 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679512+0000", 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.678662+0000", 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9991498 2024-09-13T07:04:33.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679516+0000", 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000854789 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679530+0000", 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3492999999999999e-05 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679537+0000", 2024-09-13T07:04:33.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.323e-06 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725892+0000", 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046354244000000003 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.725975+0000", 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3356000000000006e-05 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726081+0000", 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105992 2024-09-13T07:04:33.814 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163349 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1298432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.066760180000003, 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046744247000000003, 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163349 2024-09-13T07:04:33.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679544+0000", 2024-09-13T07:04:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679543+0000", 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679546+0000", 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9639999999999999e-06 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679551+0000", 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8219999999999997e-06 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679554+0000", 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8629999999999999e-06 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726117+0000", 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046563081999999999 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726213+0000", 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6207999999999998e-05 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726288+0000", 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4741999999999997e-05 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163350 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1449984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.066747554000003, 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046935635000000003, 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.819 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163350 2024-09-13T07:04:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.820 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679556+0000", 2024-09-13T07:04:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679558+0000", 2024-09-13T07:04:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.928e-06 2024-09-13T07:04:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679561+0000", 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7659999999999999e-06 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679564+0000", 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4960000000000001e-06 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726327+0000", 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046762321000000003 2024-09-13T07:04:33.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726415+0000", 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8487999999999997e-05 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726492+0000", 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.6995000000000004e-05 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.823 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163353 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.066737445000001, 2024-09-13T07:04:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047130755000000003, 2024-09-13T07:04:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163353 2024-09-13T07:04:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.824 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679566+0000", 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679568+0000", 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5969999999999999e-06 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679571+0000", 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7429999999999998e-06 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679573+0000", 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:04:33.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726523+0000", 2024-09-13T07:04:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.046949882999999998 2024-09-13T07:04:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726616+0000", 2024-09-13T07:04:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.3236e-05 2024-09-13T07:04:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726697+0000", 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0834000000000006e-05 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:163356 3.1 3:85aa3893:::rbd_data.119453884147.0000000000000001:30 [read 2379776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.066727763000003, 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047304662999999997, 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.828 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 163356 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679576+0000", 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679578+0000", 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679582+0000", 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.679589+0000", 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1149999999999996e-06 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726723+0000", 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.047133768999999999 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726809+0000", 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5402000000000007e-05 2024-09-13T07:04:33.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:28.726881+0000", 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2152999999999998e-05 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:166663 3.b 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8129196~12979] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 64.348353248999999, 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.061564718999999997, 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:33.832 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 166663 2024-09-13T07:04:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397951+0000", 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.397947+0000", 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999967 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412144+0000", 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014196972 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412172+0000", 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7781999999999999e-05 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412182+0000", 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6069999999999997e-06 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412239+0000", 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.732e-05 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.412365+0000", 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000125356 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.413178+0000", 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00081355299999999995 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.443176+0000", 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.029997903999999999 2024-09-13T07:04:33.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459446+0000", 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016269921999999999 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459484+0000", 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8488000000000001e-05 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459515+0000", 2024-09-13T07:04:33.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0981e-05 2024-09-13T07:04:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:33.838 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:33.838 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:33.839 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T07:04:33.840 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:33.837+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:33.940 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-09-13T07:04:33.938+0000 7faf961ac700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22266) UID: 0 2024-09-13T07:04:33.993 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:33.993 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:33.993 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:33.993 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:34.003 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T07:04:34.041 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:34.038+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:34.105 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:34.106 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:34.106 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:34.115 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T07:04:34.157 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-09-13T07:04:34.158+0000 7f4a8591d700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22263) UID: 0 2024-09-13T07:04:34.242 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-09-13T07:04:34.242+0000 7fb494131700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22265) UID: 0 2024-09-13T07:04:34.272 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:34.272 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:34.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:34.273 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:34.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.273 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:12954 3.0 3:07a06967:::rbd_header.1194c5936dae:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T07:04:34.273 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:34.273 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 170.77938737400001, 2024-09-13T07:04:34.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97129882099999998, 2024-09-13T07:04:34.273 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.273 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.273 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.273 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.273 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.273 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 12954 2024-09-13T07:04:34.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.274 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:34.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493636+0000", 2024-09-13T07:04:34.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493634+0000", 2024-09-13T07:04:34.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:34.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493647+0000", 2024-09-13T07:04:34.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2813000000000001e-05 2024-09-13T07:04:34.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493672+0000", 2024-09-13T07:04:34.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5463e-05 2024-09-13T07:04:34.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493685+0000", 2024-09-13T07:04:34.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.258e-05 2024-09-13T07:04:34.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493780+0000", 2024-09-13T07:04:34.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4968000000000003e-05 2024-09-13T07:04:34.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:34.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:43.493796+0000", 2024-09-13T07:04:34.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6237999999999999e-05 2024-09-13T07:04:34.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464329+0000", 2024-09-13T07:04:34.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.97053276899999996 2024-09-13T07:04:34.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464584+0000", 2024-09-13T07:04:34.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00025492599999999999 2024-09-13T07:04:34.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:44.464935+0000", 2024-09-13T07:04:34.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000351367 2024-09-13T07:04:34.278 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46571 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 122880~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.29290381000001, 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054246351999999998, 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46571 2024-09-13T07:04:34.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.280 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:34.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:34.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980120+0000", 2024-09-13T07:04:34.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.281 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.281 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980122+0000", 2024-09-13T07:04:34.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.322e-06 2024-09-13T07:04:34.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980127+0000", 2024-09-13T07:04:34.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1739999999999999e-06 2024-09-13T07:04:34.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980134+0000", 2024-09-13T07:04:34.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.6039999999999998e-06 2024-09-13T07:04:34.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.032400+0000", 2024-09-13T07:04:34.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.052265905000000001 2024-09-13T07:04:34.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034249+0000", 2024-09-13T07:04:34.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0018496719999999999 2024-09-13T07:04:34.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034366+0000", 2024-09-13T07:04:34.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00011698500000000001 2024-09-13T07:04:34.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.283 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.283 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.284 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46577 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 90112~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.284 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:34.284 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.29283388100001, 2024-09-13T07:04:34.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054458270000000003, 2024-09-13T07:04:34.284 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.284 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.284 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.284 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46577 2024-09-13T07:04:34.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.285 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:34.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.285 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.285 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980190+0000", 2024-09-13T07:04:34.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980189+0000", 2024-09-13T07:04:34.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980192+0000", 2024-09-13T07:04:34.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4650000000000001e-06 2024-09-13T07:04:34.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980197+0000", 2024-09-13T07:04:34.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1000000000000003e-06 2024-09-13T07:04:34.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980202+0000", 2024-09-13T07:04:34.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:04:34.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034410+0000", 2024-09-13T07:04:34.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054208244000000003 2024-09-13T07:04:34.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034516+0000", 2024-09-13T07:04:34.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000105914 2024-09-13T07:04:34.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034648+0000", 2024-09-13T07:04:34.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00013224999999999999 2024-09-13T07:04:34.288 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.288 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.288 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46578 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 126976~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.292818009, 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054705003000000002, 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46578 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:34.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980206+0000", 2024-09-13T07:04:34.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980205+0000", 2024-09-13T07:04:34.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980208+0000", 2024-09-13T07:04:34.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3429999999999998e-06 2024-09-13T07:04:34.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980213+0000", 2024-09-13T07:04:34.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1950000000000002e-06 2024-09-13T07:04:34.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980218+0000", 2024-09-13T07:04:34.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.741e-06 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034695+0000", 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054477415000000001 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034790+0000", 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5183e-05 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034911+0000", 2024-09-13T07:04:34.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012045 2024-09-13T07:04:34.293 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.293 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.293 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.293 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46585 3.0 3:023a53e7:::rbd_data.119437d9d50d.0000000000000005:13 [read 94208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.293 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:34.293 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.29274755200001, 2024-09-13T07:04:34.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054848225, 2024-09-13T07:04:34.293 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.293 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.293 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.294 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.294 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.294 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46585 2024-09-13T07:04:34.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.294 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:34.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.294 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.294 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980276+0000", 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980278+0000", 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.7800000000000001e-06 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980284+0000", 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5829999999999997e-06 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.980289+0000", 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9339999999999997e-06 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.034950+0000", 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.054661462000000001 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035030+0000", 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9297999999999999e-05 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.035124+0000", 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4770000000000002e-05 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.297 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46638 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.28939551400001, 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053455849, 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46638 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:34.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:34.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983628+0000", 2024-09-13T07:04:34.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983630+0000", 2024-09-13T07:04:34.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3549999999999999e-06 2024-09-13T07:04:34.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983636+0000", 2024-09-13T07:04:34.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4829999999999999e-06 2024-09-13T07:04:34.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983640+0000", 2024-09-13T07:04:34.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.6210000000000003e-06 2024-09-13T07:04:34.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037075+0000", 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053434322999999999 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037084+0000", 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4120000000000007e-06 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.302 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46642 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.302 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:34.302 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.28937921299999, 2024-09-13T07:04:34.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053487692000000003, 2024-09-13T07:04:34.302 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.302 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.302 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.302 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.302 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.302 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46642 2024-09-13T07:04:34.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.302 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:34.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983645+0000", 2024-09-13T07:04:34.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983644+0000", 2024-09-13T07:04:34.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983647+0000", 2024-09-13T07:04:34.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3760000000000002e-06 2024-09-13T07:04:34.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983654+0000", 2024-09-13T07:04:34.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:04:34.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983660+0000", 2024-09-13T07:04:34.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4180000000000002e-06 2024-09-13T07:04:34.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037123+0000", 2024-09-13T07:04:34.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053462724000000003 2024-09-13T07:04:34.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037132+0000", 2024-09-13T07:04:34.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2730000000000008e-06 2024-09-13T07:04:34.305 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46643 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.28935942800001, 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053514994000000003, 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.306 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46643 2024-09-13T07:04:34.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.307 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:34.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:34.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983664+0000", 2024-09-13T07:04:34.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983666+0000", 2024-09-13T07:04:34.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2110000000000001e-06 2024-09-13T07:04:34.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.308 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.308 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983672+0000", 2024-09-13T07:04:34.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6319999999999996e-06 2024-09-13T07:04:34.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983677+0000", 2024-09-13T07:04:34.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7620000000000003e-06 2024-09-13T07:04:34.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037171+0000", 2024-09-13T07:04:34.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053494182000000001 2024-09-13T07:04:34.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037179+0000", 2024-09-13T07:04:34.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5809999999999997e-06 2024-09-13T07:04:34.312 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.312 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.312 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.313 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46645 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.313 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:34.313 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.28934315500001, 2024-09-13T07:04:34.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053544298999999997, 2024-09-13T07:04:34.313 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.313 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.313 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.313 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46645 2024-09-13T07:04:34.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983681+0000", 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983680+0000", 2024-09-13T07:04:34.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983683+0000", 2024-09-13T07:04:34.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3690000000000001e-06 2024-09-13T07:04:34.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.315 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.315 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.315 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983688+0000", 2024-09-13T07:04:34.315 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3070000000000002e-06 2024-09-13T07:04:34.315 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983693+0000", 2024-09-13T07:04:34.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.758e-06 2024-09-13T07:04:34.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037215+0000", 2024-09-13T07:04:34.316 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053522198999999999 2024-09-13T07:04:34.316 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.316 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.316 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.316 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037225+0000", 2024-09-13T07:04:34.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9680000000000001e-06 2024-09-13T07:04:34.317 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.317 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.317 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.317 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.317 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.317 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46647 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.317 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:34.317 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.28932533, 2024-09-13T07:04:34.317 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053570575000000002, 2024-09-13T07:04:34.317 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.317 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.317 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.318 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.318 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.318 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46647 2024-09-13T07:04:34.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.318 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.318 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:34.318 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.318 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.318 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.318 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983698+0000", 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983701+0000", 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.424e-06 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.319 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983706+0000", 2024-09-13T07:04:34.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1429999999999997e-06 2024-09-13T07:04:34.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983710+0000", 2024-09-13T07:04:34.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.634e-06 2024-09-13T07:04:34.320 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.320 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.320 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.320 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037260+0000", 2024-09-13T07:04:34.320 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053549883 2024-09-13T07:04:34.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.321 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.321 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037269+0000", 2024-09-13T07:04:34.321 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8400000000000001e-06 2024-09-13T07:04:34.321 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.321 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.321 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.321 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.321 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.321 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46650 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.321 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:34.321 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.289309237, 2024-09-13T07:04:34.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053583504999999997, 2024-09-13T07:04:34.322 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.322 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.322 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.322 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.322 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.322 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46650 2024-09-13T07:04:34.322 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.322 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.322 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.322 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.322 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:34.322 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983715+0000", 2024-09-13T07:04:34.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.323 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.323 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983714+0000", 2024-09-13T07:04:34.323 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.323 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.323 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983720+0000", 2024-09-13T07:04:34.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:04:34.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983725+0000", 2024-09-13T07:04:34.324 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7370000000000003e-06 2024-09-13T07:04:34.324 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.324 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.324 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.324 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983731+0000", 2024-09-13T07:04:34.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7440000000000004e-06 2024-09-13T07:04:34.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.325 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037288+0000", 2024-09-13T07:04:34.325 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053556751999999999 2024-09-13T07:04:34.325 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.325 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.325 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037298+0000", 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0127e-05 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46652 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.289288051, 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053585199, 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.326 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46652 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983736+0000", 2024-09-13T07:04:34.327 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983735+0000", 2024-09-13T07:04:34.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.328 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983738+0000", 2024-09-13T07:04:34.328 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5349999999999999e-06 2024-09-13T07:04:34.328 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.328 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.328 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983743+0000", 2024-09-13T07:04:34.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0780000000000003e-06 2024-09-13T07:04:34.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983748+0000", 2024-09-13T07:04:34.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5750000000000002e-06 2024-09-13T07:04:34.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.329 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.329 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.329 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037314+0000", 2024-09-13T07:04:34.329 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566510999999997 2024-09-13T07:04:34.329 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.330 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.330 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037321+0000", 2024-09-13T07:04:34.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8750000000000002e-06 2024-09-13T07:04:34.330 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.330 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.330 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.330 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.330 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.330 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46654 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.330 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:34.330 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.28927201799999, 2024-09-13T07:04:34.330 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053594546999999999, 2024-09-13T07:04:34.331 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.331 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.331 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.331 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.331 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.331 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46654 2024-09-13T07:04:34.331 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.331 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.331 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.331 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.331 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:34.331 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983752+0000", 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983751+0000", 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983754+0000", 2024-09-13T07:04:34.332 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.678e-06 2024-09-13T07:04:34.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983760+0000", 2024-09-13T07:04:34.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4739999999999996e-06 2024-09-13T07:04:34.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.333 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983771+0000", 2024-09-13T07:04:34.333 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.1408e-05 2024-09-13T07:04:34.333 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.333 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.333 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037337+0000", 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566074999999998 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037346+0000", 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2569999999999996e-06 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46657 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.334 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.289222779, 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053588231, 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46657 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.335 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983801+0000", 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983800+0000", 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983803+0000", 2024-09-13T07:04:34.336 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9500000000000001e-06 2024-09-13T07:04:34.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983809+0000", 2024-09-13T07:04:34.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0610000000000001e-06 2024-09-13T07:04:34.337 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.337 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.337 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.337 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983815+0000", 2024-09-13T07:04:34.337 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:04:34.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037381+0000", 2024-09-13T07:04:34.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053566067000000002 2024-09-13T07:04:34.338 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.338 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.338 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.338 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037389+0000", 2024-09-13T07:04:34.338 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5879999999999998e-06 2024-09-13T07:04:34.338 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.339 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.339 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.339 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.339 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.339 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46659 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.339 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:34.339 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.289204719, 2024-09-13T07:04:34.339 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053616901000000002, 2024-09-13T07:04:34.339 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.339 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.339 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.339 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.339 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.340 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46659 2024-09-13T07:04:34.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.340 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:34.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.340 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.340 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.340 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:34.340 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.340 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983819+0000", 2024-09-13T07:04:34.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.341 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983821+0000", 2024-09-13T07:04:34.341 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8260000000000001e-06 2024-09-13T07:04:34.341 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.341 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.341 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983827+0000", 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3419999999999998e-06 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983831+0000", 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7469999999999996e-06 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037425+0000", 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053593505999999999 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.342 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.343 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.343 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037436+0000", 2024-09-13T07:04:34.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0953e-05 2024-09-13T07:04:34.343 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.343 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.343 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.343 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.343 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.343 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46661 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.343 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:34.343 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.289188204, 2024-09-13T07:04:34.343 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053641842000000002, 2024-09-13T07:04:34.344 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.344 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.344 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.344 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.344 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.344 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46661 2024-09-13T07:04:34.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.344 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.344 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.344 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.344 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:34.344 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.344 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983836+0000", 2024-09-13T07:04:34.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983835+0000", 2024-09-13T07:04:34.345 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.345 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.345 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.345 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.345 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983837+0000", 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2290000000000002e-06 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983843+0000", 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5079999999999996e-06 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983847+0000", 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5600000000000004e-06 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.346 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037469+0000", 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053621448000000002 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037477+0000", 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4780000000000008e-06 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46665 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.347 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.28916983900001, 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053668051000000001, 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46665 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:34.348 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:34.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.349 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983854+0000", 2024-09-13T07:04:34.349 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.349 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.349 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.349 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983856+0000", 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3180000000000001e-06 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983861+0000", 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1209999999999998e-06 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983866+0000", 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4859999999999999e-06 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.350 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037512+0000", 2024-09-13T07:04:34.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053646155000000001 2024-09-13T07:04:34.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.351 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.351 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.351 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037522+0000", 2024-09-13T07:04:34.351 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0281e-05 2024-09-13T07:04:34.351 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.351 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.351 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.351 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.352 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.352 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46667 3.0 3.f211020 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.352 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:34.352 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.28915374100001, 2024-09-13T07:04:34.352 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053693013999999997, 2024-09-13T07:04:34.352 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.352 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.352 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.352 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.352 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.352 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46667 2024-09-13T07:04:34.352 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.352 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:34.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.353 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:34.353 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.353 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.353 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.353 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983870+0000", 2024-09-13T07:04:34.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983872+0000", 2024-09-13T07:04:34.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3719999999999999e-06 2024-09-13T07:04:34.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.354 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.354 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.354 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983877+0000", 2024-09-13T07:04:34.354 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.366e-06 2024-09-13T07:04:34.354 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983882+0000", 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4440000000000001e-06 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037554+0000", 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053672603999999999 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037563+0000", 2024-09-13T07:04:34.355 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5790000000000004e-06 2024-09-13T07:04:34.356 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.356 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.356 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.356 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.356 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46669 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.356 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:34.356 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.28913780600001, 2024-09-13T07:04:34.356 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053725361999999999, 2024-09-13T07:04:34.356 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.356 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.356 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.356 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.357 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.357 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46669 2024-09-13T07:04:34.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.357 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:34.357 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.357 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.357 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.357 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.357 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983886+0000", 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983888+0000", 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4260000000000002e-06 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.358 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983893+0000", 2024-09-13T07:04:34.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.118e-06 2024-09-13T07:04:34.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983898+0000", 2024-09-13T07:04:34.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.792e-06 2024-09-13T07:04:34.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.359 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.359 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.359 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037602+0000", 2024-09-13T07:04:34.359 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053703753999999999 2024-09-13T07:04:34.359 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.360 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.360 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037611+0000", 2024-09-13T07:04:34.360 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6490000000000003e-06 2024-09-13T07:04:34.360 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.360 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.360 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.360 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.360 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.360 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:46670 3.0 3.e7ca5c40 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:34.360 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:34.360 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 156.289121309, 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053751572999999997, 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 46670 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.361 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983902+0000", 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983904+0000", 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.5509999999999998e-06 2024-09-13T07:04:34.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983911+0000", 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1820000000000003e-06 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:57.983916+0000", 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.006e-06 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037645+0000", 2024-09-13T07:04:34.363 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.053729094999999998 2024-09-13T07:04:34.364 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.364 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.364 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.364 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:58.037654+0000", 2024-09-13T07:04:34.364 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2930000000000006e-06 2024-09-13T07:04:34.364 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.364 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.364 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.364 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.364 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.364 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:34.366 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T07:04:34.367 INFO:teuthology.orchestra.run.smithi118.stdout:All operations completed A-OK! 2024-09-13T07:04:34.368 DEBUG:teuthology.parallel:result is None 2024-09-13T07:04:34.368 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-13T07:04:34.380 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-13T07:04:34.396 INFO:tasks.thrashosds:joining thrashosds 2024-09-13T07:04:34.396 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-13T07:04:34.361+0000 7f2334dc2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22303) UID: 0 2024-09-13T07:04:34.519 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:34.519 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:34.519 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:34.519 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:34.529 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T07:04:34.637 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:34.638 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:34.638 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:34.647 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T07:04:34.757 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:34.757 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:34.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:34.757 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:34.757 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.757 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:1528 3.0 e27/22 3:03d51ede:::rbd_data.11943a018f34.0000000000000004:head v 27'13, mlcod=26'11)", 2024-09-13T07:04:34.757 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:34.757 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 188.69900751399999, 2024-09-13T07:04:34.757 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019163765999999999, 2024-09-13T07:04:34.757 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.757 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056591+0000", 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.758 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.056590+0000", 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060936+0000", 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0043455660000000004 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060960+0000", 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.4366e-05 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.759 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.060970+0000", 2024-09-13T07:04:34.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8630000000000003e-06 2024-09-13T07:04:34.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061038+0000", 2024-09-13T07:04:34.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8501e-05 2024-09-13T07:04:34.760 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.760 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.760 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.760 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.061069+0000", 2024-09-13T07:04:34.760 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0518000000000002e-05 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075624+0000", 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014555238 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:01:26.075755+0000", 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000130834 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.761 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133499 3.4 3:21cdf6da:::rbd_data.11947146491c.0000000000000004:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 104.880791157, 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013125286999999999, 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133499 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.762 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874808+0000", 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874806+0000", 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:34.763 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874813+0000", 2024-09-13T07:04:34.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.285e-06 2024-09-13T07:04:34.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.764 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.764 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874822+0000", 2024-09-13T07:04:34.764 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4700000000000008e-06 2024-09-13T07:04:34.764 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.764 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874833+0000", 2024-09-13T07:04:34.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0672e-05 2024-09-13T07:04:34.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887407+0000", 2024-09-13T07:04:34.765 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.012573859999999999 2024-09-13T07:04:34.765 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.765 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.765 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.765 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887501+0000", 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4423e-05 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887933+0000", 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00043174199999999999 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133500 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:34.766 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 104.880741769, 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013655051, 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133500 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.767 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874857+0000", 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874856+0000", 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874860+0000", 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.8999999999999999e-06 2024-09-13T07:04:34.768 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874869+0000", 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.5350000000000004e-06 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874875+0000", 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8359999999999997e-06 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.887958+0000", 2024-09-13T07:04:34.769 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013083322 2024-09-13T07:04:34.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888065+0000", 2024-09-13T07:04:34.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010728799999999999 2024-09-13T07:04:34.770 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.770 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.770 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.770 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888512+0000", 2024-09-13T07:04:34.770 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00044683100000000002 2024-09-13T07:04:34.770 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.770 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.770 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.771 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.771 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133501 3.4 3:2569ba1a:::rbd_data.11947146491c.0000000000000005:25 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:34.771 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:34.771 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 104.880719062, 2024-09-13T07:04:34.771 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014137184000000001, 2024-09-13T07:04:34.771 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.771 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.771 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.771 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.771 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.771 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133501 2024-09-13T07:04:34.771 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.772 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:34.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.772 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.772 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.772 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874880+0000", 2024-09-13T07:04:34.772 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.772 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874879+0000", 2024-09-13T07:04:34.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874882+0000", 2024-09-13T07:04:34.773 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8490000000000002e-06 2024-09-13T07:04:34.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.773 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.773 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.773 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874890+0000", 2024-09-13T07:04:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2800000000000003e-06 2024-09-13T07:04:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874895+0000", 2024-09-13T07:04:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0409999999999997e-06 2024-09-13T07:04:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888540+0000", 2024-09-13T07:04:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013644576 2024-09-13T07:04:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.774 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.888610+0000", 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.0264000000000004e-05 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889017+0000", 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00040668099999999999 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.775 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:133510 3.4 3:21163ef6:::rbd_data.11947146491c.000000000000000a:25 [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 104.880674869, 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014869667, 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 133510 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.776 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:34.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874924+0000", 2024-09-13T07:04:34.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.777 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.777 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.777 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.777 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874923+0000", 2024-09-13T07:04:34.777 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874926+0000", 2024-09-13T07:04:34.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.633e-06 2024-09-13T07:04:34.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.778 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.778 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874932+0000", 2024-09-13T07:04:34.778 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2060000000000004e-06 2024-09-13T07:04:34.778 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.778 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.874938+0000", 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4090000000000002e-06 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889050+0000", 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014111989 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889125+0000", 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4808999999999995e-05 2024-09-13T07:04:34.779 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.889794+0000", 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00066902699999999999 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:133542 3.1a e104/102 3:5e7008a9:::rbd_data.11946d7ee8e1.0000000000000007:head v 104'456, mlcod=103'454)", 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 104.84969578, 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014530895, 2024-09-13T07:04:34.780 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.781 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:34.781 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:34.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.781 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.781 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.781 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:34.781 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.781 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.905903+0000", 2024-09-13T07:04:34.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915878+0000", 2024-09-13T07:04:34.782 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0099750229999999995 2024-09-13T07:04:34.782 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.782 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.782 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.782 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915883+0000", 2024-09-13T07:04:34.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6160000000000001e-06 2024-09-13T07:04:34.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915888+0000", 2024-09-13T07:04:34.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.425e-06 2024-09-13T07:04:34.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.783 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.783 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.783 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915935+0000", 2024-09-13T07:04:34.783 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7627000000000003e-05 2024-09-13T07:04:34.783 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.915954+0000", 2024-09-13T07:04:34.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8439000000000001e-05 2024-09-13T07:04:34.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:34.784 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920372+0000", 2024-09-13T07:04:34.784 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0044186620000000003 2024-09-13T07:04:34.784 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.784 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.784 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.785 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:49.920434+0000", 2024-09-13T07:04:34.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1393e-05 2024-09-13T07:04:34.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.785 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.785 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.785 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.785 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.785 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:137457 3.18 3:1a596dcc:::rbd_data.1194af11889b.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T07:04:34.785 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:34.785 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 90.104073970000002, 2024-09-13T07:04:34.785 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033734403000000003, 2024-09-13T07:04:34.785 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.786 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:34.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.786 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.786 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 137457 2024-09-13T07:04:34.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.786 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.786 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.786 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:34.786 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.786 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.786 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651525+0000", 2024-09-13T07:04:34.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651523+0000", 2024-09-13T07:04:34.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:34.787 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.787 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.787 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.787 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651531+0000", 2024-09-13T07:04:34.787 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.328e-06 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651545+0000", 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4270000000000001e-05 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651555+0000", 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5210000000000008e-06 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.788 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651626+0000", 2024-09-13T07:04:34.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1915999999999994e-05 2024-09-13T07:04:34.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.651761+0000", 2024-09-13T07:04:34.789 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000134838 2024-09-13T07:04:34.789 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.789 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.789 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:34.789 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.658577+0000", 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0068151820000000004 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685181+0000", 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.026604200000000001 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685193+0000", 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2128e-05 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.790 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:34.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685226+0000", 2024-09-13T07:04:34.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3504999999999999e-05 2024-09-13T07:04:34.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.791 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.791 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:04.685259+0000", 2024-09-13T07:04:34.791 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2866e-05 2024-09-13T07:04:34.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.791 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.791 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.791 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.791 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.792 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138565 3.0 3:002d0425:::rbd_data.11949f62878b.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:04:34.792 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:34.792 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 86.090703477999995, 2024-09-13T07:04:34.792 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.033394489999999999, 2024-09-13T07:04:34.792 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.792 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:34.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.792 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.792 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138565 2024-09-13T07:04:34.792 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.792 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:34.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664895+0000", 2024-09-13T07:04:34.793 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.793 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.793 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.793 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.793 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664893+0000", 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664906+0000", 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2996000000000001e-05 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664936+0000", 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9609999999999999e-05 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.794 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.664949+0000", 2024-09-13T07:04:34.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3383e-05 2024-09-13T07:04:34.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665046+0000", 2024-09-13T07:04:34.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6459000000000006e-05 2024-09-13T07:04:34.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.795 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.795 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.795 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.665259+0000", 2024-09-13T07:04:34.795 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000213141 2024-09-13T07:04:34.795 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.672522+0000", 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0072632110000000003 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.697982+0000", 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.025459699999999998 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698018+0000", 2024-09-13T07:04:34.796 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6375999999999999e-05 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698166+0000", 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014818700000000001 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.698290+0000", 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012333999999999999 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.797 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.798 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138587 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T07:04:34.798 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:34.798 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 86.042237811000007, 2024-09-13T07:04:34.798 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019224502000000001, 2024-09-13T07:04:34.798 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.798 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:34.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.798 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.798 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138587 2024-09-13T07:04:34.798 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.798 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.798 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:34.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713361+0000", 2024-09-13T07:04:34.799 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.799 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.799 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.799 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.799 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713360+0000", 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713368+0000", 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0630000000000006e-06 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713383+0000", 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5176e-05 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.800 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713390+0000", 2024-09-13T07:04:34.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8669999999999996e-06 2024-09-13T07:04:34.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713449+0000", 2024-09-13T07:04:34.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8938999999999999e-05 2024-09-13T07:04:34.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.801 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.801 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.801 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.713549+0000", 2024-09-13T07:04:34.801 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010009 2024-09-13T07:04:34.801 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.716087+0000", 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0025383820000000001 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732351+0000", 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016263564000000001 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732450+0000", 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.9529000000000002e-05 2024-09-13T07:04:34.802 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732532+0000", 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1526000000000005e-05 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:08.732585+0000", 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3819000000000001e-05 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.803 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:138786 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 85.608022555999995, 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.018847452000000001, 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 138786 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.804 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:34.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147576+0000", 2024-09-13T07:04:34.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.805 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.805 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.805 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.805 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147575+0000", 2024-09-13T07:04:34.805 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147583+0000", 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4579999999999993e-06 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147601+0000", 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7961e-05 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147610+0000", 2024-09-13T07:04:34.806 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1460000000000002e-06 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147661+0000", 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0447e-05 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.147754+0000", 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.2974999999999997e-05 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:34.807 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.150580+0000", 2024-09-13T07:04:34.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.002826566 2024-09-13T07:04:34.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:34.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166135+0000", 2024-09-13T07:04:34.808 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015554772 2024-09-13T07:04:34.808 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.808 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.808 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:34.808 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166225+0000", 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0258000000000002e-05 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166334+0000", 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00010816500000000001 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.166424+0000", 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0116000000000003e-05 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.809 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:139127 3.18 3:1e978163:::rbd_data.11949f62878b.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 84.980971592000003, 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.019295931999999998, 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 139127 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:34.810 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774627+0000", 2024-09-13T07:04:34.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.811 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.811 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774626+0000", 2024-09-13T07:04:34.811 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:34.811 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.811 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774635+0000", 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.5419999999999994e-06 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774654+0000", 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.8359000000000002e-05 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774663+0000", 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1759999999999999e-06 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.812 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774718+0000", 2024-09-13T07:04:34.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5315000000000001e-05 2024-09-13T07:04:34.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.774805+0000", 2024-09-13T07:04:34.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.7284e-05 2024-09-13T07:04:34.813 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.813 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.813 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 3", 2024-09-13T07:04:34.813 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.779319+0000", 2024-09-13T07:04:34.813 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0045138310000000003 2024-09-13T07:04:34.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:34.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793772+0000", 2024-09-13T07:04:34.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.014452692 2024-09-13T07:04:34.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:34.814 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793821+0000", 2024-09-13T07:04:34.814 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9563e-05 2024-09-13T07:04:34.814 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.814 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.814 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:34.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793878+0000", 2024-09-13T07:04:34.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6953999999999997e-05 2024-09-13T07:04:34.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.815 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.815 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:09.793923+0000", 2024-09-13T07:04:34.815 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4781999999999999e-05 2024-09-13T07:04:34.815 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.815 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.815 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.815 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.815 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.815 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:164691 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.697011717999999, 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.015985102000000001, 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 164691 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.816 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058587+0000", 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.817 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058589+0000", 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2510000000000001e-06 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058594+0000", 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.9089999999999999e-06 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.058598+0000", 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.2069999999999997e-06 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.818 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064175+0000", 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0055769909999999999 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.064210+0000", 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4966999999999999e-05 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074572+0000", 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.010362219000000001 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.819 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165202 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7831552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.694466113999994, 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013116245, 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165202 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.820 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061133+0000", 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061132+0000", 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.821 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061135+0000", 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.3410000000000001e-06 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061142+0000", 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7079999999999997e-06 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061147+0000", 2024-09-13T07:04:34.822 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.8930000000000004e-06 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074180+0000", 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013032346 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074217+0000", 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7552999999999998e-05 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.823 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074249+0000", 2024-09-13T07:04:34.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1711000000000002e-05 2024-09-13T07:04:34.824 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.824 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.824 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.824 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.824 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.824 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165203 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:34.824 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:34.824 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.694447999999994, 2024-09-13T07:04:34.824 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013180211000000001, 2024-09-13T07:04:34.824 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.824 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.824 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.825 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.825 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.825 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165203 2024-09-13T07:04:34.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.825 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:34.825 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.825 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.825 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.825 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.825 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061151+0000", 2024-09-13T07:04:34.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061150+0000", 2024-09-13T07:04:34.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.826 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.826 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.826 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061153+0000", 2024-09-13T07:04:34.826 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.108e-06 2024-09-13T07:04:34.826 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061158+0000", 2024-09-13T07:04:34.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0649999999999998e-06 2024-09-13T07:04:34.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.827 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061162+0000", 2024-09-13T07:04:34.827 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0080000000000004e-06 2024-09-13T07:04:34.827 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.827 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.827 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074262+0000", 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013100557000000001 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074299+0000", 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6981999999999999e-05 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074331+0000", 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1801000000000003e-05 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.828 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.829 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.829 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.829 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.829 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165204 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8093696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:34.829 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:34.829 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.694433289000003, 2024-09-13T07:04:34.829 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0132388, 2024-09-13T07:04:34.829 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.829 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.829 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.829 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165204 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.830 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061165+0000", 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061167+0000", 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1409999999999999e-06 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061175+0000", 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5519999999999999e-06 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.831 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061179+0000", 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.172e-06 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074342+0000", 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013162821999999999 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074373+0000", 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1103999999999999e-05 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.832 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.833 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.833 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074404+0000", 2024-09-13T07:04:34.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1359999999999998e-05 2024-09-13T07:04:34.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.833 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.833 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.833 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.833 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.833 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165205 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:34.833 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:34.833 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.694416329999996, 2024-09-13T07:04:34.833 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013300635, 2024-09-13T07:04:34.833 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.834 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.834 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.834 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165205 2024-09-13T07:04:34.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.834 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.834 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.834 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:34.834 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.834 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.834 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061182+0000", 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061184+0000", 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.0779999999999998e-06 2024-09-13T07:04:34.835 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061192+0000", 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.7209999999999994e-06 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061196+0000", 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4469999999999999e-06 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074417+0000", 2024-09-13T07:04:34.836 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013220750999999999 2024-09-13T07:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074452+0000", 2024-09-13T07:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4814999999999998e-05 2024-09-13T07:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074483+0000", 2024-09-13T07:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1118000000000003e-05 2024-09-13T07:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:165206 3.0 3:07d254f7:::rbd_data.11948a4f2bb0.000000000000000b:33 [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.694398208999999, 2024-09-13T07:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013364454, 2024-09-13T07:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 165206 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061201+0000", 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.839 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061200+0000", 2024-09-13T07:04:34.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:34.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061203+0000", 2024-09-13T07:04:34.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.559e-06 2024-09-13T07:04:34.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.840 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.840 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.840 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061210+0000", 2024-09-13T07:04:34.840 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.52e-06 2024-09-13T07:04:34.840 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.061214+0000", 2024-09-13T07:04:34.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.0999999999999997e-06 2024-09-13T07:04:34.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.841 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.841 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074494+0000", 2024-09-13T07:04:34.841 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013279094 2024-09-13T07:04:34.841 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.841 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074530+0000", 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6538999999999997e-05 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.074565+0000", 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.4972999999999999e-05 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.842 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169377 3.0 3:07b65a69:::rbd_data.1194e181a50b.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-13T07:04:34.843 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:34.843 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 62.398731814000001, 2024-09-13T07:04:34.843 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.032744151999999999, 2024-09-13T07:04:34.843 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.843 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:34.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.843 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.843 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169377 2024-09-13T07:04:34.843 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.843 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.843 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.843 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356867+0000", 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356864+0000", 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.844 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356880+0000", 2024-09-13T07:04:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6022999999999999e-05 2024-09-13T07:04:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356916+0000", 2024-09-13T07:04:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5921999999999999e-05 2024-09-13T07:04:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.356931+0000", 2024-09-13T07:04:34.845 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4311000000000001e-05 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357011+0000", 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.9992000000000003e-05 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.357143+0000", 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000131822 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:34.846 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.361578+0000", 2024-09-13T07:04:34.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00443525 2024-09-13T07:04:34.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:34.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389466+0000", 2024-09-13T07:04:34.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.027888342999999999 2024-09-13T07:04:34.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.847 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:34.847 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389529+0000", 2024-09-13T07:04:34.847 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.2460999999999997e-05 2024-09-13T07:04:34.847 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.847 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389574+0000", 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5405999999999999e-05 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:32.389611+0000", 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7122000000000001e-05 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.848 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:169731 3.0 3:031f8e93:::rbd_data.1194e181a50b.0000000000000006:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-13T07:04:34.849 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:34.849 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 61.474306016, 2024-09-13T07:04:34.849 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.021389756999999999, 2024-09-13T07:04:34.849 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.849 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:34.849 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.849 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.849 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.849 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 169731 2024-09-13T07:04:34.849 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.849 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.849 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.849 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281293+0000", 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281291+0000", 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.850 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281302+0000", 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0651e-05 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281324+0000", 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.1713999999999999e-05 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281340+0000", 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6521999999999999e-05 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.851 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281450+0000", 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000110086 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.281593+0000", 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00014306 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 6", 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.286356+0000", 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0047626320000000002 2024-09-13T07:04:34.852 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:34.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302404+0000", 2024-09-13T07:04:34.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.016048605000000001 2024-09-13T07:04:34.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:34.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302476+0000", 2024-09-13T07:04:34.853 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.1483000000000007e-05 2024-09-13T07:04:34.853 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.853 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.853 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:34.853 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302578+0000", 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000101883 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:33.302683+0000", 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000104681 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:178874 3.16 3:68eb4b70:::journal.1194897e7c0a:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e189)", 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:24.159236+0000", 2024-09-13T07:04:34.854 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 10.596363233, 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.95629728899999999, 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "reached pg", 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 178874 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159236+0000", 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.855 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159236+0000", 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159234+0000", 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159241+0000", 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.277e-06 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.856 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159294+0000", 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.2166000000000002e-05 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159302+0000", 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159393+0000", 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.1423999999999997e-05 2024-09-13T07:04:34.857 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:24.159408+0000", 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.4605e-05 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:25.115368+0000", 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.955959534 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:25.115533+0000", 2024-09-13T07:04:34.858 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016526900000000001 2024-09-13T07:04:34.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.859 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.859 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:34.859 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:34.859 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:34.860 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T07:04:34.971 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:34.971 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:34.971 INFO:teuthology.orchestra.run.smithi118.stdout: "num_ops": 0 2024-09-13T07:04:34.971 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:34.981 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T07:04:35.088 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [], 2024-09-13T07:04:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: "complaint_time": 30, 2024-09-13T07:04:35.088 INFO:teuthology.orchestra.run.smithi118.stdout: "num_blocked_ops": 0 2024-09-13T07:04:35.089 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:35.098 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T07:04:35.191 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T07:04:35.218 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:04:35.218 INFO:teuthology.orchestra.run.smithi118.stdout: "size": 20, 2024-09-13T07:04:35.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 600, 2024-09-13T07:04:35.219 INFO:teuthology.orchestra.run.smithi118.stdout: "ops": [ 2024-09-13T07:04:35.219 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.219 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:86838 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8536064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.219 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:35.219 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.096101887, 2024-09-13T07:04:35.219 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042735624999999999, 2024-09-13T07:04:35.219 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.219 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.219 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.219 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.219 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.219 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 86838 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.220 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121278+0000", 2024-09-13T07:04:35.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121281+0000", 2024-09-13T07:04:35.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.959e-06 2024-09-13T07:04:35.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.221 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.221 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.221 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121287+0000", 2024-09-13T07:04:35.221 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8379999999999998e-06 2024-09-13T07:04:35.221 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.121293+0000", 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.5189999999999998e-06 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163832+0000", 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042538869 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.163916+0000", 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3561999999999997e-05 2024-09-13T07:04:35.222 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.164014+0000", 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8448000000000003e-05 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87038 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7520256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.089393215, 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042851268999999997, 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.223 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.224 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.224 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.224 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.224 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87038 2024-09-13T07:04:35.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.224 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.224 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:35.224 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.224 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.224 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:35.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127987+0000", 2024-09-13T07:04:35.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:35.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.225 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.225 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.225 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127988+0000", 2024-09-13T07:04:35.225 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6419999999999999e-06 2024-09-13T07:04:35.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127991+0000", 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.88e-06 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127994+0000", 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.892e-06 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170631+0000", 2024-09-13T07:04:35.226 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042636595999999999 2024-09-13T07:04:35.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170713+0000", 2024-09-13T07:04:35.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2515999999999994e-05 2024-09-13T07:04:35.227 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.227 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.227 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.227 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170838+0000", 2024-09-13T07:04:35.227 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00012497499999999999 2024-09-13T07:04:35.227 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.227 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.227 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.228 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.228 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87039 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7716864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.228 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:35.228 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.089380772, 2024-09-13T07:04:35.228 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042979455999999999, 2024-09-13T07:04:35.228 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.228 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.228 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.228 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.228 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.228 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87039 2024-09-13T07:04:35.228 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.229 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:35.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.229 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.229 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.229 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:35.229 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.229 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.127999+0000", 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128003+0000", 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.9940000000000002e-06 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128007+0000", 2024-09-13T07:04:35.230 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.2799999999999999e-06 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128009+0000", 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.824e-06 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170862+0000", 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042852488000000001 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.231 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170915+0000", 2024-09-13T07:04:35.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3186000000000002e-05 2024-09-13T07:04:35.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.232 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.232 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.170979+0000", 2024-09-13T07:04:35.232 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3909000000000001e-05 2024-09-13T07:04:35.232 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.232 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.232 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.232 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.232 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.232 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87040 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 7880704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.232 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.089363174, 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043183628000000002, 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87040 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:35.233 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:35.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.234 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128017+0000", 2024-09-13T07:04:35.234 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.234 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.234 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.234 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128019+0000", 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7069999999999999e-06 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128022+0000", 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0920000000000002e-06 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128025+0000", 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.942e-06 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.235 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171033+0000", 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043008805999999997 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171106+0000", 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2675999999999993e-05 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171201+0000", 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.4649999999999997e-05 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.236 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.237 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.237 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.237 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.237 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87041 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8077312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.237 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:35.237 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.08935247100001, 2024-09-13T07:04:35.237 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344639999999997, 2024-09-13T07:04:35.237 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.237 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.237 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.237 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.237 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.237 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87041 2024-09-13T07:04:35.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.238 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:35.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.238 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.238 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:35.238 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.238 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.238 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128028+0000", 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128029+0000", 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.607e-06 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128032+0000", 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9239999999999999e-06 2024-09-13T07:04:35.239 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128035+0000", 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0319999999999999e-06 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171227+0000", 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043192377999999997 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171284+0000", 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6410999999999999e-05 2024-09-13T07:04:35.240 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.241 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.241 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171372+0000", 2024-09-13T07:04:35.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8549999999999998e-05 2024-09-13T07:04:35.241 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.241 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.241 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.241 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.241 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.241 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87042 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8437760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.241 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:35.241 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.08930853800001, 2024-09-13T07:04:35.241 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511702999999999, 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87042 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.242 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128072+0000", 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128071+0000", 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128075+0000", 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7000000000000002e-06 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.243 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128083+0000", 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2400000000000007e-06 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128088+0000", 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4920000000000001e-06 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171433+0000", 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043344448000000001 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.244 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171502+0000", 2024-09-13T07:04:35.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8948999999999993e-05 2024-09-13T07:04:35.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.245 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.245 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171583+0000", 2024-09-13T07:04:35.245 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.1570000000000007e-05 2024-09-13T07:04:35.245 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.245 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.245 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.245 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.245 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87043 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8634368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.08928875199999, 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043641314, 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87043 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.246 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128091+0000", 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.247 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128093+0000", 2024-09-13T07:04:35.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.2529999999999999e-06 2024-09-13T07:04:35.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128101+0000", 2024-09-13T07:04:35.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2429999999999999e-06 2024-09-13T07:04:35.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.248 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.248 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128105+0000", 2024-09-13T07:04:35.248 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.1450000000000001e-06 2024-09-13T07:04:35.248 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.248 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171616+0000", 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043511439999999998 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171660+0000", 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.3476000000000002e-05 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171733+0000", 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3140999999999996e-05 2024-09-13T07:04:35.249 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87044 3.1e 3:7d024042:::rbd_data.11948b05a38f.0000000000000000:1a [read 8798208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.089273002, 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043885256999999997, 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87044 2024-09-13T07:04:35.250 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.251 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:35.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.251 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:35.251 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.251 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.251 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.251 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128107+0000", 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128109+0000", 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.7320000000000001e-06 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128112+0000", 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8700000000000001e-06 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.252 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128115+0000", 2024-09-13T07:04:35.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.428e-06 2024-09-13T07:04:35.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171821+0000", 2024-09-13T07:04:35.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043706217999999998 2024-09-13T07:04:35.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.253 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.253 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.253 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171902+0000", 2024-09-13T07:04:35.253 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.0496999999999999e-05 2024-09-13T07:04:35.253 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.171993+0000", 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0731999999999998e-05 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87084 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.08926174999999, 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044006238000000003, 2024-09-13T07:04:35.254 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87084 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.255 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128119+0000", 2024-09-13T07:04:35.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128118+0000", 2024-09-13T07:04:35.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:35.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.256 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.256 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.256 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128120+0000", 2024-09-13T07:04:35.256 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.412e-06 2024-09-13T07:04:35.256 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128123+0000", 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.411e-06 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128127+0000", 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.6040000000000001e-06 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172015+0000", 2024-09-13T07:04:35.257 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.043888688000000002 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172063+0000", 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7611999999999997e-05 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172125+0000", 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1718999999999996e-05 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.258 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87085 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.08925115100001, 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044138535, 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87085 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.259 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128129+0000", 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.260 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128130+0000", 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6050000000000001e-06 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128133+0000", 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.897e-06 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128136+0000", 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8040000000000002e-06 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.261 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172155+0000", 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044019007999999998 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172209+0000", 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.3634999999999997e-05 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172268+0000", 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8794e-05 2024-09-13T07:04:35.262 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87086 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6373376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.08924165799999, 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044274385999999999, 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.263 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87086 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128139+0000", 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.264 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128138+0000", 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128140+0000", 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6559999999999999e-06 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128143+0000", 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.869e-06 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.265 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128146+0000", 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8420000000000001e-06 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172296+0000", 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044150611999999999 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172356+0000", 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.9159000000000001e-05 2024-09-13T07:04:35.266 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172413+0000", 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.7475000000000001e-05 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87087 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6733824~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.089232114, 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044393126999999998, 2024-09-13T07:04:35.267 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87087 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.268 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128148+0000", 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128150+0000", 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.677e-06 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.269 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128153+0000", 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.0240000000000002e-06 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128156+0000", 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.5530000000000001e-06 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172439+0000", 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044282491 2024-09-13T07:04:35.270 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172487+0000", 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.7932000000000001e-05 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172541+0000", 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4659e-05 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.271 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87088 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 6930432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.089199511, 2024-09-13T07:04:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044521922999999998, 2024-09-13T07:04:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87088 2024-09-13T07:04:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.272 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128181+0000", 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128180+0000", 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.273 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128182+0000", 2024-09-13T07:04:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.9080000000000002e-06 2024-09-13T07:04:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128185+0000", 2024-09-13T07:04:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.1580000000000001e-06 2024-09-13T07:04:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.274 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128190+0000", 2024-09-13T07:04:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.5240000000000002e-06 2024-09-13T07:04:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172567+0000", 2024-09-13T07:04:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044377293999999998 2024-09-13T07:04:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.275 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172628+0000", 2024-09-13T07:04:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.1308000000000003e-05 2024-09-13T07:04:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172703+0000", 2024-09-13T07:04:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4204999999999999e-05 2024-09-13T07:04:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87089 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 7094272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.276 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.089186488, 2024-09-13T07:04:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044714561999999999, 2024-09-13T07:04:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87089 2024-09-13T07:04:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.277 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128194+0000", 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967296 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.278 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128195+0000", 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.5850000000000001e-06 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128198+0000", 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.8279999999999999e-06 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.128201+0000", 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.841e-06 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.279 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172729+0000", 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.044528098000000002 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172793+0000", 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.3958e-05 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.172908+0000", 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000115456 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.280 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87095 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8568832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.08581522599999, 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042719343, 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87095 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.281 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131565+0000", 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131564+0000", 2024-09-13T07:04:35.282 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131568+0000", 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.3330000000000001e-06 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131575+0000", 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.2880000000000004e-06 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.283 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131580+0000", 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.4580000000000001e-06 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174120+0000", 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042539529999999999 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174194+0000", 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4185999999999997e-05 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.284 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174284+0000", 2024-09-13T07:04:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.0270000000000001e-05 2024-09-13T07:04:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:87096 3.1e 3:7f3ac574:::rbd_data.11948b05a38f.0000000000000004:1a [read 8765440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-13T07:04:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 133.08579502500001, 2024-09-13T07:04:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042909201000000001, 2024-09-13T07:04:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "started", 2024-09-13T07:04:35.285 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 87096 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:35.286 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131585+0000", 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131588+0000", 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.937e-06 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.287 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131593+0000", 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0760000000000002e-06 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.131599+0000", 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.6169999999999998e-06 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174313+0000", 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.042714658000000003 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.288 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174396+0000", 2024-09-13T07:04:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.2997999999999994e-05 2024-09-13T07:04:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:02:22.174494+0000", 2024-09-13T07:04:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.8267e-05 2024-09-13T07:04:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.289 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_repop(client.4500.0:166663 3.b e135/117 3:d7869255:::rbd_data.11942e4f5fc7.0000000000000001:head v 135'409, mlcod=132'407)", 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 65.803206059999994, 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.045108144000000003, 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.290 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414174+0000", 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.414172+0000", 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427200+0000", 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.013028098 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.291 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427229+0000", 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 2.9178000000000001e-05 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427239+0000", 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 9.6870000000000006e-06 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427393+0000", 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00015382899999999999 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.292 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.427451+0000", 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.8301000000000003e-05 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459232+0000", 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.031780695999999997 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:03:29.459282+0000", 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.0701000000000001e-05 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.293 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:176785 3.7 3:e640995f:::rbd_data.1194987e4741.0000000000000011:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:09.441952+0000", 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.775428014999999, 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86669136400000002, 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 176785 2024-09-13T07:04:35.294 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441952+0000", 2024-09-13T07:04:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441952+0000", 2024-09-13T07:04:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.295 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441951+0000", 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441958+0000", 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3089999999999998e-06 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441970+0000", 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2404e-05 2024-09-13T07:04:35.296 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.441977+0000", 2024-09-13T07:04:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.8979999999999999e-06 2024-09-13T07:04:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442051+0000", 2024-09-13T07:04:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.3337000000000006e-05 2024-09-13T07:04:35.297 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442063+0000", 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.2931e-05 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305282+0000", 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86321888999999996 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305447+0000", 2024-09-13T07:04:35.298 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00016446500000000001 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305579+0000", 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000132452 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.307846+0000", 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0022664790000000001 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:35.299 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308562+0000", 2024-09-13T07:04:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00071642899999999998 2024-09-13T07:04:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308606+0000", 2024-09-13T07:04:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4.4073999999999998e-05 2024-09-13T07:04:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.300 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308644+0000", 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 3.7348999999999998e-05 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:176787 3.15 3:af329162:::rbd_data.1194987e4741.0000000000000013:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:09.442992+0000", 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.774388271999999, 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86944086799999998, 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:35.301 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 176787 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442992+0000", 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442992+0000", 2024-09-13T07:04:35.302 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442990+0000", 2024-09-13T07:04:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.442999+0000", 2024-09-13T07:04:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.8019999999999998e-06 2024-09-13T07:04:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.303 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443015+0000", 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6002999999999999e-05 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443024+0000", 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 8.3610000000000001e-06 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443092+0000", 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.7967999999999997e-05 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.304 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.443105+0000", 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.3969e-05 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.303912+0000", 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86080616200000004 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.304190+0000", 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00027807599999999999 2024-09-13T07:04:35.305 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.304362+0000", 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017219199999999999 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.305588+0000", 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.001226499 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312299+0000", 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0067103789999999998 2024-09-13T07:04:35.306 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312377+0000", 2024-09-13T07:04:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.8291e-05 2024-09-13T07:04:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312433+0000", 2024-09-13T07:04:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.5755999999999999e-05 2024-09-13T07:04:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.307 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "description": "osd_op(client.4500.0:176791 3.7 3:e2fcfe96:::rbd_data.1194987e4741.0000000000000018:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "initiated_at": "2024-09-13T07:04:09.444146+0000", 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "age": 25.773233812000001, 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86828808400000002, 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "type_data": { 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "client_info": { 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "client": "client.4500", 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "tid": 176791 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: "events": [ 2024-09-13T07:04:35.308 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "initiated", 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444146+0000", 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "throttled", 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444146+0000", 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "header_read", 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444145+0000", 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 4294967295.999999 2024-09-13T07:04:35.309 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "all_read", 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444151+0000", 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.0050000000000001e-06 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "dispatched", 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444168+0000", 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.6399e-05 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444175+0000", 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.5610000000000001e-06 2024-09-13T07:04:35.310 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444240+0000", 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 6.4893000000000004e-05 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:09.444250+0000", 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 1.0029e-05 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "reached_pg", 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.306520+0000", 2024-09-13T07:04:35.311 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.86226998899999996 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "started", 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.306696+0000", 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.00017564800000000001 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "waiting for subops from 7", 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.306820+0000", 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.000123726 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "op_commit", 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.308963+0000", 2024-09-13T07:04:35.312 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0021437990000000001 2024-09-13T07:04:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312309+0000", 2024-09-13T07:04:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 0.0033458590000000001 2024-09-13T07:04:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "commit_sent", 2024-09-13T07:04:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312384+0000", 2024-09-13T07:04:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 7.4184000000000007e-05 2024-09-13T07:04:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-09-13T07:04:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-09-13T07:04:35.313 INFO:teuthology.orchestra.run.smithi118.stdout: "event": "done", 2024-09-13T07:04:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: "time": "2024-09-13T07:04:10.312435+0000", 2024-09-13T07:04:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: "duration": 5.1019000000000001e-05 2024-09-13T07:04:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-09-13T07:04:35.314 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-09-13T07:04:35.314 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:04:35.315 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T07:04:35.430 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:35.430 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:35.430 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:35.430 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:35.440 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T07:04:35.543 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:35.543 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:35.543 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:35.543 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:35.543 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:35.552 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T07:04:35.674 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:04:35.669+0000 7f60a2f3d880 -1 osd.5 147 log_to_monitors true 2024-09-13T07:04:35.678 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:35.678 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:35.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:35.679 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:35.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.679 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129085 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.679 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:35.679 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.075319373, 2024-09-13T07:04:35.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338848399999994, 2024-09-13T07:04:35.679 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.679 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.679 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.679 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.679 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.679 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129085 2024-09-13T07:04:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601347+0000", 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601351+0000", 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3980000000000001e-06 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601361+0000", 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251000000000001e-05 2024-09-13T07:04:35.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601366+0000", 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1440000000000002e-06 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601497+0000", 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131037 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601510+0000", 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2948e-05 2024-09-13T07:04:35.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514727+0000", 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91321671100000001 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514736+0000", 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6489999999999997e-06 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129086 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.683 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:35.684 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.07528827199999, 2024-09-13T07:04:35.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338608099999996, 2024-09-13T07:04:35.684 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.684 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.684 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129086 2024-09-13T07:04:35.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.684 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601378+0000", 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601382+0000", 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8199999999999998e-06 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601390+0000", 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4279999999999996e-06 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601396+0000", 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0240000000000003e-06 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601555+0000", 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015909100000000001 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601571+0000", 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5642e-05 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514758+0000", 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91318671699999998 2024-09-13T07:04:35.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514764+0000", 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129087 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.075265958, 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339235100000005, 2024-09-13T07:04:35.688 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129087 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601401+0000", 2024-09-13T07:04:35.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601400+0000", 2024-09-13T07:04:35.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601404+0000", 2024-09-13T07:04:35.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.501e-06 2024-09-13T07:04:35.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601412+0000", 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7980000000000006e-06 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601418+0000", 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1009999999999997e-06 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601607+0000", 2024-09-13T07:04:35.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018916199999999999 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601620+0000", 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2755000000000001e-05 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514786+0000", 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91316639600000005 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514793+0000", 2024-09-13T07:04:35.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0899999999999999e-06 2024-09-13T07:04:35.693 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.693 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.693 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.693 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129088 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.693 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:35.693 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.07524292700001, 2024-09-13T07:04:35.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339740000000003, 2024-09-13T07:04:35.693 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.693 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129088 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601424+0000", 2024-09-13T07:04:35.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601423+0000", 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601427+0000", 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3400000000000002e-06 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601435+0000", 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1210000000000007e-06 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601442+0000", 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8190000000000003e-06 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024110900000000001 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601705+0000", 2024-09-13T07:04:35.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2648999999999999e-05 2024-09-13T07:04:35.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514814+0000", 2024-09-13T07:04:35.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91310874799999997 2024-09-13T07:04:35.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514821+0000", 2024-09-13T07:04:35.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1010000000000003e-06 2024-09-13T07:04:35.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.698 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.698 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.698 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129089 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.698 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:35.698 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.075218377, 2024-09-13T07:04:35.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91340203799999997, 2024-09-13T07:04:35.698 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.698 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.699 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129089 2024-09-13T07:04:35.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.699 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:35.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601448+0000", 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601451+0000", 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601459+0000", 2024-09-13T07:04:35.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9109999999999993e-06 2024-09-13T07:04:35.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601466+0000", 2024-09-13T07:04:35.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6170000000000003e-06 2024-09-13T07:04:35.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601766+0000", 2024-09-13T07:04:35.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030008900000000002 2024-09-13T07:04:35.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601778+0000", 2024-09-13T07:04:35.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2418999999999999e-05 2024-09-13T07:04:35.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514842+0000", 2024-09-13T07:04:35.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91306407599999995 2024-09-13T07:04:35.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514850+0000", 2024-09-13T07:04:35.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8630000000000008e-06 2024-09-13T07:04:35.703 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.703 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.703 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.703 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129090 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.703 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:35.703 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.075182668, 2024-09-13T07:04:35.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339563199999996, 2024-09-13T07:04:35.703 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129090 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601484+0000", 2024-09-13T07:04:35.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601483+0000", 2024-09-13T07:04:35.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601487+0000", 2024-09-13T07:04:35.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8240000000000001e-06 2024-09-13T07:04:35.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601495+0000", 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601502+0000", 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7689999999999999e-06 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601825+0000", 2024-09-13T07:04:35.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032287100000000002 2024-09-13T07:04:35.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601837+0000", 2024-09-13T07:04:35.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.151e-05 2024-09-13T07:04:35.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514871+0000", 2024-09-13T07:04:35.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91303454900000003 2024-09-13T07:04:35.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514880+0000", 2024-09-13T07:04:35.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5159999999999994e-06 2024-09-13T07:04:35.708 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.708 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.708 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.708 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129091 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.708 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:35.708 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.075157989, 2024-09-13T07:04:35.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339837599999996, 2024-09-13T07:04:35.708 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.708 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.709 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129091 2024-09-13T07:04:35.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.709 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:35.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601509+0000", 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601508+0000", 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601512+0000", 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5669999999999999e-06 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601519+0000", 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4610000000000002e-06 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601526+0000", 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5259999999999999e-06 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601874+0000", 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000348134 2024-09-13T07:04:35.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601884+0000", 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169e-05 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514900+0000", 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91301573700000005 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514907+0000", 2024-09-13T07:04:35.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2849999999999997e-06 2024-09-13T07:04:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129092 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.075124715, 2024-09-13T07:04:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339340899999999, 2024-09-13T07:04:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129092 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601542+0000", 2024-09-13T07:04:35.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601541+0000", 2024-09-13T07:04:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601545+0000", 2024-09-13T07:04:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9210000000000002e-06 2024-09-13T07:04:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601553+0000", 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7640000000000005e-06 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601562+0000", 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9439999999999994e-06 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601944+0000", 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000382419 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601955+0000", 2024-09-13T07:04:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0814e-05 2024-09-13T07:04:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514928+0000", 2024-09-13T07:04:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91297286499999997 2024-09-13T07:04:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514935+0000", 2024-09-13T07:04:35.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129093 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.075097927, 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91345948600000004, 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129093 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601569+0000", 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601568+0000", 2024-09-13T07:04:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601572+0000", 2024-09-13T07:04:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9450000000000003e-06 2024-09-13T07:04:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601580+0000", 2024-09-13T07:04:35.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4870000000000002e-06 2024-09-13T07:04:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601587+0000", 2024-09-13T07:04:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000001e-06 2024-09-13T07:04:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602005+0000", 2024-09-13T07:04:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041773199999999998 2024-09-13T07:04:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602016+0000", 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.514956+0000", 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91294030599999998 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515028+0000", 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2247000000000001e-05 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.722 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129094 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.075073373, 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346516099999997, 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129094 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.723 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601593+0000", 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601596+0000", 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4800000000000001e-06 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601608+0000", 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601614+0000", 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9859999999999997e-06 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602053+0000", 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000438236 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602063+0000", 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0421999999999999e-05 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515051+0000", 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91298795399999999 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515058+0000", 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.34e-06 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129095 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.727 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.075043604, 2024-09-13T07:04:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91346281500000004, 2024-09-13T07:04:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129095 2024-09-13T07:04:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601623+0000", 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601627+0000", 2024-09-13T07:04:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2429999999999999e-06 2024-09-13T07:04:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601635+0000", 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0539999999999995e-06 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601642+0000", 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.81e-06 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602104+0000", 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046188000000000002 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602114+0000", 2024-09-13T07:04:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0419e-05 2024-09-13T07:04:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515079+0000", 2024-09-13T07:04:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91296489999999997 2024-09-13T07:04:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515086+0000", 2024-09-13T07:04:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9700000000000002e-06 2024-09-13T07:04:35.732 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129096 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.074983761, 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91343129300000003, 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.733 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129096 2024-09-13T07:04:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601683+0000", 2024-09-13T07:04:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601682+0000", 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601687+0000", 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000001e-06 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601708+0000", 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0645e-05 2024-09-13T07:04:35.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601714+0000", 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5979999999999996e-06 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602159+0000", 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044567100000000002 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602170+0000", 2024-09-13T07:04:35.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0263e-05 2024-09-13T07:04:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515107+0000", 2024-09-13T07:04:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.912937373 2024-09-13T07:04:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515114+0000", 2024-09-13T07:04:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1450000000000002e-06 2024-09-13T07:04:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.737 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129097 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.074916911, 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91339171200000002, 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129097 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.738 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601750+0000", 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601749+0000", 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601753+0000", 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0289999999999999e-06 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601768+0000", 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5182e-05 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601776+0000", 2024-09-13T07:04:35.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3660000000000003e-06 2024-09-13T07:04:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602217+0000", 2024-09-13T07:04:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044122900000000001 2024-09-13T07:04:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602258+0000", 2024-09-13T07:04:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1516e-05 2024-09-13T07:04:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515134+0000", 2024-09-13T07:04:35.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91287596000000004 2024-09-13T07:04:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515141+0000", 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310000000000001e-06 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129098 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.07488440900001, 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91338666800000001, 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129098 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601782+0000", 2024-09-13T07:04:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601785+0000", 2024-09-13T07:04:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6100000000000002e-06 2024-09-13T07:04:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601796+0000", 2024-09-13T07:04:35.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0824e-05 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601803+0000", 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0040000000000002e-06 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602297+0000", 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049421100000000004 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602308+0000", 2024-09-13T07:04:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0771000000000001e-05 2024-09-13T07:04:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515162+0000", 2024-09-13T07:04:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91285376900000004 2024-09-13T07:04:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515169+0000", 2024-09-13T07:04:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0049999999999998e-06 2024-09-13T07:04:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.749 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129099 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.07484978799999, 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91337997000000004, 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129099 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.750 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601817+0000", 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601816+0000", 2024-09-13T07:04:35.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601820+0000", 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.703e-06 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601828+0000", 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2099999999999993e-06 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601838+0000", 2024-09-13T07:04:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3579999999999992e-06 2024-09-13T07:04:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602348+0000", 2024-09-13T07:04:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051023799999999999 2024-09-13T07:04:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602358+0000", 2024-09-13T07:04:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173000000000001e-05 2024-09-13T07:04:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515190+0000", 2024-09-13T07:04:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91283160699999999 2024-09-13T07:04:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515197+0000", 2024-09-13T07:04:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1799999999999999e-06 2024-09-13T07:04:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129100 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.07479955700001, 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91335717699999996, 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129100 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601867+0000", 2024-09-13T07:04:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601870+0000", 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9330000000000003e-06 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601879+0000", 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7900000000000005e-06 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601886+0000", 2024-09-13T07:04:35.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6420000000000001e-06 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602410+0000", 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052381699999999999 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602417+0000", 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515218+0000", 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91280015599999997 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515224+0000", 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129101 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:35.759 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.074751371, 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333718699999999, 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129101 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601915+0000", 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601918+0000", 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6710000000000001e-06 2024-09-13T07:04:35.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601927+0000", 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0019999999999995e-06 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601935+0000", 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0050000000000004e-06 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602445+0000", 2024-09-13T07:04:35.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050979200000000001 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602453+0000", 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9449999999999993e-06 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515245+0000", 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91279215700000005 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515252+0000", 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1470000000000004e-06 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129102 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.074724733, 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91333816499999998, 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.764 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129102 2024-09-13T07:04:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601942+0000", 2024-09-13T07:04:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601941+0000", 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601945+0000", 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7440000000000001e-06 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601954+0000", 2024-09-13T07:04:35.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4139999999999994e-06 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601960+0000", 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602480+0000", 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052026199999999996 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602488+0000", 2024-09-13T07:04:35.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7840000000000004e-06 2024-09-13T07:04:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515273+0000", 2024-09-13T07:04:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91278496399999998 2024-09-13T07:04:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515280+0000", 2024-09-13T07:04:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-13T07:04:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.768 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129103 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.074677898, 2024-09-13T07:04:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91331889099999997, 2024-09-13T07:04:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129103 2024-09-13T07:04:35.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601989+0000", 2024-09-13T07:04:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601988+0000", 2024-09-13T07:04:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.601992+0000", 2024-09-13T07:04:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9419999999999997e-06 2024-09-13T07:04:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602007+0000", 2024-09-13T07:04:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5038000000000001e-05 2024-09-13T07:04:35.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602015+0000", 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7649999999999993e-06 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602525+0000", 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050968200000000002 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.602534+0000", 2024-09-13T07:04:35.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2269999999999999e-06 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515300+0000", 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91276656899999997 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.515308+0000", 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:129160 3.9 3.175975a9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 109.07132348099999, 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91376461899999994, 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 129160 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605343+0000", 2024-09-13T07:04:35.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605346+0000", 2024-09-13T07:04:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3620000000000002e-06 2024-09-13T07:04:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605353+0000", 2024-09-13T07:04:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9609999999999999e-06 2024-09-13T07:04:35.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605358+0000", 2024-09-13T07:04:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4709999999999998e-06 2024-09-13T07:04:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605614+0000", 2024-09-13T07:04:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025561200000000001 2024-09-13T07:04:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:46.605621+0000", 2024-09-13T07:04:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1380000000000001e-06 2024-09-13T07:04:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.517059+0000", 2024-09-13T07:04:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.91143757999999997 2024-09-13T07:04:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:47.519108+0000", 2024-09-13T07:04:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020489789999999998 2024-09-13T07:04:35.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:35.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:35.779 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:35.780 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T07:04:35.892 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:35.893 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:35.893 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:35.893 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:35.902 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T07:04:36.009 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:36.009 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:36.009 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:36.009 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:36.009 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:36.019 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T07:04:36.128 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:1533 3.a 3:508e9972:::rbd_data.11943a018f34.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T07:04:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 190.06766610899999, 2024-09-13T07:04:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046956596000000003, 2024-09-13T07:04:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1533 2024-09-13T07:04:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058921+0000", 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.058916+0000", 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999948 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065680+0000", 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067640119999999998 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065708+0000", 2024-09-13T07:04:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8677e-05 2024-09-13T07:04:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065724+0000", 2024-09-13T07:04:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5945000000000001e-05 2024-09-13T07:04:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.065806+0000", 2024-09-13T07:04:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1280999999999996e-05 2024-09-13T07:04:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066022+0000", 2024-09-13T07:04:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021606300000000001 2024-09-13T07:04:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.066607+0000", 2024-09-13T07:04:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058525199999999997 2024-09-13T07:04:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.100684+0000", 2024-09-13T07:04:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034077463000000002 2024-09-13T07:04:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105729+0000", 2024-09-13T07:04:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050449659999999997 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105802+0000", 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2506e-05 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105877+0000", 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5480000000000002e-05 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93980 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 2899968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.50988223100001, 2024-09-13T07:04:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042387637999999998, 2024-09-13T07:04:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93980 2024-09-13T07:04:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616705+0000", 2024-09-13T07:04:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616704+0000", 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616709+0000", 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5390000000000001e-06 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616717+0000", 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8540000000000003e-06 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616724+0000", 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.658959+0000", 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042235083 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659028+0000", 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8848000000000002e-05 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659092+0000", 2024-09-13T07:04:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4663000000000001e-05 2024-09-13T07:04:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93981 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.509859264, 2024-09-13T07:04:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042521454, 2024-09-13T07:04:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93981 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616728+0000", 2024-09-13T07:04:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616727+0000", 2024-09-13T07:04:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616730+0000", 2024-09-13T07:04:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6900000000000001e-06 2024-09-13T07:04:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616734+0000", 2024-09-13T07:04:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5879999999999999e-06 2024-09-13T07:04:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616739+0000", 2024-09-13T07:04:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0829999999999997e-06 2024-09-13T07:04:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659117+0000", 2024-09-13T07:04:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042378374000000003 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659184+0000", 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6950000000000001e-05 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659249+0000", 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5110999999999994e-05 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93982 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.50984208599999, 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04266992, 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93982 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616745+0000", 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616744+0000", 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616747+0000", 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2749999999999998e-06 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616751+0000", 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.549e-06 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616755+0000", 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-13T07:04:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659275+0000", 2024-09-13T07:04:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042519455999999997 2024-09-13T07:04:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659347+0000", 2024-09-13T07:04:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2024e-05 2024-09-13T07:04:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659415+0000", 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7858e-05 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93983 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3194880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.50982822099999, 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042821887000000003, 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93983 2024-09-13T07:04:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616759+0000", 2024-09-13T07:04:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616758+0000", 2024-09-13T07:04:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616761+0000", 2024-09-13T07:04:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.289e-06 2024-09-13T07:04:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616765+0000", 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2520000000000001e-06 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616769+0000", 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.878e-06 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659439+0000", 2024-09-13T07:04:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042670642000000002 2024-09-13T07:04:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659512+0000", 2024-09-13T07:04:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-05 2024-09-13T07:04:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659580+0000", 2024-09-13T07:04:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8678999999999998e-05 2024-09-13T07:04:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93984 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3309568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.509815076, 2024-09-13T07:04:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042972854999999997, 2024-09-13T07:04:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93984 2024-09-13T07:04:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616772+0000", 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616771+0000", 2024-09-13T07:04:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616774+0000", 2024-09-13T07:04:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.278e-06 2024-09-13T07:04:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616778+0000", 2024-09-13T07:04:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-13T07:04:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616782+0000", 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0589999999999996e-06 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659606+0000", 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823965999999998 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659678+0000", 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2464000000000003e-05 2024-09-13T07:04:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659745+0000", 2024-09-13T07:04:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6324999999999999e-05 2024-09-13T07:04:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93985 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.509801869, 2024-09-13T07:04:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043122187999999999, 2024-09-13T07:04:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93985 2024-09-13T07:04:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616785+0000", 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616787+0000", 2024-09-13T07:04:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2019999999999998e-06 2024-09-13T07:04:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616792+0000", 2024-09-13T07:04:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9599999999999999e-06 2024-09-13T07:04:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616796+0000", 2024-09-13T07:04:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2100000000000003e-06 2024-09-13T07:04:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659770+0000", 2024-09-13T07:04:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042974105999999998 2024-09-13T07:04:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659841+0000", 2024-09-13T07:04:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0796000000000005e-05 2024-09-13T07:04:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659907+0000", 2024-09-13T07:04:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6264000000000001e-05 2024-09-13T07:04:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93986 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.50978780200001, 2024-09-13T07:04:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043256274999999997, 2024-09-13T07:04:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93986 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616799+0000", 2024-09-13T07:04:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616801+0000", 2024-09-13T07:04:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1969999999999999e-06 2024-09-13T07:04:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616806+0000", 2024-09-13T07:04:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1909999999999999e-06 2024-09-13T07:04:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616813+0000", 2024-09-13T07:04:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5370000000000003e-06 2024-09-13T07:04:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.659933+0000", 2024-09-13T07:04:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043120190000000003 2024-09-13T07:04:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660000+0000", 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7669999999999994e-05 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660055+0000", 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4836000000000001e-05 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93987 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.509767788, 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043526417999999997, 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93987 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616819+0000", 2024-09-13T07:04:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616821+0000", 2024-09-13T07:04:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-13T07:04:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616826+0000", 2024-09-13T07:04:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7450000000000003e-06 2024-09-13T07:04:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616833+0000", 2024-09-13T07:04:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9949999999999999e-06 2024-09-13T07:04:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660087+0000", 2024-09-13T07:04:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043254125999999997 2024-09-13T07:04:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660215+0000", 2024-09-13T07:04:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012809900000000001 2024-09-13T07:04:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660345+0000", 2024-09-13T07:04:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013024299999999999 2024-09-13T07:04:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93988 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.509748038, 2024-09-13T07:04:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043785335000000002, 2024-09-13T07:04:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93988 2024-09-13T07:04:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616839+0000", 2024-09-13T07:04:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616838+0000", 2024-09-13T07:04:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616842+0000", 2024-09-13T07:04:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5250000000000001e-06 2024-09-13T07:04:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616851+0000", 2024-09-13T07:04:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.132e-06 2024-09-13T07:04:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616858+0000", 2024-09-13T07:04:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0330000000000003e-06 2024-09-13T07:04:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660386+0000", 2024-09-13T07:04:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043527608000000002 2024-09-13T07:04:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660491+0000", 2024-09-13T07:04:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105887 2024-09-13T07:04:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660624+0000", 2024-09-13T07:04:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013265799999999999 2024-09-13T07:04:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93989 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.509723232, 2024-09-13T07:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043995478999999997, 2024-09-13T07:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93989 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616864+0000", 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616863+0000", 2024-09-13T07:04:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616866+0000", 2024-09-13T07:04:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2179999999999999e-06 2024-09-13T07:04:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616875+0000", 2024-09-13T07:04:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849999999999992e-06 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616882+0000", 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9809999999999997e-06 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660661+0000", 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043779039999999998 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660753+0000", 2024-09-13T07:04:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1892999999999996e-05 2024-09-13T07:04:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660859+0000", 2024-09-13T07:04:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106382 2024-09-13T07:04:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93990 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.50969896699999, 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044183192000000003, 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93990 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616888+0000", 2024-09-13T07:04:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616887+0000", 2024-09-13T07:04:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616891+0000", 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709999999999999e-06 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616908+0000", 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7011999999999999e-05 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616916+0000", 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0080000000000002e-06 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660896+0000", 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043980588000000001 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.660982+0000", 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5156999999999998e-05 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661071+0000", 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9462000000000005e-05 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93991 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.509657234, 2024-09-13T07:04:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044276619000000003, 2024-09-13T07:04:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93991 2024-09-13T07:04:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616930+0000", 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616929+0000", 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616933+0000", 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3109999999999999e-06 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616951+0000", 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7652e-05 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616959+0000", 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1380000000000007e-06 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661098+0000", 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044139460999999998 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661147+0000", 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8260999999999998e-05 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661206+0000", 2024-09-13T07:04:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9432000000000003e-05 2024-09-13T07:04:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:93992 3.0 3:00670bfc:::rbd_data.1194a0700cb4.0000000000000004:1c [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.50962094499999, 2024-09-13T07:04:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044411715999999997, 2024-09-13T07:04:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93992 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616966+0000", 2024-09-13T07:04:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616965+0000", 2024-09-13T07:04:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616969+0000", 2024-09-13T07:04:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7120000000000002e-06 2024-09-13T07:04:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616978+0000", 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6880000000000005e-06 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.616985+0000", 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-06 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661231+0000", 2024-09-13T07:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044245648999999998 2024-09-13T07:04:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661308+0000", 2024-09-13T07:04:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430999999999998e-05 2024-09-13T07:04:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661378+0000", 2024-09-13T07:04:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.949e-05 2024-09-13T07:04:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126630 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2310144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.066517613, 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042606894999999999, 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126630 2024-09-13T07:04:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060069+0000", 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060072+0000", 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7460000000000001e-06 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060077+0000", 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7980000000000002e-06 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060082+0000", 2024-09-13T07:04:36.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8040000000000001e-06 2024-09-13T07:04:36.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101424+0000", 2024-09-13T07:04:36.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.041341777000000003 2024-09-13T07:04:36.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.101490+0000", 2024-09-13T07:04:36.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6464999999999995e-05 2024-09-13T07:04:36.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102676+0000", 2024-09-13T07:04:36.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011856530000000001 2024-09-13T07:04:36.201 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.201 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.201 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.201 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126631 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2342912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.066500731, 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042823970000000003, 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126631 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:36.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:36.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060086+0000", 2024-09-13T07:04:36.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060088+0000", 2024-09-13T07:04:36.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2249999999999999e-06 2024-09-13T07:04:36.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060093+0000", 2024-09-13T07:04:36.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4809999999999997e-06 2024-09-13T07:04:36.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.060098+0000", 2024-09-13T07:04:36.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5940000000000004e-06 2024-09-13T07:04:36.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102707+0000", 2024-09-13T07:04:36.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042608673 2024-09-13T07:04:36.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102779+0000", 2024-09-13T07:04:36.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1952000000000005e-05 2024-09-13T07:04:36.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.102910+0000", 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013143399999999999 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126634 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2441216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.06545017400001, 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042405453000000003, 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.206 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126634 2024-09-13T07:04:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061137+0000", 2024-09-13T07:04:36.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061136+0000", 2024-09-13T07:04:36.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061139+0000", 2024-09-13T07:04:36.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.072e-06 2024-09-13T07:04:36.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061145+0000", 2024-09-13T07:04:36.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2809999999999999e-06 2024-09-13T07:04:36.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.061150+0000", 2024-09-13T07:04:36.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0340000000000004e-06 2024-09-13T07:04:36.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103372+0000", 2024-09-13T07:04:36.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042222016000000001 2024-09-13T07:04:36.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103433+0000", 2024-09-13T07:04:36.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699000000000007e-05 2024-09-13T07:04:36.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.103542+0000", 2024-09-13T07:04:36.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108786 2024-09-13T07:04:36.210 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.210 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.210 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126639 3.0 3:011f8b38:::rbd_data.119487223aab.0000000000000001:23 [read 2605056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.06377562500001, 2024-09-13T07:04:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042358326000000002, 2024-09-13T07:04:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126639 2024-09-13T07:04:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.212 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:36.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:36.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062811+0000", 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062814+0000", 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2160000000000002e-06 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062820+0000", 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.062825+0000", 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0379999999999999e-06 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.104975+0000", 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042149589000000001 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105036+0000", 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1779999999999995e-05 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.105169+0000", 2024-09-13T07:04:36.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000133065 2024-09-13T07:04:36.215 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.215 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.215 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.215 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:133901 3.d 3.27b5dc2d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-13T07:04:36.215 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:36.215 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.543193235, 2024-09-13T07:04:36.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95839632299999999, 2024-09-13T07:04:36.215 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 133901 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583394+0000", 2024-09-13T07:04:36.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583391+0000", 2024-09-13T07:04:36.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:36.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583404+0000", 2024-09-13T07:04:36.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3015e-05 2024-09-13T07:04:36.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583432+0000", 2024-09-13T07:04:36.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7497000000000001e-05 2024-09-13T07:04:36.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583445+0000", 2024-09-13T07:04:36.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-13T07:04:36.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583520+0000", 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5377999999999996e-05 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:50.583531+0000", 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1234e-05 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541759+0000", 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.95822760399999996 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:51.541790+0000", 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0987e-05 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172880 2.2 2:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e149)", 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.988896988999997, 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.97218948500000002, 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.220 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.221 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.221 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172880 2024-09-13T07:04:36.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.221 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:36.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137690+0000", 2024-09-13T07:04:36.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137687+0000", 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137702+0000", 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5023e-05 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137730+0000", 2024-09-13T07:04:36.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7719e-05 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137748+0000", 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8189000000000002e-05 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137891+0000", 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014330699999999999 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:36.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.137903+0000", 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1688999999999999e-05 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142887+0000", 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004984006 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.142898+0000", 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.079e-05 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109703+0000", 2024-09-13T07:04:36.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96680553899999999 2024-09-13T07:04:36.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:45.109879+0000", 2024-09-13T07:04:36.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017579800000000001 2024-09-13T07:04:36.225 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.225 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.225 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.225 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.225 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.225 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:36.227 INFO:teuthology.orchestra.run.smithi118.stderr:noscrub is unset 2024-09-13T07:04:36.227 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T07:04:36.228 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T07:04:36.247 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:04:36.278 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-13T07:04:36.277+0000 7f609682e700 -1 osd.5 200 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T07:04:36.329 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:36.329 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:36.339 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T07:04:36.347 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:36.347 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:36.347 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:36.347 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:36.440 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:36.441 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:36.441 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:36.441 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:36.441 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:36.450 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T07:04:36.560 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-13T07:04:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:1524 3.14 e27/22 3:28c7c923:::rbd_data.11943a018f34.0000000000000001:head v 27'12, mlcod=27'10)", 2024-09-13T07:04:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 190.505268677, 2024-09-13T07:04:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051424076999999999, 2024-09-13T07:04:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054172+0000", 2024-09-13T07:04:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.054171+0000", 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059708+0000", 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055367990000000002 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059739+0000", 2024-09-13T07:04:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0423e-05 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059752+0000", 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3822000000000001e-05 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059881+0000", 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000128195 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.059916+0000", 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5289e-05 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105540+0000", 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045624248999999999 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:26.105597+0000", 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6341000000000001e-05 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48369 3.f 3:f245db6b:::rbd_data.1194fc792839.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 158.347387443, 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.086557162000000007, 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48369 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212054+0000", 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.212052+0000", 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237720+0000", 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025668854000000001 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237757+0000", 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6241000000000001e-05 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237774+0000", 2024-09-13T07:04:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237864+0000", 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9721000000000004e-05 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.238172+0000", 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000307962 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.239305+0000", 2024-09-13T07:04:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001132645 2024-09-13T07:04:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.297849+0000", 2024-09-13T07:04:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058544185999999998 2024-09-13T07:04:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298355+0000", 2024-09-13T07:04:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050614599999999996 2024-09-13T07:04:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298523+0000", 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016816699999999999 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298611+0000", 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7764999999999997e-05 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48371 3.f 3:f499cffe:::rbd_data.1194fc792839.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 158.32161773199999, 2024-09-13T07:04:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.061230540999999999, 2024-09-13T07:04:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48371 2024-09-13T07:04:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237823+0000", 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.237822+0000", 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258398+0000", 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020576865 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258423+0000", 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4417999999999999e-05 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258435+0000", 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1834000000000001e-05 2024-09-13T07:04:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258582+0000", 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014743199999999999 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258723+0000", 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140822 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.259256+0000", 2024-09-13T07:04:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053322299999999999 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298642+0000", 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039385864 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.298942+0000", 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029975200000000003 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299002+0000", 2024-09-13T07:04:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0387999999999998e-05 2024-09-13T07:04:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299054+0000", 2024-09-13T07:04:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1817999999999998e-05 2024-09-13T07:04:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:48375 3.11 3:89e977b7:::rbd_data.1194fc792839.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,zero 13697024~2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T07:04:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 158.30046656600001, 2024-09-13T07:04:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.058859331000000001, 2024-09-13T07:04:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 48375 2024-09-13T07:04:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258975+0000", 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.258974+0000", 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.999999 2024-09-13T07:04:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268502+0000", 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095280859999999998 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268525+0000", 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3691000000000001e-05 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268538+0000", 2024-09-13T07:04:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3059e-05 2024-09-13T07:04:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268621+0000", 2024-09-13T07:04:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2884000000000001e-05 2024-09-13T07:04:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.268812+0000", 2024-09-13T07:04:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190955 2024-09-13T07:04:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.269376+0000", 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056368399999999995 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.299248+0000", 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02987192 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317676+0000", 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018428488 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317754+0000", 2024-09-13T07:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7774000000000004e-05 2024-09-13T07:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:01:58.317834+0000", 2024-09-13T07:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9715999999999994e-05 2024-09-13T07:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:64545 3.f 3:f1cfd605:::rbd_object_map.119478fa7495:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 148.18342180299999, 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051509424999999998, 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 64545 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376019+0000", 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376021+0000", 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376025+0000", 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9639999999999997e-06 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.376029+0000", 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6619999999999998e-06 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379303+0000", 2024-09-13T07:04:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032744089999999998 2024-09-13T07:04:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379421+0000", 2024-09-13T07:04:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117253 2024-09-13T07:04:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-13T07:04:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.379893+0000", 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047182099999999998 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.381419+0000", 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001526548 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427407+0000", 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045987515 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427481+0000", 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4729999999999998e-05 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:08.427529+0000", 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7444000000000002e-05 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:94597 3.13 3:ce60cd0e:::rbd_data.1194a0700cb4.0000000000000000:1c [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T07:04:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.89842455799999, 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044413176999999998, 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 94597 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661017+0000", 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661016+0000", 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661019+0000", 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4060000000000001e-06 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661025+0000", 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.981e-06 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.661030+0000", 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6419999999999998e-06 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705352+0000", 2024-09-13T07:04:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044322385999999998 2024-09-13T07:04:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705389+0000", 2024-09-13T07:04:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7104000000000003e-05 2024-09-13T07:04:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:26.705430+0000", 2024-09-13T07:04:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0327999999999997e-05 2024-09-13T07:04:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107061 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1010688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.64671542000001, 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044482304, 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107061 2024-09-13T07:04:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912726+0000", 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912725+0000", 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912728+0000", 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6649999999999999e-06 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912735+0000", 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0940000000000002e-06 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912740+0000", 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8080000000000004e-06 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957075+0000", 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044335303 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957143+0000", 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8106000000000001e-05 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957208+0000", 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4660999999999996e-05 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107062 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.646697386, 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689541999999999, 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107062 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912744+0000", 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912743+0000", 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912746+0000", 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7039999999999999e-06 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912751+0000", 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000001e-06 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912756+0000", 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957251+0000", 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044494958000000001 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957336+0000", 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4849e-05 2024-09-13T07:04:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957433+0000", 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7639000000000005e-05 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107063 3.d 3:b421626d:::rbd_data.1194f8f64d7f.0000000000000017:1f [read 1025024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.646681306, 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044872013000000002, 2024-09-13T07:04:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107063 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912760+0000", 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912759+0000", 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912762+0000", 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.429e-06 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912767+0000", 2024-09-13T07:04:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1340000000000003e-06 2024-09-13T07:04:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912772+0000", 2024-09-13T07:04:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8539999999999996e-06 2024-09-13T07:04:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957462+0000", 2024-09-13T07:04:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044689801000000001 2024-09-13T07:04:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957539+0000", 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7420000000000001e-05 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957632+0000", 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2738999999999994e-05 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107064 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.64666563, 2024-09-13T07:04:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044905686, 2024-09-13T07:04:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107064 2024-09-13T07:04:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912776+0000", 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912775+0000", 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912784+0000", 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4090000000000003e-06 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912789+0000", 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5690000000000004e-06 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912794+0000", 2024-09-13T07:04:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7319999999999998e-06 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957672+0000", 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044878253999999999 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957681+0000", 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.054e-06 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107065 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.646642708, 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044921316000000003, 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107065 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912798+0000", 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912801+0000", 2024-09-13T07:04:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912806+0000", 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3399999999999997e-06 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912811+0000", 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5349999999999998e-06 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957710+0000", 2024-09-13T07:04:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044899465 2024-09-13T07:04:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957720+0000", 2024-09-13T07:04:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6840000000000008e-06 2024-09-13T07:04:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107066 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.646626611, 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044931928000000003, 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107066 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912815+0000", 2024-09-13T07:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912814+0000", 2024-09-13T07:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912816+0000", 2024-09-13T07:04:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2299999999999998e-06 2024-09-13T07:04:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912822+0000", 2024-09-13T07:04:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.057e-06 2024-09-13T07:04:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.912828+0000", 2024-09-13T07:04:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5540000000000003e-06 2024-09-13T07:04:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957741+0000", 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044913058999999998 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957746+0000", 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3480000000000003e-06 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107069 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.64602499900001, 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044431452000000003, 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107069 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913416+0000", 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913418+0000", 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4889999999999998e-06 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913424+0000", 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2850000000000002e-06 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913428+0000", 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6410000000000001e-06 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957838+0000", 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044409584000000002 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957848+0000", 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8079999999999999e-06 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:107070 3.d 3.b646842d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T07:04:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.645982223, 2024-09-13T07:04:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044438371999999997, 2024-09-13T07:04:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107070 2024-09-13T07:04:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913459+0000", 2024-09-13T07:04:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913458+0000", 2024-09-13T07:04:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913461+0000", 2024-09-13T07:04:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-13T07:04:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913469+0000", 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7160000000000003e-06 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.913474+0000", 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957886+0000", 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044412041999999999 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:31.957897+0000", 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0845e-05 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:126662 3.11 3.e71a9b11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.48896445299999, 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051528820000000003, 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 126662 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070477+0000", 2024-09-13T07:04:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070476+0000", 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999995 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070479+0000", 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1569999999999998e-06 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070484+0000", 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.113e-06 2024-09-13T07:04:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.070488+0000", 2024-09-13T07:04:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4519999999999999e-06 2024-09-13T07:04:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.113541+0000", 2024-09-13T07:04:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043052836999999997 2024-09-13T07:04:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:02:42.122006+0000", 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084645870000000008 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:145848 3.d 3.eb4f998d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.636587523000003, 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.17360805400000001, 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145848 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922854+0000", 2024-09-13T07:04:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922852+0000", 2024-09-13T07:04:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922866+0000", 2024-09-13T07:04:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-13T07:04:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922879+0000", 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3404000000000001e-05 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922888+0000", 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7420000000000003e-06 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922962+0000", 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4049999999999997e-05 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:13.922976+0000", 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4124000000000001e-05 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096392+0000", 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.173416391 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:14.096462+0000", 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9286e-05 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4500.0:164179 3.14 e135/67 3:2fc485bd:::rbd_data.11942e4f5fc7.0000000000000002:head v 135'401, mlcod=132'398)", 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 67.613647068999995, 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.063492174999999998, 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945794+0000", 2024-09-13T07:04:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.945792+0000", 2024-09-13T07:04:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999981 2024-09-13T07:04:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954811+0000", 2024-09-13T07:04:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090187940000000001 2024-09-13T07:04:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954840+0000", 2024-09-13T07:04:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8659000000000001e-05 2024-09-13T07:04:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954852+0000", 2024-09-13T07:04:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1966e-05 2024-09-13T07:04:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954938+0000", 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6539000000000006e-05 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:28.954975+0000", 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6303999999999997e-05 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009206+0000", 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.054231044999999999 2024-09-13T07:04:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.009286+0000", 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0700999999999999e-05 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:165953 3.d 3:b3a20b45:::rbd_data.11942e4f5fc7.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8796117~39260] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 67.337199510999994, 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.095290105, 2024-09-13T07:04:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T07:04:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 165953 2024-09-13T07:04:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222242+0000", 2024-09-13T07:04:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.222240+0000", 2024-09-13T07:04:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999986 2024-09-13T07:04:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258929+0000", 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.036688401000000002 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258971+0000", 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2212000000000002e-05 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.258989+0000", 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7674000000000001e-05 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259072+0000", 2024-09-13T07:04:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3656e-05 2024-09-13T07:04:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.259297+0000", 2024-09-13T07:04:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022450700000000001 2024-09-13T07:04:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.260463+0000", 2024-09-13T07:04:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011665340000000001 2024-09-13T07:04:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.316214+0000", 2024-09-13T07:04:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.055750268999999998 2024-09-13T07:04:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-13T07:04:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317335+0000", 2024-09-13T07:04:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011211719999999999 2024-09-13T07:04:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317451+0000", 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011616 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:29.317532+0000", 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0860999999999997e-05 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172852 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [call lock.unlock in=52b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 53.360742363999996, 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.90390594199999996, 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172852 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198699+0000", 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198696+0000", 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198710+0000", 2024-09-13T07:04:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3548e-05 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198739+0000", 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9606000000000001e-05 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198752+0000", 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3033e-05 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.198869+0000", 2024-09-13T07:04:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116552 2024-09-13T07:04:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199018+0000", 2024-09-13T07:04:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014923700000000001 2024-09-13T07:04:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-13T07:04:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.199308+0000", 2024-09-13T07:04:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028947199999999998 2024-09-13T07:04:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:43.200731+0000", 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014229799999999999 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059443+0000", 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.85871189999999997 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059467+0000", 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.478e-05 2024-09-13T07:04:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102380+0000", 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042912463999999997 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102605+0000", 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022489499999999999 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4500.0:172853 3.d 3:b199bddb:::rbd_header.1194c4b8a5ed:head [watch reconnect cookie 140335574242864 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.508993504999999, 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.056267725999999997, 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4500", 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.118:0/3004759609", 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 172853 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050448+0000", 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050445+0000", 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4294967295.9999976 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-13T07:04:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050461+0000", 2024-09-13T07:04:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5783000000000001e-05 2024-09-13T07:04:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-13T07:04:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050497+0000", 2024-09-13T07:04:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5995999999999999e-05 2024-09-13T07:04:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-13T07:04:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050516+0000", 2024-09-13T07:04:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9579999999999999e-05 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050637+0000", 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120536 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.050647+0000", 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6989999999999998e-06 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059533+0000", 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088858749999999997 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.059545+0000", 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2473000000000001e-05 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.102962+0000", 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043416656999999997 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-13T07:04:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.103612+0000", 2024-09-13T07:04:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000650374 2024-09-13T07:04:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-13T07:04:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-13T07:04:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-13T07:04:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-13T07:03:44.106715+0000", 2024-09-13T07:04:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031033229999999998 2024-09-13T07:04:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-13T07:04:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-13T07:04:36.657 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:36.659 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T07:04:36.773 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-13T07:04:36.774 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:36.784 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T07:04:36.890 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-13T07:04:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-13T07:04:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-13T07:04:36.890 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:36.899 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T07:04:37.007 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-13T07:04:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-13T07:04:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-13T07:04:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [] 2024-09-13T07:04:37.007 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-13T07:04:37.215 INFO:teuthology.orchestra.run.smithi118.stderr:nodeep-scrub is unset 2024-09-13T07:04:41.358 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-13T07:04:41.358 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-13T07:04:41.359 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-09-13T07:04:41.681 INFO:tasks.ceph.mon.a.smithi118.stderr:2024-09-13T07:04:41.677+0000 7fd1ec319700 -1 mon.a@0(leader).osd e205 definitely_dead 0 2024-09-13T07:04:42.248 INFO:teuthology.orchestra.run.smithi118.stderr:marked in osd.7. 2024-09-13T07:04:42.265 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-13T07:04:42.265 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-13T07:04:42.390 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a: {} 2024-09-13T07:04:42.402 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b: {} 2024-09-13T07:04:42.413 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c: {} 2024-09-13T07:04:42.415 INFO:teuthology.orchestra.run.smithi118.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T07:04:42.415 INFO:teuthology.orchestra.run.smithi118.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T07:04:42.415 INFO:teuthology.orchestra.run.smithi118.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T07:04:42.427 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T07:04:43.265 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T07:04:43.278 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-13T07:04:44.122 INFO:teuthology.orchestra.run.smithi118.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-13T07:04:44.135 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T07:04:45.133 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T07:04:45.150 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-13T07:04:46.130 INFO:teuthology.orchestra.run.smithi118.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-13T07:04:46.145 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T07:04:47.135 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T07:04:47.151 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-13T07:04:48.129 INFO:teuthology.orchestra.run.smithi118.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-13T07:04:48.144 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T07:04:49.122 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T07:04:49.134 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-13T07:04:50.127 INFO:teuthology.orchestra.run.smithi118.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-13T07:04:50.141 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T07:04:51.129 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T07:04:51.141 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-13T07:04:52.127 INFO:teuthology.orchestra.run.smithi118.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-13T07:04:52.141 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T07:04:53.131 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T07:04:53.144 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-13T07:04:54.123 INFO:teuthology.orchestra.run.smithi118.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-13T07:04:54.135 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T07:04:55.301 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T07:04:55.315 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-13T07:04:56.307 INFO:teuthology.orchestra.run.smithi118.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-13T07:04:56.320 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T07:04:57.321 INFO:teuthology.orchestra.run.smithi118.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T07:04:57.333 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-13T07:04:58.326 INFO:teuthology.orchestra.run.smithi118.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-13T07:04:58.338 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-13T07:04:58.339 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-13T07:04:58.339 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-13T07:04:58.339 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-13T07:04:58.339 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T07:04:58.339 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T07:04:58.645 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:04:58.645 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":222,"fsid":"25bb30d1-9541-4e40-8659-18d83bb1bd8a","created":"2024-09-13T07:00:49.323883+0000","modified":"2024-09-13T07:04:58.313385+0000","last_up_change":"2024-09-13T07:04:37.195137+0000","last_in_change":"2024-09-13T07:04:42.234060+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T07:01:00.675365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T07:01:07.404263+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T07:01:17.321011+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":103,"snap_epoch":199,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21736eee-00cd-4eef-9725-eca7776957b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6819","nonce":3731386737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6823","nonce":3731386737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6831","nonce":3731386737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6827","nonce":3731386737}]},"public_addr":"172.21.15.118:6819/3731386737","cluster_addr":"172.21.15.118:6823/3731386737","heartbeat_back_addr":"172.21.15.118:6831/3731386737","heartbeat_front_addr":"172.21.15.118:6827/3731386737","state":["exists","up"]},{"osd":1,"uuid":"05818db2-841d-406d-be17-8fbdab4fff69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6818","nonce":876310814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6822","nonce":876310814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6830","nonce":876310814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6826","nonce":876310814}]},"public_addr":"172.21.15.118:6818/876310814","cluster_addr":"172.21.15.118:6822/876310814","heartbeat_back_addr":"172.21.15.118:6830/876310814","heartbeat_front_addr":"172.21.15.118:6826/876310814","state":["exists","up"]},{"osd":2,"uuid":"2387761c-42d2-481b-b5b5-b16d3a943b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6809","nonce":690188762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6811","nonce":690188762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6815","nonce":690188762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6813","nonce":690188762}]},"public_addr":"172.21.15.118:6809/690188762","cluster_addr":"172.21.15.118:6811/690188762","heartbeat_back_addr":"172.21.15.118:6815/690188762","heartbeat_front_addr":"172.21.15.118:6813/690188762","state":["exists","up"]},{"osd":3,"uuid":"56500e90-f76b-47c8-a649-893400359c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6801","nonce":2198175481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6803","nonce":2198175481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6807","nonce":2198175481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6805","nonce":2198175481}]},"public_addr":"172.21.15.118:6801/2198175481","cluster_addr":"172.21.15.118:6803/2198175481","heartbeat_back_addr":"172.21.15.118:6807/2198175481","heartbeat_front_addr":"172.21.15.118:6805/2198175481","state":["exists","up"]},{"osd":4,"uuid":"f594a612-aa07-4565-a09e-08368cf397b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6813","nonce":1835601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6819","nonce":1835601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6830","nonce":1835601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6825","nonce":1835601651}]},"public_addr":"172.21.15.164:6813/1835601651","cluster_addr":"172.21.15.164:6819/1835601651","heartbeat_back_addr":"172.21.15.164:6830/1835601651","heartbeat_front_addr":"172.21.15.164:6825/1835601651","state":["exists","up"]},{"osd":5,"uuid":"1013b81f-e6b4-430b-938d-88759d43b881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":147,"up_from":201,"up_thru":210,"down_at":148,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2019781558},{"type":"v1","addr":"172.21.15.164:6801","nonce":2019781558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2019781558},{"type":"v1","addr":"172.21.15.164:6803","nonce":2019781558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2019781558},{"type":"v1","addr":"172.21.15.164:6807","nonce":2019781558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2019781558},{"type":"v1","addr":"172.21.15.164:6805","nonce":2019781558}]},"public_addr":"172.21.15.164:6801/2019781558","cluster_addr":"172.21.15.164:6803/2019781558","heartbeat_back_addr":"172.21.15.164:6807/2019781558","heartbeat_front_addr":"172.21.15.164:6805/2019781558","state":["exists","up"]},{"osd":6,"uuid":"367f5f5b-5919-4534-b88c-229f327e4950","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6812","nonce":131309660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6815","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6818","nonce":131309660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6831","nonce":131309660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6824","nonce":131309660}]},"public_addr":"172.21.15.164:6812/131309660","cluster_addr":"172.21.15.164:6818/131309660","heartbeat_back_addr":"172.21.15.164:6831/131309660","heartbeat_front_addr":"172.21.15.164:6824/131309660","state":["exists","up"]},{"osd":7,"uuid":"90d9e486-e1ae-4305-80a6-6f483e534643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6811","nonce":652173699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6816","nonce":652173699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6829","nonce":652173699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6823","nonce":652173699}]},"public_addr":"172.21.15.164:6811/652173699","cluster_addr":"172.21.15.164:6816/652173699","heartbeat_back_addr":"172.21.15.164:6829/652173699","heartbeat_front_addr":"172.21.15.164:6823/652173699","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.059233+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.083130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.848896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.847456+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.914135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.834284+0000","dead_epoch":148},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.003715+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.033514+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:04:58.660 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T07:04:58.660 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T07:04:58.965 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:04:58.966 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":222,"fsid":"25bb30d1-9541-4e40-8659-18d83bb1bd8a","created":"2024-09-13T07:00:49.323883+0000","modified":"2024-09-13T07:04:58.313385+0000","last_up_change":"2024-09-13T07:04:37.195137+0000","last_in_change":"2024-09-13T07:04:42.234060+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T07:01:00.675365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T07:01:07.404263+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T07:01:17.321011+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":103,"snap_epoch":199,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21736eee-00cd-4eef-9725-eca7776957b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6819","nonce":3731386737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6823","nonce":3731386737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6831","nonce":3731386737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6827","nonce":3731386737}]},"public_addr":"172.21.15.118:6819/3731386737","cluster_addr":"172.21.15.118:6823/3731386737","heartbeat_back_addr":"172.21.15.118:6831/3731386737","heartbeat_front_addr":"172.21.15.118:6827/3731386737","state":["exists","up"]},{"osd":1,"uuid":"05818db2-841d-406d-be17-8fbdab4fff69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6818","nonce":876310814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6822","nonce":876310814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6830","nonce":876310814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6826","nonce":876310814}]},"public_addr":"172.21.15.118:6818/876310814","cluster_addr":"172.21.15.118:6822/876310814","heartbeat_back_addr":"172.21.15.118:6830/876310814","heartbeat_front_addr":"172.21.15.118:6826/876310814","state":["exists","up"]},{"osd":2,"uuid":"2387761c-42d2-481b-b5b5-b16d3a943b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6809","nonce":690188762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6811","nonce":690188762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6815","nonce":690188762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6813","nonce":690188762}]},"public_addr":"172.21.15.118:6809/690188762","cluster_addr":"172.21.15.118:6811/690188762","heartbeat_back_addr":"172.21.15.118:6815/690188762","heartbeat_front_addr":"172.21.15.118:6813/690188762","state":["exists","up"]},{"osd":3,"uuid":"56500e90-f76b-47c8-a649-893400359c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6801","nonce":2198175481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6803","nonce":2198175481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6807","nonce":2198175481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6805","nonce":2198175481}]},"public_addr":"172.21.15.118:6801/2198175481","cluster_addr":"172.21.15.118:6803/2198175481","heartbeat_back_addr":"172.21.15.118:6807/2198175481","heartbeat_front_addr":"172.21.15.118:6805/2198175481","state":["exists","up"]},{"osd":4,"uuid":"f594a612-aa07-4565-a09e-08368cf397b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6813","nonce":1835601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6819","nonce":1835601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6830","nonce":1835601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6825","nonce":1835601651}]},"public_addr":"172.21.15.164:6813/1835601651","cluster_addr":"172.21.15.164:6819/1835601651","heartbeat_back_addr":"172.21.15.164:6830/1835601651","heartbeat_front_addr":"172.21.15.164:6825/1835601651","state":["exists","up"]},{"osd":5,"uuid":"1013b81f-e6b4-430b-938d-88759d43b881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":147,"up_from":201,"up_thru":210,"down_at":148,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2019781558},{"type":"v1","addr":"172.21.15.164:6801","nonce":2019781558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2019781558},{"type":"v1","addr":"172.21.15.164:6803","nonce":2019781558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2019781558},{"type":"v1","addr":"172.21.15.164:6807","nonce":2019781558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2019781558},{"type":"v1","addr":"172.21.15.164:6805","nonce":2019781558}]},"public_addr":"172.21.15.164:6801/2019781558","cluster_addr":"172.21.15.164:6803/2019781558","heartbeat_back_addr":"172.21.15.164:6807/2019781558","heartbeat_front_addr":"172.21.15.164:6805/2019781558","state":["exists","up"]},{"osd":6,"uuid":"367f5f5b-5919-4534-b88c-229f327e4950","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6812","nonce":131309660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6815","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6818","nonce":131309660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6831","nonce":131309660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6824","nonce":131309660}]},"public_addr":"172.21.15.164:6812/131309660","cluster_addr":"172.21.15.164:6818/131309660","heartbeat_back_addr":"172.21.15.164:6831/131309660","heartbeat_front_addr":"172.21.15.164:6824/131309660","state":["exists","up"]},{"osd":7,"uuid":"90d9e486-e1ae-4305-80a6-6f483e534643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6811","nonce":652173699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6816","nonce":652173699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6829","nonce":652173699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6823","nonce":652173699}]},"public_addr":"172.21.15.164:6811/652173699","cluster_addr":"172.21.15.164:6816/652173699","heartbeat_back_addr":"172.21.15.164:6829/652173699","heartbeat_front_addr":"172.21.15.164:6823/652173699","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.059233+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.083130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.848896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.847456+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.914135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.834284+0000","dead_epoch":148},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.003715+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.033514+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:04:58.981 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T07:04:58.981 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T07:04:58.982 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T07:04:58.982 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T07:04:58.982 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T07:04:58.983 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T07:04:58.983 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T07:04:58.984 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T07:04:59.181 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:04:59.182 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T07:04:59.193 INFO:teuthology.orchestra.run.smithi118.stdout:55834574900 2024-09-13T07:04:59.194 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T07:04:59.203 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:04:59.203 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T07:04:59.209 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:04:59.209 INFO:teuthology.orchestra.run.smithi118.stdout:863288426502 2024-09-13T07:04:59.210 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T07:04:59.210 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T07:04:59.211 INFO:teuthology.orchestra.run.smithi118.stdout:55834574900 2024-09-13T07:04:59.212 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T07:04:59.219 INFO:teuthology.orchestra.run.smithi118.stdout:55834574900 2024-09-13T07:04:59.219 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T07:04:59.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:04:59.337+0000 7f599aa20700 0 --2- 172.21.15.118:0/873768015 >> v2:172.21.15.118:3300/0 conn(0x7f5994070e40 0x7f5994147370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T07:04:59.402 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:04:59.403 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T07:04:59.635 INFO:teuthology.orchestra.run.smithi118.stdout:55834574898 2024-09-13T07:04:59.651 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:04:59.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574898 for osd.2 2024-09-13T07:04:59.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574900 got 55834574899 for osd.1 2024-09-13T07:04:59.703 INFO:teuthology.orchestra.run.smithi118.stdout:55834574898 2024-09-13T07:04:59.712 INFO:teuthology.orchestra.run.smithi118.stdout:863288426501 2024-09-13T07:04:59.717 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:04:59.721 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574898 for osd.3 2024-09-13T07:04:59.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 863288426502 got 863288426501 for osd.5 2024-09-13T07:04:59.730 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574900 got 55834574899 for osd.6 2024-09-13T07:04:59.749 INFO:teuthology.orchestra.run.smithi118.stdout:55834574898 2024-09-13T07:04:59.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574898 for osd.4 2024-09-13T07:04:59.769 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:04:59.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574900 got 55834574899 for osd.7 2024-09-13T07:04:59.802 INFO:teuthology.orchestra.run.smithi118.stdout:55834574898 2024-09-13T07:04:59.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574898 for osd.0 2024-09-13T07:05:00.653 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T07:05:00.673 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T07:05:00.722 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T07:05:00.726 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T07:05:00.731 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T07:05:00.770 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T07:05:00.782 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T07:05:00.815 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T07:05:01.102 INFO:teuthology.orchestra.run.smithi118.stdout:55834574898 2024-09-13T07:05:01.120 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574898 for osd.2 2024-09-13T07:05:01.173 INFO:teuthology.orchestra.run.smithi118.stdout:863288426501 2024-09-13T07:05:01.182 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:05:01.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 863288426502 got 863288426501 for osd.5 2024-09-13T07:05:01.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574900 got 55834574899 for osd.1 2024-09-13T07:05:01.200 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:05:01.202 INFO:teuthology.orchestra.run.smithi118.stdout:55834574898 2024-09-13T07:05:01.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574900 got 55834574899 for osd.6 2024-09-13T07:05:01.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574898 for osd.3 2024-09-13T07:05:01.225 INFO:teuthology.orchestra.run.smithi118.stdout:55834574898 2024-09-13T07:05:01.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574898 for osd.4 2024-09-13T07:05:01.257 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:05:01.264 INFO:teuthology.orchestra.run.smithi118.stdout:55834574898 2024-09-13T07:05:01.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574900 got 55834574899 for osd.7 2024-09-13T07:05:01.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574898 for osd.0 2024-09-13T07:05:02.121 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T07:05:02.192 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T07:05:02.200 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T07:05:02.215 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T07:05:02.220 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T07:05:02.239 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T07:05:02.279 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T07:05:02.282 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T07:05:02.445 INFO:teuthology.orchestra.run.smithi118.stderr:2024-09-13T07:05:02.445+0000 7f0d1d6d8700 0 --2- 172.21.15.118:0/2347274932 >> [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] conn(0x7f0d1806f900 0x7f0d1813cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T07:05:02.632 INFO:teuthology.orchestra.run.smithi118.stdout:55834574900 2024-09-13T07:05:02.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574900 got 55834574900 for osd.1 2024-09-13T07:05:02.652 DEBUG:teuthology.parallel:result is None 2024-09-13T07:05:02.655 INFO:teuthology.orchestra.run.smithi118.stdout:863288426502 2024-09-13T07:05:02.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 863288426502 got 863288426502 for osd.5 2024-09-13T07:05:02.672 DEBUG:teuthology.parallel:result is None 2024-09-13T07:05:02.685 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:05:02.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574899 for osd.4 2024-09-13T07:05:02.702 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:05:02.702 DEBUG:teuthology.parallel:result is None 2024-09-13T07:05:02.703 INFO:teuthology.orchestra.run.smithi118.stdout:55834574900 2024-09-13T07:05:02.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574900 got 55834574900 for osd.7 2024-09-13T07:05:02.717 DEBUG:teuthology.parallel:result is None 2024-09-13T07:05:02.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574899 for osd.3 2024-09-13T07:05:02.720 DEBUG:teuthology.parallel:result is None 2024-09-13T07:05:02.723 INFO:teuthology.orchestra.run.smithi118.stdout:55834574900 2024-09-13T07:05:02.730 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:05:02.736 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574900 got 55834574900 for osd.6 2024-09-13T07:05:02.736 DEBUG:teuthology.parallel:result is None 2024-09-13T07:05:02.736 INFO:teuthology.orchestra.run.smithi118.stdout:55834574899 2024-09-13T07:05:02.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574899 for osd.2 2024-09-13T07:05:02.744 DEBUG:teuthology.parallel:result is None 2024-09-13T07:05:02.749 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574899 for osd.0 2024-09-13T07:05:02.749 DEBUG:teuthology.parallel:result is None 2024-09-13T07:05:02.750 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T07:05:02.750 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:05:03.084 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:05:03.087 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:05:03.098 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-13T07:05:02.663768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":412,"num_object_clones":0,"num_object_copies":824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":412,"num_whiteouts":404,"num_read":239562,"num_read_kb":1058974,"num_write":24247,"num_write_kb":570384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5266,"num_bytes_recovered":116470768,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20858,"ondisk_log_size":20858,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":371516,"kb_used_data":22156,"kb_used_omap":46,"kb_used_meta":349201,"kb_avail":754603204,"statfs":{"total":773094113280,"available":772713680896,"internally_reserved":0,"allocated":22687744,"data_stored":11560198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48002,"internal_metadata":357581950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-672,"num_bytes_recovered":-590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.366275"},"pg_stats":[{"pgid":"3.1f","version":"202'415","reported_seq":6819,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322192+0000","last_change":"2024-09-13T07:04:48.134119+0000","last_active":"2024-09-13T07:04:57.322192+0000","last_peered":"2024-09-13T07:04:57.322192+0000","last_clean":"2024-09-13T07:04:57.322192+0000","last_became_active":"2024-09-13T07:04:48.131331+0000","last_became_peered":"2024-09-13T07:04:48.131331+0000","last_unstale":"2024-09-13T07:04:57.322192+0000","last_undegraded":"2024-09-13T07:04:57.322192+0000","last_fullsized":"2024-09-13T07:04:57.322192+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:37:36.544267+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098858737000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6370,"num_read_kb":30920,"num_write":459,"num_write_kb":15662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":4960629,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"195'379","reported_seq":4118,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317289+0000","last_change":"2024-09-13T07:04:51.127712+0000","last_active":"2024-09-13T07:04:57.317289+0000","last_peered":"2024-09-13T07:04:57.317289+0000","last_clean":"2024-09-13T07:04:57.317289+0000","last_became_active":"2024-09-13T07:04:51.125903+0000","last_became_peered":"2024-09-13T07:04:51.125903+0000","last_unstale":"2024-09-13T07:04:57.317289+0000","last_undegraded":"2024-09-13T07:04:57.317289+0000","last_fullsized":"2024-09-13T07:04:57.317289+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:39.870655+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001432102,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4275,"num_read_kb":38815,"num_write":373,"num_write_kb":7930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"202'489","reported_seq":2580,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322555+0000","last_change":"2024-09-13T07:04:38.530429+0000","last_active":"2024-09-13T07:04:57.322555+0000","last_peered":"2024-09-13T07:04:57.322555+0000","last_clean":"2024-09-13T07:04:57.322555+0000","last_became_active":"2024-09-13T07:04:38.221697+0000","last_became_peered":"2024-09-13T07:04:38.221697+0000","last_unstale":"2024-09-13T07:04:57.322555+0000","last_undegraded":"2024-09-13T07:04:57.322555+0000","last_fullsized":"2024-09-13T07:04:57.322555+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:55.624774+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.073210949999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3758,"num_read_kb":13816,"num_write":504,"num_write_kb":5767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"202'500","reported_seq":5592,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321964+0000","last_change":"2024-09-13T07:04:38.656329+0000","last_active":"2024-09-13T07:04:57.321964+0000","last_peered":"2024-09-13T07:04:57.321964+0000","last_clean":"2024-09-13T07:04:57.321964+0000","last_became_active":"2024-09-13T07:04:38.225836+0000","last_became_peered":"2024-09-13T07:04:38.225836+0000","last_unstale":"2024-09-13T07:04:57.321964+0000","last_undegraded":"2024-09-13T07:04:57.321964+0000","last_fullsized":"2024-09-13T07:04:57.321964+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:40.102083+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.078226893000000006,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6179,"num_read_kb":24975,"num_write":551,"num_write_kb":10621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"198'381","reported_seq":6393,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322169+0000","last_change":"2024-09-13T07:04:48.130936+0000","last_active":"2024-09-13T07:04:57.322169+0000","last_peered":"2024-09-13T07:04:57.322169+0000","last_clean":"2024-09-13T07:04:57.322169+0000","last_became_active":"2024-09-13T07:04:48.128577+0000","last_became_peered":"2024-09-13T07:04:48.128577+0000","last_unstale":"2024-09-13T07:04:57.322169+0000","last_undegraded":"2024-09-13T07:04:57.322169+0000","last_fullsized":"2024-09-13T07:04:57.322169+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:54.041815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065506189999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6468,"num_read_kb":53020,"num_write":414,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":148740,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"198'1119","reported_seq":8612,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321787+0000","last_change":"2024-09-13T07:04:50.123312+0000","last_active":"2024-09-13T07:04:57.321787+0000","last_peered":"2024-09-13T07:04:57.321787+0000","last_clean":"2024-09-13T07:04:57.321787+0000","last_became_active":"2024-09-13T07:04:50.121731+0000","last_became_peered":"2024-09-13T07:04:50.121731+0000","last_unstale":"2024-09-13T07:04:57.321787+0000","last_undegraded":"2024-09-13T07:04:57.321787+0000","last_fullsized":"2024-09-13T07:04:57.321787+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":1119,"log_dups_size":0,"ondisk_log_size":1119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:03.606743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002988971,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":11548,"num_read_kb":34808,"num_write":1366,"num_write_kb":17528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":3976247,"num_keys_recovered":80,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"197'628","reported_seq":7696,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318581+0000","last_change":"2024-09-13T07:04:50.125951+0000","last_active":"2024-09-13T07:04:57.318581+0000","last_peered":"2024-09-13T07:04:57.318581+0000","last_clean":"2024-09-13T07:04:57.318581+0000","last_became_active":"2024-09-13T07:04:50.123234+0000","last_became_peered":"2024-09-13T07:04:50.123234+0000","last_unstale":"2024-09-13T07:04:57.318581+0000","last_undegraded":"2024-09-13T07:04:57.318581+0000","last_fullsized":"2024-09-13T07:04:57.318581+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:05.238196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001498135,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9140,"num_read_kb":20172,"num_write":660,"num_write_kb":17799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":1288675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"198'697","reported_seq":7807,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316898+0000","last_change":"2024-09-13T07:04:51.128278+0000","last_active":"2024-09-13T07:04:57.316898+0000","last_peered":"2024-09-13T07:04:57.316898+0000","last_clean":"2024-09-13T07:04:57.316898+0000","last_became_active":"2024-09-13T07:04:51.126595+0000","last_became_peered":"2024-09-13T07:04:51.126595+0000","last_unstale":"2024-09-13T07:04:57.316898+0000","last_undegraded":"2024-09-13T07:04:57.316898+0000","last_fullsized":"2024-09-13T07:04:57.316898+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:40.666326+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001501437,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8486,"num_read_kb":30532,"num_write":647,"num_write_kb":16391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":13265380,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"202'893","reported_seq":12728,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322291+0000","last_change":"2024-09-13T07:04:45.142416+0000","last_active":"2024-09-13T07:04:57.322291+0000","last_peered":"2024-09-13T07:04:57.322291+0000","last_clean":"2024-09-13T07:04:57.322291+0000","last_became_active":"2024-09-13T07:04:45.140362+0000","last_became_peered":"2024-09-13T07:04:45.140362+0000","last_unstale":"2024-09-13T07:04:57.322291+0000","last_undegraded":"2024-09-13T07:04:57.322291+0000","last_fullsized":"2024-09-13T07:04:57.322291+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:06:57.919834+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.082577740999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":15171,"num_read_kb":70673,"num_write":1118,"num_write_kb":35612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":8426052,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"198'847","reported_seq":12699,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.323110+0000","last_change":"2024-09-13T07:04:47.128907+0000","last_active":"2024-09-13T07:04:57.323110+0000","last_peered":"2024-09-13T07:04:57.323110+0000","last_clean":"2024-09-13T07:04:57.323110+0000","last_became_active":"2024-09-13T07:04:47.126092+0000","last_became_peered":"2024-09-13T07:04:47.126092+0000","last_unstale":"2024-09-13T07:04:57.323110+0000","last_undegraded":"2024-09-13T07:04:57.323110+0000","last_fullsized":"2024-09-13T07:04:57.323110+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:17:07.217834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075556199999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":14283,"num_read_kb":24697,"num_write":1275,"num_write_kb":8273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1104561,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"198'452","reported_seq":2662,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800574+0000","last_change":"2024-09-13T07:04:57.327607+0000","last_active":"2024-09-13T07:04:58.800574+0000","last_peered":"2024-09-13T07:04:58.800574+0000","last_clean":"2024-09-13T07:04:58.800574+0000","last_became_active":"2024-09-13T07:04:57.321478+0000","last_became_peered":"2024-09-13T07:04:57.321478+0000","last_unstale":"2024-09-13T07:04:58.800574+0000","last_undegraded":"2024-09-13T07:04:58.800574+0000","last_fullsized":"2024-09-13T07:04:58.800574+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:54:35.402390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001895851,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2526,"num_read_kb":26928,"num_write":433,"num_write_kb":12456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"198'516","reported_seq":5743,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321735+0000","last_change":"2024-09-13T07:04:46.128420+0000","last_active":"2024-09-13T07:04:57.321735+0000","last_peered":"2024-09-13T07:04:57.321735+0000","last_clean":"2024-09-13T07:04:57.321735+0000","last_became_active":"2024-09-13T07:04:46.126170+0000","last_became_peered":"2024-09-13T07:04:46.126170+0000","last_unstale":"2024-09-13T07:04:57.321735+0000","last_undegraded":"2024-09-13T07:04:57.321735+0000","last_fullsized":"2024-09-13T07:04:57.321735+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:41.697482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074505329999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5002,"num_read_kb":31007,"num_write":502,"num_write_kb":34041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":6788059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357224+0000","last_change":"2024-09-13T07:04:43.266461+0000","last_active":"2024-09-13T07:04:57.357224+0000","last_peered":"2024-09-13T07:04:57.357224+0000","last_clean":"2024-09-13T07:04:57.357224+0000","last_became_active":"2024-09-13T07:04:43.266020+0000","last_became_peered":"2024-09-13T07:04:43.266020+0000","last_unstale":"2024-09-13T07:04:57.357224+0000","last_undegraded":"2024-09-13T07:04:57.357224+0000","last_fullsized":"2024-09-13T07:04:57.357224+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:26.181412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"199'459","reported_seq":3615,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322391+0000","last_change":"2024-09-13T07:04:49.126459+0000","last_active":"2024-09-13T07:04:57.322391+0000","last_peered":"2024-09-13T07:04:57.322391+0000","last_clean":"2024-09-13T07:04:57.322391+0000","last_became_active":"2024-09-13T07:04:49.125105+0000","last_became_peered":"2024-09-13T07:04:49.125105+0000","last_unstale":"2024-09-13T07:04:57.322391+0000","last_undegraded":"2024-09-13T07:04:57.322391+0000","last_fullsized":"2024-09-13T07:04:57.322391+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:44:15.141007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066392000000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2852,"num_read_kb":36914,"num_write":449,"num_write_kb":17878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4764805,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318965+0000","last_change":"2024-09-13T07:04:46.127776+0000","last_active":"2024-09-13T07:04:57.318965+0000","last_peered":"2024-09-13T07:04:57.318965+0000","last_clean":"2024-09-13T07:04:57.318965+0000","last_became_active":"2024-09-13T07:04:46.127497+0000","last_became_peered":"2024-09-13T07:04:46.127497+0000","last_unstale":"2024-09-13T07:04:57.318965+0000","last_undegraded":"2024-09-13T07:04:57.318965+0000","last_fullsized":"2024-09-13T07:04:57.318965+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:32.313060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"198'599","reported_seq":4770,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317513+0000","last_change":"2024-09-13T07:04:46.128247+0000","last_active":"2024-09-13T07:04:57.317513+0000","last_peered":"2024-09-13T07:04:57.317513+0000","last_clean":"2024-09-13T07:04:57.317513+0000","last_became_active":"2024-09-13T07:04:46.126668+0000","last_became_peered":"2024-09-13T07:04:46.126668+0000","last_unstale":"2024-09-13T07:04:57.317513+0000","last_undegraded":"2024-09-13T07:04:57.317513+0000","last_fullsized":"2024-09-13T07:04:57.317513+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:46:46.209389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001721534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5009,"num_read_kb":48072,"num_write":666,"num_write_kb":15004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":128514,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800745+0000","last_change":"2024-09-13T07:04:57.327360+0000","last_active":"2024-09-13T07:04:58.800745+0000","last_peered":"2024-09-13T07:04:58.800745+0000","last_clean":"2024-09-13T07:04:58.800745+0000","last_became_active":"2024-09-13T07:04:57.325955+0000","last_became_peered":"2024-09-13T07:04:57.325955+0000","last_unstale":"2024-09-13T07:04:58.800745+0000","last_undegraded":"2024-09-13T07:04:58.800745+0000","last_fullsized":"2024-09-13T07:04:58.800745+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:36:35.543692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"202'377","reported_seq":1798,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318846+0000","last_change":"2024-09-13T07:04:49.134895+0000","last_active":"2024-09-13T07:04:57.318846+0000","last_peered":"2024-09-13T07:04:57.318846+0000","last_clean":"2024-09-13T07:04:57.318846+0000","last_became_active":"2024-09-13T07:04:49.132150+0000","last_became_peered":"2024-09-13T07:04:49.132150+0000","last_unstale":"2024-09-13T07:04:57.318846+0000","last_undegraded":"2024-09-13T07:04:57.318846+0000","last_fullsized":"2024-09-13T07:04:57.318846+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:54:57.301919+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.088725134999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1176,"num_read_kb":23362,"num_write":393,"num_write_kb":13842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":2874205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318745+0000","last_change":"2024-09-13T07:04:46.129346+0000","last_active":"2024-09-13T07:04:57.318745+0000","last_peered":"2024-09-13T07:04:57.318745+0000","last_clean":"2024-09-13T07:04:57.318745+0000","last_became_active":"2024-09-13T07:04:46.129069+0000","last_became_peered":"2024-09-13T07:04:46.129069+0000","last_unstale":"2024-09-13T07:04:57.318745+0000","last_undegraded":"2024-09-13T07:04:57.318745+0000","last_fullsized":"2024-09-13T07:04:57.318745+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:36.800953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'708","reported_seq":5274,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322759+0000","last_change":"2024-09-13T07:04:38.491014+0000","last_active":"2024-09-13T07:04:57.322759+0000","last_peered":"2024-09-13T07:04:57.322759+0000","last_clean":"2024-09-13T07:04:57.322759+0000","last_became_active":"2024-09-13T07:04:38.218110+0000","last_became_peered":"2024-09-13T07:04:38.218110+0000","last_unstale":"2024-09-13T07:04:57.322759+0000","last_undegraded":"2024-09-13T07:04:57.322759+0000","last_fullsized":"2024-09-13T07:04:57.322759+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:23:34.445526+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.101155064,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6847,"num_read_kb":24040,"num_write":729,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"199'610","reported_seq":6642,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322366+0000","last_change":"2024-09-13T07:04:34.193235+0000","last_active":"2024-09-13T07:04:57.322366+0000","last_peered":"2024-09-13T07:04:57.322366+0000","last_clean":"2024-09-13T07:04:57.322366+0000","last_became_active":"2024-09-13T07:03:06.744590+0000","last_became_peered":"2024-09-13T07:03:06.744590+0000","last_unstale":"2024-09-13T07:04:57.322366+0000","last_undegraded":"2024-09-13T07:04:57.322366+0000","last_fullsized":"2024-09-13T07:04:57.322366+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:59:25.982967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006933257,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":6314,"num_read_kb":32619,"num_write":703,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":11860709,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":175,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322603+0000","last_change":"2024-09-13T07:04:46.408868+0000","last_active":"2024-09-13T07:04:57.322603+0000","last_peered":"2024-09-13T07:04:57.322603+0000","last_clean":"2024-09-13T07:04:57.322603+0000","last_became_active":"2024-09-13T07:04:46.125291+0000","last_became_peered":"2024-09-13T07:04:46.125291+0000","last_unstale":"2024-09-13T07:04:57.322603+0000","last_undegraded":"2024-09-13T07:04:57.322603+0000","last_fullsized":"2024-09-13T07:04:57.322603+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:50:37.881225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"197'615","reported_seq":14047,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318699+0000","last_change":"2024-09-13T07:04:50.127801+0000","last_active":"2024-09-13T07:04:57.318699+0000","last_peered":"2024-09-13T07:04:57.318699+0000","last_clean":"2024-09-13T07:04:57.318699+0000","last_became_active":"2024-09-13T07:04:50.122952+0000","last_became_peered":"2024-09-13T07:04:50.122952+0000","last_unstale":"2024-09-13T07:04:57.318699+0000","last_undegraded":"2024-09-13T07:04:57.318699+0000","last_fullsized":"2024-09-13T07:04:57.318699+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:19.383007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084358800000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13923,"num_read_kb":43420,"num_write":574,"num_write_kb":22169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":8978985,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.875181+0000","last_change":"2024-09-13T07:02:44.091196+0000","last_active":"2024-09-13T07:04:57.875181+0000","last_peered":"2024-09-13T07:04:57.875181+0000","last_clean":"2024-09-13T07:04:57.875181+0000","last_became_active":"2024-09-13T07:02:44.090715+0000","last_became_peered":"2024-09-13T07:02:44.090715+0000","last_unstale":"2024-09-13T07:04:57.875181+0000","last_undegraded":"2024-09-13T07:04:57.875181+0000","last_fullsized":"2024-09-13T07:04:57.875181+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:08.303433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"202'376","reported_seq":7117,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322070+0000","last_change":"2024-09-13T07:04:54.130710+0000","last_active":"2024-09-13T07:04:57.322070+0000","last_peered":"2024-09-13T07:04:57.322070+0000","last_clean":"2024-09-13T07:04:57.322070+0000","last_became_active":"2024-09-13T07:04:54.129673+0000","last_became_peered":"2024-09-13T07:04:54.129673+0000","last_unstale":"2024-09-13T07:04:57.322070+0000","last_undegraded":"2024-09-13T07:04:57.322070+0000","last_fullsized":"2024-09-13T07:04:57.322070+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:42.016575+0000","scrub_duration":0,"objects_trimmed":26,"snaptrim_duration":0.064131435000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6959,"num_read_kb":35900,"num_write":367,"num_write_kb":8191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":3448599,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800247+0000","last_change":"2024-09-13T07:04:57.320659+0000","last_active":"2024-09-13T07:04:58.800247+0000","last_peered":"2024-09-13T07:04:58.800247+0000","last_clean":"2024-09-13T07:04:58.800247+0000","last_became_active":"2024-09-13T07:04:57.320044+0000","last_became_peered":"2024-09-13T07:04:57.320044+0000","last_unstale":"2024-09-13T07:04:58.800247+0000","last_undegraded":"2024-09-13T07:04:58.800247+0000","last_fullsized":"2024-09-13T07:04:58.800247+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:31.189059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"205'511","reported_seq":4274,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317039+0000","last_change":"2024-09-13T07:04:51.127015+0000","last_active":"2024-09-13T07:04:57.317039+0000","last_peered":"2024-09-13T07:04:57.317039+0000","last_clean":"2024-09-13T07:04:57.317039+0000","last_became_active":"2024-09-13T07:04:51.124840+0000","last_became_peered":"2024-09-13T07:04:51.124840+0000","last_unstale":"2024-09-13T07:04:57.317039+0000","last_undegraded":"2024-09-13T07:04:57.317039+0000","last_fullsized":"2024-09-13T07:04:57.317039+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:38:08.467746+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.071371410999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6314,"num_read_kb":38281,"num_write":477,"num_write_kb":24168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":195,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321960+0000","last_change":"2024-09-13T07:04:38.219981+0000","last_active":"2024-09-13T07:04:57.321960+0000","last_peered":"2024-09-13T07:04:57.321960+0000","last_clean":"2024-09-13T07:04:57.321960+0000","last_became_active":"2024-09-13T07:04:38.218537+0000","last_became_peered":"2024-09-13T07:04:38.218537+0000","last_unstale":"2024-09-13T07:04:57.321960+0000","last_undegraded":"2024-09-13T07:04:57.321960+0000","last_fullsized":"2024-09-13T07:04:57.321960+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:49:58.271696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":343,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800461+0000","last_change":"2024-09-13T07:04:57.325982+0000","last_active":"2024-09-13T07:04:58.800461+0000","last_peered":"2024-09-13T07:04:58.800461+0000","last_clean":"2024-09-13T07:04:58.800461+0000","last_became_active":"2024-09-13T07:04:57.324203+0000","last_became_peered":"2024-09-13T07:04:57.324203+0000","last_unstale":"2024-09-13T07:04:58.800461+0000","last_undegraded":"2024-09-13T07:04:58.800461+0000","last_fullsized":"2024-09-13T07:04:58.800461+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:01.766156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"198'782","reported_seq":5552,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316797+0000","last_change":"2024-09-13T07:04:53.134367+0000","last_active":"2024-09-13T07:04:57.316797+0000","last_peered":"2024-09-13T07:04:57.316797+0000","last_clean":"2024-09-13T07:04:57.316797+0000","last_became_active":"2024-09-13T07:04:53.131823+0000","last_became_peered":"2024-09-13T07:04:53.131823+0000","last_unstale":"2024-09-13T07:04:57.316797+0000","last_undegraded":"2024-09-13T07:04:57.316797+0000","last_fullsized":"2024-09-13T07:04:57.316797+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:32:42.148081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032724400000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5361,"num_read_kb":39438,"num_write":976,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":861185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"192'432","reported_seq":9628,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322616+0000","last_change":"2024-09-13T07:04:34.192006+0000","last_active":"2024-09-13T07:04:57.322616+0000","last_peered":"2024-09-13T07:04:57.322616+0000","last_clean":"2024-09-13T07:04:57.322616+0000","last_became_active":"2024-09-13T07:02:47.511257+0000","last_became_peered":"2024-09-13T07:02:47.511257+0000","last_unstale":"2024-09-13T07:04:57.322616+0000","last_undegraded":"2024-09-13T07:04:57.322616+0000","last_fullsized":"2024-09-13T07:04:57.322616+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:04.038572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059111220000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9849,"num_read_kb":28359,"num_write":451,"num_write_kb":30112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5191019,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"198'996","reported_seq":4549,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357116+0000","last_change":"2024-09-13T07:04:34.657476+0000","last_active":"2024-09-13T07:04:57.357116+0000","last_peered":"2024-09-13T07:04:57.357116+0000","last_clean":"2024-09-13T07:04:57.357116+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:04:57.357116+0000","last_undegraded":"2024-09-13T07:04:57.357116+0000","last_fullsized":"2024-09-13T07:04:57.357116+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":996,"log_dups_size":0,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:11.537644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016753040000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7385,"num_read_kb":27871,"num_write":1272,"num_write_kb":19033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"198'599","reported_seq":7645,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316852+0000","last_change":"2024-09-13T07:04:51.128619+0000","last_active":"2024-09-13T07:04:57.316852+0000","last_peered":"2024-09-13T07:04:57.316852+0000","last_clean":"2024-09-13T07:04:57.316852+0000","last_became_active":"2024-09-13T07:04:51.126975+0000","last_became_peered":"2024-09-13T07:04:51.126975+0000","last_unstale":"2024-09-13T07:04:57.316852+0000","last_undegraded":"2024-09-13T07:04:57.316852+0000","last_fullsized":"2024-09-13T07:04:57.316852+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:29.230850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020303249999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9098,"num_read_kb":32181,"num_write":560,"num_write_kb":27201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":10029829,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"205'1700","reported_seq":13975,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322182+0000","last_change":"2024-09-13T07:04:41.334599+0000","last_active":"2024-09-13T07:04:57.322182+0000","last_peered":"2024-09-13T07:04:57.322182+0000","last_clean":"2024-09-13T07:04:57.322182+0000","last_became_active":"2024-09-13T07:04:41.239873+0000","last_became_peered":"2024-09-13T07:04:41.239873+0000","last_unstale":"2024-09-13T07:04:57.322182+0000","last_undegraded":"2024-09-13T07:04:57.322182+0000","last_fullsized":"2024-09-13T07:04:57.322182+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":1700,"log_dups_size":0,"ondisk_log_size":1700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:18.172446+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.091557893000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21059,"num_read_kb":55383,"num_write":2606,"num_write_kb":25105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"198'662","reported_seq":6936,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800373+0000","last_change":"2024-09-13T07:04:57.326646+0000","last_active":"2024-09-13T07:04:58.800373+0000","last_peered":"2024-09-13T07:04:58.800373+0000","last_clean":"2024-09-13T07:04:58.800373+0000","last_became_active":"2024-09-13T07:04:57.323373+0000","last_became_peered":"2024-09-13T07:04:57.323373+0000","last_unstale":"2024-09-13T07:04:58.800373+0000","last_undegraded":"2024-09-13T07:04:58.800373+0000","last_fullsized":"2024-09-13T07:04:58.800373+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:47:35.380836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002404799,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8026,"num_read_kb":31428,"num_write":564,"num_write_kb":31570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9245169,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"198'561","reported_seq":3683,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800239+0000","last_change":"2024-09-13T07:04:57.329473+0000","last_active":"2024-09-13T07:04:58.800239+0000","last_peered":"2024-09-13T07:04:58.800239+0000","last_clean":"2024-09-13T07:04:58.800239+0000","last_became_active":"2024-09-13T07:04:57.325924+0000","last_became_peered":"2024-09-13T07:04:57.325924+0000","last_unstale":"2024-09-13T07:04:58.800239+0000","last_undegraded":"2024-09-13T07:04:58.800239+0000","last_fullsized":"2024-09-13T07:04:58.800239+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:09.975816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086238349999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4232,"num_read_kb":25664,"num_write":732,"num_write_kb":10506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1357199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"198'909","reported_seq":3819,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800888+0000","last_change":"2024-09-13T07:04:57.330501+0000","last_active":"2024-09-13T07:04:58.800888+0000","last_peered":"2024-09-13T07:04:58.800888+0000","last_clean":"2024-09-13T07:04:58.800888+0000","last_became_active":"2024-09-13T07:04:57.326822+0000","last_became_peered":"2024-09-13T07:04:57.326822+0000","last_unstale":"2024-09-13T07:04:58.800888+0000","last_undegraded":"2024-09-13T07:04:58.800888+0000","last_fullsized":"2024-09-13T07:04:58.800888+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:47:41.007573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075911479999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5743,"num_read_kb":24275,"num_write":1279,"num_write_kb":27074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":68557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"198'419","reported_seq":2601,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357197+0000","last_change":"2024-09-13T07:04:34.657069+0000","last_active":"2024-09-13T07:04:57.357197+0000","last_peered":"2024-09-13T07:04:57.357197+0000","last_clean":"2024-09-13T07:04:57.357197+0000","last_became_active":"2024-09-13T07:03:06.743141+0000","last_became_peered":"2024-09-13T07:03:06.743141+0000","last_unstale":"2024-09-13T07:04:57.357197+0000","last_undegraded":"2024-09-13T07:04:57.357197+0000","last_fullsized":"2024-09-13T07:04:57.357197+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:03:42.072271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012751349999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2106,"num_read_kb":12966,"num_write":446,"num_write_kb":14235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":10481695,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"198'592","reported_seq":8333,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800670+0000","last_change":"2024-09-13T07:04:57.330363+0000","last_active":"2024-09-13T07:04:58.800670+0000","last_peered":"2024-09-13T07:04:58.800670+0000","last_clean":"2024-09-13T07:04:58.800670+0000","last_became_active":"2024-09-13T07:04:57.325398+0000","last_became_peered":"2024-09-13T07:04:57.325398+0000","last_unstale":"2024-09-13T07:04:58.800670+0000","last_undegraded":"2024-09-13T07:04:58.800670+0000","last_fullsized":"2024-09-13T07:04:58.800670+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:57.680683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046356670000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8997,"num_read_kb":22165,"num_write":633,"num_write_kb":13453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":102776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"198'927","reported_seq":9117,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322084+0000","last_change":"2024-09-13T07:04:47.132559+0000","last_active":"2024-09-13T07:04:57.322084+0000","last_peered":"2024-09-13T07:04:57.322084+0000","last_clean":"2024-09-13T07:04:57.322084+0000","last_became_active":"2024-09-13T07:04:47.130438+0000","last_became_peered":"2024-09-13T07:04:47.130438+0000","last_unstale":"2024-09-13T07:04:57.322084+0000","last_undegraded":"2024-09-13T07:04:57.322084+0000","last_fullsized":"2024-09-13T07:04:57.322084+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:47:15.123470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076933,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9860,"num_read_kb":45806,"num_write":1253,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":5924852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"199'588","reported_seq":4689,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800492+0000","last_change":"2024-09-13T07:04:57.329853+0000","last_active":"2024-09-13T07:04:58.800492+0000","last_peered":"2024-09-13T07:04:58.800492+0000","last_clean":"2024-09-13T07:04:58.800492+0000","last_became_active":"2024-09-13T07:04:57.325425+0000","last_became_peered":"2024-09-13T07:04:57.325425+0000","last_unstale":"2024-09-13T07:04:58.800492+0000","last_undegraded":"2024-09-13T07:04:58.800492+0000","last_fullsized":"2024-09-13T07:04:58.800492+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:05.096848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051571560000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5130,"num_read_kb":30367,"num_write":654,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":12828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":408,"num_object_clones":0,"num_object_copies":816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":408,"num_whiteouts":404,"num_read":239446,"num_read_kb":1058874,"num_write":24086,"num_write_kb":567966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5260,"num_bytes_recovered":115290013,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},"log_size":20738,"ondisk_log_size":20738,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78604,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293236,"statfs":{"total":96636764160,"available":96556273664,"internally_reserved":0,"allocated":3276800,"data_stored":1887770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77199819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71399999999999997,"15min":0.70099999999999996},"min":{"1min":0.40500000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1870000000000001,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":0.88200000000000001,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":12.583,"5min":12.583,"15min":12.583},"last":0.80700000000000005}]},{"osd":1,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76000000000000001,"15min":0.73399999999999999},"min":{"1min":0.41299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.552,"5min":2.016,"15min":2.016},"last":0.871},{"interface":"front","average":{"1min":1.5089999999999999,"5min":0.99199999999999999,"15min":0.92800000000000005},"min":{"1min":0.56599999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":12.481,"5min":12.481,"15min":12.481},"last":0.96399999999999997}]},{"osd":2,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.88900000000000001,"15min":0.871},"min":{"1min":0.46200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85699999999999998,"15min":0.873},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.2330000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.91200000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2509999999999999,"5min":2.379,"15min":2.379},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.92700000000000005,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6220000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0489999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.048,"15min":0.96999999999999997},"min":{"1min":0.46300000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":12.622,"5min":12.622,"15min":12.622},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.98799999999999999,"15min":1.0009999999999999},"min":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2729999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.127}]},{"osd":5,"last update":"Fri Sep 13 07:03:03 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0129999999999999,"15min":1.0309999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.427,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.024,"15min":1.038},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3069999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.0880000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.198,"15min":1.121},"min":{"1min":0.64700000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":12.555999999999999,"5min":12.555999999999999,"15min":12.555999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.099,"15min":1.1020000000000001},"min":{"1min":0.72499999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.1890000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33740,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338100,"statfs":{"total":96636764160,"available":96602214400,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.84299999999999997,"15min":0.875},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.262,"5min":1.538,"15min":1.538},"last":0.871}]},{"osd":1,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.327,"5min":1.484,"15min":1.484},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.83799999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78200000000000003,"15min":0.81200000000000006},"min":{"1min":0.51800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.96899999999999997,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.88800000000000001,"15min":0.93000000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2290000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.95499999999999996}]},{"osd":3,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.92200000000000004,"15min":0.94899999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3799999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.94699999999999995,"15min":0.94599999999999995},"min":{"1min":0.55000000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.98099999999999998,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":1.036}]},{"osd":4,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88800000000000001,"15min":0.89300000000000002},"min":{"1min":0.47499999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1539999999999999,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.98699999999999999,"15min":1.0189999999999999},"min":{"1min":0.45100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.29,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.141,"5min":1.579,"15min":1.579},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0349999999999999,"15min":1.0549999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3420000000000001,"5min":1.857,"15min":1.857},"last":1.091}]},{"osd":7,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.056,"15min":1.1040000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3080000000000001,"5min":1.607,"15min":1.607},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.0429999999999999,"15min":1.093},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2450000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.98299999999999998}]}]},{"osd":5,"up_from":201,"seq":863288426503,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35732,"kb_used_data":2628,"kb_used_omap":35,"kb_used_meta":33052,"kb_avail":94336108,"statfs":{"total":96636764160,"available":96600174592,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36852,"internal_metadata":33845260},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69840,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302000,"statfs":{"total":96636764160,"available":96565248000,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.40600000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.99099999999999999,"15min":0.96299999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.1479999999999999,"5min":8.2089999999999996,"15min":8.2089999999999996},"last":0.95399999999999996}]},{"osd":1,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.93100000000000005,"15min":0.81999999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0549999999999999,"5min":10.932,"15min":10.932},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.84999999999999998,"15min":0.873},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.101,"5min":2.101,"15min":2.101},"last":1.123}]},{"osd":2,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":0.91400000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":5.9359999999999999,"5min":12.968999999999999,"15min":12.968999999999999},"last":1.0760000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.89200000000000002,"15min":0.91700000000000004},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.26},{"interface":"front","average":{"1min":0.754,"5min":0.80700000000000005,"15min":0.82099999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97899999999999998,"5min":1.159,"15min":1.159},"last":0.84099999999999997}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":0.95499999999999996,"15min":0.873},"min":{"1min":0.56899999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":8.2550000000000008,"5min":8.2550000000000008,"15min":8.2550000000000008},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.198,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.1519999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.83899999999999997,"15min":0.86499999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.123,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0960000000000001,"15min":1.03},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1240000000000001,"5min":10.885,"15min":10.885},"last":1.224}]},{"osd":7,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.92500000000000004,"15min":0.93200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.92800000000000005,"15min":0.95899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1830000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.3129999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51788,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320052,"statfs":{"total":96636764160,"available":96583733248,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.34399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.161,"5min":1.581,"15min":1.581},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.38900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.0049999999999999}]},{"osd":1,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.72699999999999998},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.143,"5min":1.577,"15min":1.577},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.073,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.872}]},{"osd":2,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.026,"5min":1.145,"15min":1.145},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2430000000000001,"5min":1.375,"15min":1.375},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92100000000000004,"15min":0.93400000000000005},"min":{"1min":0.57199999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.466,"5min":1.488,"15min":1.488},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.94299999999999995,"15min":0.95699999999999996},"min":{"1min":0.64600000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3340000000000001,"5min":1.512,"15min":1.512},"last":0.97899999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3149999999999999,"5min":1.369,"15min":1.369},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.002,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4299999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.1359999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.008,"15min":1.022},"min":{"1min":0.63800000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.04},{"interface":"front","average":{"1min":0.998,"5min":1.0169999999999999,"15min":1.0249999999999999},"min":{"1min":0.69499999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.0720000000000001}]},{"osd":7,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.0960000000000001,"15min":1.0920000000000001},"min":{"1min":0.73999999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7549999999999999,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.0109999999999999,"15min":0.99199999999999999},"min":{"1min":0.68100000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.1919999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33748,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338092,"statfs":{"total":96636764160,"available":96602206208,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.91600000000000004,"15min":1.018},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2070000000000001,"5min":6.867,"15min":6.867},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.66700000000000004},"min":{"1min":0.45000000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":1.0009999999999999,"5min":1.234,"15min":1.234},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79600000000000004,"15min":0.79000000000000004},"min":{"1min":0.46100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.306,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93200000000000005,"15min":0.94399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.016,"5min":0.999,"15min":1.006},"min":{"1min":0.70799999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.631,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.88}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.97099999999999997,"15min":0.96899999999999997},"min":{"1min":0.58299999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1,"15min":1.004},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.4670000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.018,"15min":1.0109999999999999},"min":{"1min":0.68899999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4690000000000001,"5min":2.109,"15min":2.109},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.03,"5min":1.0680000000000001,"15min":1.0760000000000001},"min":{"1min":0.61499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4330000000000001,"5min":1.748,"15min":1.748},"last":0.746}]},{"osd":7,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0369999999999999,"15min":1.0349999999999999},"min":{"1min":0.745,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.036,"15min":1.038},"min":{"1min":0.627,"5min":0.505,"15min":0.505},"max":{"1min":1.4610000000000001,"5min":1.7,"15min":1.7},"last":0.93400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29648,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342192,"statfs":{"total":96636764160,"available":96606404608,"internally_reserved":0,"allocated":2691072,"data_stored":1296855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.34499999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.0960000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65500000000000003,"15min":0.64500000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.92300000000000004}]},{"osd":2,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.47599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.70599999999999996,"15min":0.68600000000000005},"min":{"1min":0.41599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.79300000000000004}]},{"osd":3,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.751,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.157,"5min":1.171,"15min":1.171},"last":1.034},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.73499999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.0089999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90500000000000003,"15min":0.89700000000000002},"min":{"1min":0.53500000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.474,"5min":1.925,"15min":1.925},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.008,"5min":1.1599999999999999,"15min":1.0049999999999999},"min":{"1min":0.69199999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.319,"5min":13.701000000000001,"15min":13.701000000000001},"last":1.119}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.006,"15min":1.01},"min":{"1min":0.66400000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3400000000000001,"5min":1.48,"15min":1.48},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.98899999999999999,"15min":0.995},"min":{"1min":0.624,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.409,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.1919999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0129999999999999,"15min":1.014},"min":{"1min":0.56200000000000006,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2410000000000001,"5min":1.575,"15min":1.575},"last":1.258},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0509999999999999,"15min":1.046},"min":{"1min":0.66000000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.544}]},{"osd":7,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.633,"15min":1.282},"min":{"1min":0.73399999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.73,"5min":59.652000000000001,"15min":59.652000000000001},"last":1.415},{"interface":"front","average":{"1min":1.028,"5min":1.589,"15min":1.2390000000000001},"min":{"1min":0.64400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.423,"5min":59.679000000000002,"15min":59.679000000000002},"last":1.153}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38416,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333424,"statfs":{"total":96636764160,"available":96597426176,"internally_reserved":0,"allocated":3276800,"data_stored":1887185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.30199999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.548,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.33700000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.629}]},{"osd":2,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.35299999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.1399999999999999,"5min":1.629,"15min":1.629},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78900000000000003,"15min":0.80100000000000005},"min":{"1min":0.439,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2889999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.89100000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.038,"5min":1.635,"15min":1.635},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4710000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.86799999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.52800000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93300000000000005,"15min":0.94899999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3360000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.1020000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.02,"15min":1.034},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1870000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.873,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.254,"5min":1.724,"15min":1.724},"last":1.1180000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.0209999999999999,"15min":1.0429999999999999},"min":{"1min":0.50700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5469999999999999,"5min":1.552,"15min":1.552},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0509999999999999,"15min":1.077},"min":{"1min":0.55400000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.5780000000000001,"5min":2.024,"15min":2.024},"last":1.024}]},{"osd":7,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.079,"15min":1.093},"min":{"1min":0.59599999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5389999999999999,"5min":2,"15min":2},"last":1.073},{"interface":"front","average":{"1min":0.999,"5min":1.0900000000000001,"15min":1.1279999999999999},"min":{"1min":0.69099999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.389,"5min":1.744,"15min":1.744},"last":1.0029999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:05:03.102 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:05:03.388 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:05:03.389 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:05:03.400 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-13T07:05:02.663768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":412,"num_object_clones":0,"num_object_copies":824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":412,"num_whiteouts":404,"num_read":239562,"num_read_kb":1058974,"num_write":24247,"num_write_kb":570384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5266,"num_bytes_recovered":116470768,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20858,"ondisk_log_size":20858,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":371516,"kb_used_data":22156,"kb_used_omap":46,"kb_used_meta":349201,"kb_avail":754603204,"statfs":{"total":773094113280,"available":772713680896,"internally_reserved":0,"allocated":22687744,"data_stored":11560198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48002,"internal_metadata":357581950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-672,"num_bytes_recovered":-590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.366275"},"pg_stats":[{"pgid":"3.1f","version":"202'415","reported_seq":6819,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322192+0000","last_change":"2024-09-13T07:04:48.134119+0000","last_active":"2024-09-13T07:04:57.322192+0000","last_peered":"2024-09-13T07:04:57.322192+0000","last_clean":"2024-09-13T07:04:57.322192+0000","last_became_active":"2024-09-13T07:04:48.131331+0000","last_became_peered":"2024-09-13T07:04:48.131331+0000","last_unstale":"2024-09-13T07:04:57.322192+0000","last_undegraded":"2024-09-13T07:04:57.322192+0000","last_fullsized":"2024-09-13T07:04:57.322192+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:37:36.544267+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098858737000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6370,"num_read_kb":30920,"num_write":459,"num_write_kb":15662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":4960629,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"195'379","reported_seq":4118,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317289+0000","last_change":"2024-09-13T07:04:51.127712+0000","last_active":"2024-09-13T07:04:57.317289+0000","last_peered":"2024-09-13T07:04:57.317289+0000","last_clean":"2024-09-13T07:04:57.317289+0000","last_became_active":"2024-09-13T07:04:51.125903+0000","last_became_peered":"2024-09-13T07:04:51.125903+0000","last_unstale":"2024-09-13T07:04:57.317289+0000","last_undegraded":"2024-09-13T07:04:57.317289+0000","last_fullsized":"2024-09-13T07:04:57.317289+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:39.870655+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001432102,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4275,"num_read_kb":38815,"num_write":373,"num_write_kb":7930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"202'489","reported_seq":2580,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322555+0000","last_change":"2024-09-13T07:04:38.530429+0000","last_active":"2024-09-13T07:04:57.322555+0000","last_peered":"2024-09-13T07:04:57.322555+0000","last_clean":"2024-09-13T07:04:57.322555+0000","last_became_active":"2024-09-13T07:04:38.221697+0000","last_became_peered":"2024-09-13T07:04:38.221697+0000","last_unstale":"2024-09-13T07:04:57.322555+0000","last_undegraded":"2024-09-13T07:04:57.322555+0000","last_fullsized":"2024-09-13T07:04:57.322555+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:55.624774+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.073210949999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3758,"num_read_kb":13816,"num_write":504,"num_write_kb":5767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"202'500","reported_seq":5592,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321964+0000","last_change":"2024-09-13T07:04:38.656329+0000","last_active":"2024-09-13T07:04:57.321964+0000","last_peered":"2024-09-13T07:04:57.321964+0000","last_clean":"2024-09-13T07:04:57.321964+0000","last_became_active":"2024-09-13T07:04:38.225836+0000","last_became_peered":"2024-09-13T07:04:38.225836+0000","last_unstale":"2024-09-13T07:04:57.321964+0000","last_undegraded":"2024-09-13T07:04:57.321964+0000","last_fullsized":"2024-09-13T07:04:57.321964+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:40.102083+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.078226893000000006,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6179,"num_read_kb":24975,"num_write":551,"num_write_kb":10621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"198'381","reported_seq":6393,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322169+0000","last_change":"2024-09-13T07:04:48.130936+0000","last_active":"2024-09-13T07:04:57.322169+0000","last_peered":"2024-09-13T07:04:57.322169+0000","last_clean":"2024-09-13T07:04:57.322169+0000","last_became_active":"2024-09-13T07:04:48.128577+0000","last_became_peered":"2024-09-13T07:04:48.128577+0000","last_unstale":"2024-09-13T07:04:57.322169+0000","last_undegraded":"2024-09-13T07:04:57.322169+0000","last_fullsized":"2024-09-13T07:04:57.322169+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:54.041815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065506189999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6468,"num_read_kb":53020,"num_write":414,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":148740,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"198'1119","reported_seq":8612,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321787+0000","last_change":"2024-09-13T07:04:50.123312+0000","last_active":"2024-09-13T07:04:57.321787+0000","last_peered":"2024-09-13T07:04:57.321787+0000","last_clean":"2024-09-13T07:04:57.321787+0000","last_became_active":"2024-09-13T07:04:50.121731+0000","last_became_peered":"2024-09-13T07:04:50.121731+0000","last_unstale":"2024-09-13T07:04:57.321787+0000","last_undegraded":"2024-09-13T07:04:57.321787+0000","last_fullsized":"2024-09-13T07:04:57.321787+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":1119,"log_dups_size":0,"ondisk_log_size":1119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:03.606743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002988971,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":11548,"num_read_kb":34808,"num_write":1366,"num_write_kb":17528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":3976247,"num_keys_recovered":80,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"197'628","reported_seq":7696,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318581+0000","last_change":"2024-09-13T07:04:50.125951+0000","last_active":"2024-09-13T07:04:57.318581+0000","last_peered":"2024-09-13T07:04:57.318581+0000","last_clean":"2024-09-13T07:04:57.318581+0000","last_became_active":"2024-09-13T07:04:50.123234+0000","last_became_peered":"2024-09-13T07:04:50.123234+0000","last_unstale":"2024-09-13T07:04:57.318581+0000","last_undegraded":"2024-09-13T07:04:57.318581+0000","last_fullsized":"2024-09-13T07:04:57.318581+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:05.238196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001498135,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9140,"num_read_kb":20172,"num_write":660,"num_write_kb":17799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":1288675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"198'697","reported_seq":7807,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316898+0000","last_change":"2024-09-13T07:04:51.128278+0000","last_active":"2024-09-13T07:04:57.316898+0000","last_peered":"2024-09-13T07:04:57.316898+0000","last_clean":"2024-09-13T07:04:57.316898+0000","last_became_active":"2024-09-13T07:04:51.126595+0000","last_became_peered":"2024-09-13T07:04:51.126595+0000","last_unstale":"2024-09-13T07:04:57.316898+0000","last_undegraded":"2024-09-13T07:04:57.316898+0000","last_fullsized":"2024-09-13T07:04:57.316898+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:40.666326+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001501437,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8486,"num_read_kb":30532,"num_write":647,"num_write_kb":16391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":13265380,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"202'893","reported_seq":12728,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322291+0000","last_change":"2024-09-13T07:04:45.142416+0000","last_active":"2024-09-13T07:04:57.322291+0000","last_peered":"2024-09-13T07:04:57.322291+0000","last_clean":"2024-09-13T07:04:57.322291+0000","last_became_active":"2024-09-13T07:04:45.140362+0000","last_became_peered":"2024-09-13T07:04:45.140362+0000","last_unstale":"2024-09-13T07:04:57.322291+0000","last_undegraded":"2024-09-13T07:04:57.322291+0000","last_fullsized":"2024-09-13T07:04:57.322291+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:06:57.919834+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.082577740999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":15171,"num_read_kb":70673,"num_write":1118,"num_write_kb":35612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":8426052,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"198'847","reported_seq":12699,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.323110+0000","last_change":"2024-09-13T07:04:47.128907+0000","last_active":"2024-09-13T07:04:57.323110+0000","last_peered":"2024-09-13T07:04:57.323110+0000","last_clean":"2024-09-13T07:04:57.323110+0000","last_became_active":"2024-09-13T07:04:47.126092+0000","last_became_peered":"2024-09-13T07:04:47.126092+0000","last_unstale":"2024-09-13T07:04:57.323110+0000","last_undegraded":"2024-09-13T07:04:57.323110+0000","last_fullsized":"2024-09-13T07:04:57.323110+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:17:07.217834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075556199999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":14283,"num_read_kb":24697,"num_write":1275,"num_write_kb":8273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1104561,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"198'452","reported_seq":2662,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800574+0000","last_change":"2024-09-13T07:04:57.327607+0000","last_active":"2024-09-13T07:04:58.800574+0000","last_peered":"2024-09-13T07:04:58.800574+0000","last_clean":"2024-09-13T07:04:58.800574+0000","last_became_active":"2024-09-13T07:04:57.321478+0000","last_became_peered":"2024-09-13T07:04:57.321478+0000","last_unstale":"2024-09-13T07:04:58.800574+0000","last_undegraded":"2024-09-13T07:04:58.800574+0000","last_fullsized":"2024-09-13T07:04:58.800574+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:54:35.402390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001895851,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2526,"num_read_kb":26928,"num_write":433,"num_write_kb":12456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"198'516","reported_seq":5743,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321735+0000","last_change":"2024-09-13T07:04:46.128420+0000","last_active":"2024-09-13T07:04:57.321735+0000","last_peered":"2024-09-13T07:04:57.321735+0000","last_clean":"2024-09-13T07:04:57.321735+0000","last_became_active":"2024-09-13T07:04:46.126170+0000","last_became_peered":"2024-09-13T07:04:46.126170+0000","last_unstale":"2024-09-13T07:04:57.321735+0000","last_undegraded":"2024-09-13T07:04:57.321735+0000","last_fullsized":"2024-09-13T07:04:57.321735+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:41.697482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074505329999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5002,"num_read_kb":31007,"num_write":502,"num_write_kb":34041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":6788059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357224+0000","last_change":"2024-09-13T07:04:43.266461+0000","last_active":"2024-09-13T07:04:57.357224+0000","last_peered":"2024-09-13T07:04:57.357224+0000","last_clean":"2024-09-13T07:04:57.357224+0000","last_became_active":"2024-09-13T07:04:43.266020+0000","last_became_peered":"2024-09-13T07:04:43.266020+0000","last_unstale":"2024-09-13T07:04:57.357224+0000","last_undegraded":"2024-09-13T07:04:57.357224+0000","last_fullsized":"2024-09-13T07:04:57.357224+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:26.181412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"199'459","reported_seq":3615,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322391+0000","last_change":"2024-09-13T07:04:49.126459+0000","last_active":"2024-09-13T07:04:57.322391+0000","last_peered":"2024-09-13T07:04:57.322391+0000","last_clean":"2024-09-13T07:04:57.322391+0000","last_became_active":"2024-09-13T07:04:49.125105+0000","last_became_peered":"2024-09-13T07:04:49.125105+0000","last_unstale":"2024-09-13T07:04:57.322391+0000","last_undegraded":"2024-09-13T07:04:57.322391+0000","last_fullsized":"2024-09-13T07:04:57.322391+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:44:15.141007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066392000000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2852,"num_read_kb":36914,"num_write":449,"num_write_kb":17878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4764805,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318965+0000","last_change":"2024-09-13T07:04:46.127776+0000","last_active":"2024-09-13T07:04:57.318965+0000","last_peered":"2024-09-13T07:04:57.318965+0000","last_clean":"2024-09-13T07:04:57.318965+0000","last_became_active":"2024-09-13T07:04:46.127497+0000","last_became_peered":"2024-09-13T07:04:46.127497+0000","last_unstale":"2024-09-13T07:04:57.318965+0000","last_undegraded":"2024-09-13T07:04:57.318965+0000","last_fullsized":"2024-09-13T07:04:57.318965+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:32.313060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"198'599","reported_seq":4770,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317513+0000","last_change":"2024-09-13T07:04:46.128247+0000","last_active":"2024-09-13T07:04:57.317513+0000","last_peered":"2024-09-13T07:04:57.317513+0000","last_clean":"2024-09-13T07:04:57.317513+0000","last_became_active":"2024-09-13T07:04:46.126668+0000","last_became_peered":"2024-09-13T07:04:46.126668+0000","last_unstale":"2024-09-13T07:04:57.317513+0000","last_undegraded":"2024-09-13T07:04:57.317513+0000","last_fullsized":"2024-09-13T07:04:57.317513+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:46:46.209389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001721534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5009,"num_read_kb":48072,"num_write":666,"num_write_kb":15004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":128514,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800745+0000","last_change":"2024-09-13T07:04:57.327360+0000","last_active":"2024-09-13T07:04:58.800745+0000","last_peered":"2024-09-13T07:04:58.800745+0000","last_clean":"2024-09-13T07:04:58.800745+0000","last_became_active":"2024-09-13T07:04:57.325955+0000","last_became_peered":"2024-09-13T07:04:57.325955+0000","last_unstale":"2024-09-13T07:04:58.800745+0000","last_undegraded":"2024-09-13T07:04:58.800745+0000","last_fullsized":"2024-09-13T07:04:58.800745+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:36:35.543692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"202'377","reported_seq":1798,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318846+0000","last_change":"2024-09-13T07:04:49.134895+0000","last_active":"2024-09-13T07:04:57.318846+0000","last_peered":"2024-09-13T07:04:57.318846+0000","last_clean":"2024-09-13T07:04:57.318846+0000","last_became_active":"2024-09-13T07:04:49.132150+0000","last_became_peered":"2024-09-13T07:04:49.132150+0000","last_unstale":"2024-09-13T07:04:57.318846+0000","last_undegraded":"2024-09-13T07:04:57.318846+0000","last_fullsized":"2024-09-13T07:04:57.318846+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:54:57.301919+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.088725134999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1176,"num_read_kb":23362,"num_write":393,"num_write_kb":13842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":2874205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318745+0000","last_change":"2024-09-13T07:04:46.129346+0000","last_active":"2024-09-13T07:04:57.318745+0000","last_peered":"2024-09-13T07:04:57.318745+0000","last_clean":"2024-09-13T07:04:57.318745+0000","last_became_active":"2024-09-13T07:04:46.129069+0000","last_became_peered":"2024-09-13T07:04:46.129069+0000","last_unstale":"2024-09-13T07:04:57.318745+0000","last_undegraded":"2024-09-13T07:04:57.318745+0000","last_fullsized":"2024-09-13T07:04:57.318745+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:36.800953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'708","reported_seq":5274,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322759+0000","last_change":"2024-09-13T07:04:38.491014+0000","last_active":"2024-09-13T07:04:57.322759+0000","last_peered":"2024-09-13T07:04:57.322759+0000","last_clean":"2024-09-13T07:04:57.322759+0000","last_became_active":"2024-09-13T07:04:38.218110+0000","last_became_peered":"2024-09-13T07:04:38.218110+0000","last_unstale":"2024-09-13T07:04:57.322759+0000","last_undegraded":"2024-09-13T07:04:57.322759+0000","last_fullsized":"2024-09-13T07:04:57.322759+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:23:34.445526+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.101155064,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6847,"num_read_kb":24040,"num_write":729,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"199'610","reported_seq":6642,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322366+0000","last_change":"2024-09-13T07:04:34.193235+0000","last_active":"2024-09-13T07:04:57.322366+0000","last_peered":"2024-09-13T07:04:57.322366+0000","last_clean":"2024-09-13T07:04:57.322366+0000","last_became_active":"2024-09-13T07:03:06.744590+0000","last_became_peered":"2024-09-13T07:03:06.744590+0000","last_unstale":"2024-09-13T07:04:57.322366+0000","last_undegraded":"2024-09-13T07:04:57.322366+0000","last_fullsized":"2024-09-13T07:04:57.322366+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:59:25.982967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006933257,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":6314,"num_read_kb":32619,"num_write":703,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":11860709,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":175,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322603+0000","last_change":"2024-09-13T07:04:46.408868+0000","last_active":"2024-09-13T07:04:57.322603+0000","last_peered":"2024-09-13T07:04:57.322603+0000","last_clean":"2024-09-13T07:04:57.322603+0000","last_became_active":"2024-09-13T07:04:46.125291+0000","last_became_peered":"2024-09-13T07:04:46.125291+0000","last_unstale":"2024-09-13T07:04:57.322603+0000","last_undegraded":"2024-09-13T07:04:57.322603+0000","last_fullsized":"2024-09-13T07:04:57.322603+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:50:37.881225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"197'615","reported_seq":14047,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318699+0000","last_change":"2024-09-13T07:04:50.127801+0000","last_active":"2024-09-13T07:04:57.318699+0000","last_peered":"2024-09-13T07:04:57.318699+0000","last_clean":"2024-09-13T07:04:57.318699+0000","last_became_active":"2024-09-13T07:04:50.122952+0000","last_became_peered":"2024-09-13T07:04:50.122952+0000","last_unstale":"2024-09-13T07:04:57.318699+0000","last_undegraded":"2024-09-13T07:04:57.318699+0000","last_fullsized":"2024-09-13T07:04:57.318699+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:19.383007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084358800000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13923,"num_read_kb":43420,"num_write":574,"num_write_kb":22169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":8978985,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.875181+0000","last_change":"2024-09-13T07:02:44.091196+0000","last_active":"2024-09-13T07:04:57.875181+0000","last_peered":"2024-09-13T07:04:57.875181+0000","last_clean":"2024-09-13T07:04:57.875181+0000","last_became_active":"2024-09-13T07:02:44.090715+0000","last_became_peered":"2024-09-13T07:02:44.090715+0000","last_unstale":"2024-09-13T07:04:57.875181+0000","last_undegraded":"2024-09-13T07:04:57.875181+0000","last_fullsized":"2024-09-13T07:04:57.875181+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:08.303433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"202'376","reported_seq":7117,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322070+0000","last_change":"2024-09-13T07:04:54.130710+0000","last_active":"2024-09-13T07:04:57.322070+0000","last_peered":"2024-09-13T07:04:57.322070+0000","last_clean":"2024-09-13T07:04:57.322070+0000","last_became_active":"2024-09-13T07:04:54.129673+0000","last_became_peered":"2024-09-13T07:04:54.129673+0000","last_unstale":"2024-09-13T07:04:57.322070+0000","last_undegraded":"2024-09-13T07:04:57.322070+0000","last_fullsized":"2024-09-13T07:04:57.322070+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:42.016575+0000","scrub_duration":0,"objects_trimmed":26,"snaptrim_duration":0.064131435000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6959,"num_read_kb":35900,"num_write":367,"num_write_kb":8191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":3448599,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800247+0000","last_change":"2024-09-13T07:04:57.320659+0000","last_active":"2024-09-13T07:04:58.800247+0000","last_peered":"2024-09-13T07:04:58.800247+0000","last_clean":"2024-09-13T07:04:58.800247+0000","last_became_active":"2024-09-13T07:04:57.320044+0000","last_became_peered":"2024-09-13T07:04:57.320044+0000","last_unstale":"2024-09-13T07:04:58.800247+0000","last_undegraded":"2024-09-13T07:04:58.800247+0000","last_fullsized":"2024-09-13T07:04:58.800247+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:31.189059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"205'511","reported_seq":4274,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317039+0000","last_change":"2024-09-13T07:04:51.127015+0000","last_active":"2024-09-13T07:04:57.317039+0000","last_peered":"2024-09-13T07:04:57.317039+0000","last_clean":"2024-09-13T07:04:57.317039+0000","last_became_active":"2024-09-13T07:04:51.124840+0000","last_became_peered":"2024-09-13T07:04:51.124840+0000","last_unstale":"2024-09-13T07:04:57.317039+0000","last_undegraded":"2024-09-13T07:04:57.317039+0000","last_fullsized":"2024-09-13T07:04:57.317039+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:38:08.467746+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.071371410999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6314,"num_read_kb":38281,"num_write":477,"num_write_kb":24168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":195,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321960+0000","last_change":"2024-09-13T07:04:38.219981+0000","last_active":"2024-09-13T07:04:57.321960+0000","last_peered":"2024-09-13T07:04:57.321960+0000","last_clean":"2024-09-13T07:04:57.321960+0000","last_became_active":"2024-09-13T07:04:38.218537+0000","last_became_peered":"2024-09-13T07:04:38.218537+0000","last_unstale":"2024-09-13T07:04:57.321960+0000","last_undegraded":"2024-09-13T07:04:57.321960+0000","last_fullsized":"2024-09-13T07:04:57.321960+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:49:58.271696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":343,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800461+0000","last_change":"2024-09-13T07:04:57.325982+0000","last_active":"2024-09-13T07:04:58.800461+0000","last_peered":"2024-09-13T07:04:58.800461+0000","last_clean":"2024-09-13T07:04:58.800461+0000","last_became_active":"2024-09-13T07:04:57.324203+0000","last_became_peered":"2024-09-13T07:04:57.324203+0000","last_unstale":"2024-09-13T07:04:58.800461+0000","last_undegraded":"2024-09-13T07:04:58.800461+0000","last_fullsized":"2024-09-13T07:04:58.800461+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:01.766156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"198'782","reported_seq":5552,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316797+0000","last_change":"2024-09-13T07:04:53.134367+0000","last_active":"2024-09-13T07:04:57.316797+0000","last_peered":"2024-09-13T07:04:57.316797+0000","last_clean":"2024-09-13T07:04:57.316797+0000","last_became_active":"2024-09-13T07:04:53.131823+0000","last_became_peered":"2024-09-13T07:04:53.131823+0000","last_unstale":"2024-09-13T07:04:57.316797+0000","last_undegraded":"2024-09-13T07:04:57.316797+0000","last_fullsized":"2024-09-13T07:04:57.316797+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:32:42.148081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032724400000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5361,"num_read_kb":39438,"num_write":976,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":861185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"192'432","reported_seq":9628,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322616+0000","last_change":"2024-09-13T07:04:34.192006+0000","last_active":"2024-09-13T07:04:57.322616+0000","last_peered":"2024-09-13T07:04:57.322616+0000","last_clean":"2024-09-13T07:04:57.322616+0000","last_became_active":"2024-09-13T07:02:47.511257+0000","last_became_peered":"2024-09-13T07:02:47.511257+0000","last_unstale":"2024-09-13T07:04:57.322616+0000","last_undegraded":"2024-09-13T07:04:57.322616+0000","last_fullsized":"2024-09-13T07:04:57.322616+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:04.038572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059111220000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9849,"num_read_kb":28359,"num_write":451,"num_write_kb":30112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5191019,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"198'996","reported_seq":4549,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357116+0000","last_change":"2024-09-13T07:04:34.657476+0000","last_active":"2024-09-13T07:04:57.357116+0000","last_peered":"2024-09-13T07:04:57.357116+0000","last_clean":"2024-09-13T07:04:57.357116+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:04:57.357116+0000","last_undegraded":"2024-09-13T07:04:57.357116+0000","last_fullsized":"2024-09-13T07:04:57.357116+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":996,"log_dups_size":0,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:11.537644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016753040000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7385,"num_read_kb":27871,"num_write":1272,"num_write_kb":19033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"198'599","reported_seq":7645,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316852+0000","last_change":"2024-09-13T07:04:51.128619+0000","last_active":"2024-09-13T07:04:57.316852+0000","last_peered":"2024-09-13T07:04:57.316852+0000","last_clean":"2024-09-13T07:04:57.316852+0000","last_became_active":"2024-09-13T07:04:51.126975+0000","last_became_peered":"2024-09-13T07:04:51.126975+0000","last_unstale":"2024-09-13T07:04:57.316852+0000","last_undegraded":"2024-09-13T07:04:57.316852+0000","last_fullsized":"2024-09-13T07:04:57.316852+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:29.230850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020303249999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9098,"num_read_kb":32181,"num_write":560,"num_write_kb":27201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":10029829,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"205'1700","reported_seq":13975,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322182+0000","last_change":"2024-09-13T07:04:41.334599+0000","last_active":"2024-09-13T07:04:57.322182+0000","last_peered":"2024-09-13T07:04:57.322182+0000","last_clean":"2024-09-13T07:04:57.322182+0000","last_became_active":"2024-09-13T07:04:41.239873+0000","last_became_peered":"2024-09-13T07:04:41.239873+0000","last_unstale":"2024-09-13T07:04:57.322182+0000","last_undegraded":"2024-09-13T07:04:57.322182+0000","last_fullsized":"2024-09-13T07:04:57.322182+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":1700,"log_dups_size":0,"ondisk_log_size":1700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:18.172446+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.091557893000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21059,"num_read_kb":55383,"num_write":2606,"num_write_kb":25105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"198'662","reported_seq":6936,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800373+0000","last_change":"2024-09-13T07:04:57.326646+0000","last_active":"2024-09-13T07:04:58.800373+0000","last_peered":"2024-09-13T07:04:58.800373+0000","last_clean":"2024-09-13T07:04:58.800373+0000","last_became_active":"2024-09-13T07:04:57.323373+0000","last_became_peered":"2024-09-13T07:04:57.323373+0000","last_unstale":"2024-09-13T07:04:58.800373+0000","last_undegraded":"2024-09-13T07:04:58.800373+0000","last_fullsized":"2024-09-13T07:04:58.800373+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:47:35.380836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002404799,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8026,"num_read_kb":31428,"num_write":564,"num_write_kb":31570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9245169,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"198'561","reported_seq":3683,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800239+0000","last_change":"2024-09-13T07:04:57.329473+0000","last_active":"2024-09-13T07:04:58.800239+0000","last_peered":"2024-09-13T07:04:58.800239+0000","last_clean":"2024-09-13T07:04:58.800239+0000","last_became_active":"2024-09-13T07:04:57.325924+0000","last_became_peered":"2024-09-13T07:04:57.325924+0000","last_unstale":"2024-09-13T07:04:58.800239+0000","last_undegraded":"2024-09-13T07:04:58.800239+0000","last_fullsized":"2024-09-13T07:04:58.800239+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:09.975816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086238349999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4232,"num_read_kb":25664,"num_write":732,"num_write_kb":10506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1357199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"198'909","reported_seq":3819,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800888+0000","last_change":"2024-09-13T07:04:57.330501+0000","last_active":"2024-09-13T07:04:58.800888+0000","last_peered":"2024-09-13T07:04:58.800888+0000","last_clean":"2024-09-13T07:04:58.800888+0000","last_became_active":"2024-09-13T07:04:57.326822+0000","last_became_peered":"2024-09-13T07:04:57.326822+0000","last_unstale":"2024-09-13T07:04:58.800888+0000","last_undegraded":"2024-09-13T07:04:58.800888+0000","last_fullsized":"2024-09-13T07:04:58.800888+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:47:41.007573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075911479999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5743,"num_read_kb":24275,"num_write":1279,"num_write_kb":27074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":68557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"198'419","reported_seq":2601,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357197+0000","last_change":"2024-09-13T07:04:34.657069+0000","last_active":"2024-09-13T07:04:57.357197+0000","last_peered":"2024-09-13T07:04:57.357197+0000","last_clean":"2024-09-13T07:04:57.357197+0000","last_became_active":"2024-09-13T07:03:06.743141+0000","last_became_peered":"2024-09-13T07:03:06.743141+0000","last_unstale":"2024-09-13T07:04:57.357197+0000","last_undegraded":"2024-09-13T07:04:57.357197+0000","last_fullsized":"2024-09-13T07:04:57.357197+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:03:42.072271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012751349999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2106,"num_read_kb":12966,"num_write":446,"num_write_kb":14235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":10481695,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"198'592","reported_seq":8333,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800670+0000","last_change":"2024-09-13T07:04:57.330363+0000","last_active":"2024-09-13T07:04:58.800670+0000","last_peered":"2024-09-13T07:04:58.800670+0000","last_clean":"2024-09-13T07:04:58.800670+0000","last_became_active":"2024-09-13T07:04:57.325398+0000","last_became_peered":"2024-09-13T07:04:57.325398+0000","last_unstale":"2024-09-13T07:04:58.800670+0000","last_undegraded":"2024-09-13T07:04:58.800670+0000","last_fullsized":"2024-09-13T07:04:58.800670+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:57.680683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046356670000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8997,"num_read_kb":22165,"num_write":633,"num_write_kb":13453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":102776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"198'927","reported_seq":9117,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322084+0000","last_change":"2024-09-13T07:04:47.132559+0000","last_active":"2024-09-13T07:04:57.322084+0000","last_peered":"2024-09-13T07:04:57.322084+0000","last_clean":"2024-09-13T07:04:57.322084+0000","last_became_active":"2024-09-13T07:04:47.130438+0000","last_became_peered":"2024-09-13T07:04:47.130438+0000","last_unstale":"2024-09-13T07:04:57.322084+0000","last_undegraded":"2024-09-13T07:04:57.322084+0000","last_fullsized":"2024-09-13T07:04:57.322084+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:47:15.123470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076933,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9860,"num_read_kb":45806,"num_write":1253,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":5924852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"199'588","reported_seq":4689,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800492+0000","last_change":"2024-09-13T07:04:57.329853+0000","last_active":"2024-09-13T07:04:58.800492+0000","last_peered":"2024-09-13T07:04:58.800492+0000","last_clean":"2024-09-13T07:04:58.800492+0000","last_became_active":"2024-09-13T07:04:57.325425+0000","last_became_peered":"2024-09-13T07:04:57.325425+0000","last_unstale":"2024-09-13T07:04:58.800492+0000","last_undegraded":"2024-09-13T07:04:58.800492+0000","last_fullsized":"2024-09-13T07:04:58.800492+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:05.096848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051571560000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5130,"num_read_kb":30367,"num_write":654,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":12828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":408,"num_object_clones":0,"num_object_copies":816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":408,"num_whiteouts":404,"num_read":239446,"num_read_kb":1058874,"num_write":24086,"num_write_kb":567966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5260,"num_bytes_recovered":115290013,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},"log_size":20738,"ondisk_log_size":20738,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78604,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293236,"statfs":{"total":96636764160,"available":96556273664,"internally_reserved":0,"allocated":3276800,"data_stored":1887770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77199819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71399999999999997,"15min":0.70099999999999996},"min":{"1min":0.40500000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1870000000000001,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":0.88200000000000001,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":12.583,"5min":12.583,"15min":12.583},"last":0.80700000000000005}]},{"osd":1,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76000000000000001,"15min":0.73399999999999999},"min":{"1min":0.41299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.552,"5min":2.016,"15min":2.016},"last":0.871},{"interface":"front","average":{"1min":1.5089999999999999,"5min":0.99199999999999999,"15min":0.92800000000000005},"min":{"1min":0.56599999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":12.481,"5min":12.481,"15min":12.481},"last":0.96399999999999997}]},{"osd":2,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.88900000000000001,"15min":0.871},"min":{"1min":0.46200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85699999999999998,"15min":0.873},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.2330000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.91200000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2509999999999999,"5min":2.379,"15min":2.379},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.92700000000000005,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6220000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0489999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.048,"15min":0.96999999999999997},"min":{"1min":0.46300000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":12.622,"5min":12.622,"15min":12.622},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.98799999999999999,"15min":1.0009999999999999},"min":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2729999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.127}]},{"osd":5,"last update":"Fri Sep 13 07:03:03 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0129999999999999,"15min":1.0309999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.427,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.024,"15min":1.038},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3069999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.0880000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.198,"15min":1.121},"min":{"1min":0.64700000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":12.555999999999999,"5min":12.555999999999999,"15min":12.555999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.099,"15min":1.1020000000000001},"min":{"1min":0.72499999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.1890000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33740,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338100,"statfs":{"total":96636764160,"available":96602214400,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.84299999999999997,"15min":0.875},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.262,"5min":1.538,"15min":1.538},"last":0.871}]},{"osd":1,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.327,"5min":1.484,"15min":1.484},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.83799999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78200000000000003,"15min":0.81200000000000006},"min":{"1min":0.51800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.96899999999999997,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.88800000000000001,"15min":0.93000000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2290000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.95499999999999996}]},{"osd":3,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.92200000000000004,"15min":0.94899999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3799999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.94699999999999995,"15min":0.94599999999999995},"min":{"1min":0.55000000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.98099999999999998,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":1.036}]},{"osd":4,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88800000000000001,"15min":0.89300000000000002},"min":{"1min":0.47499999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1539999999999999,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.98699999999999999,"15min":1.0189999999999999},"min":{"1min":0.45100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.29,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.141,"5min":1.579,"15min":1.579},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0349999999999999,"15min":1.0549999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3420000000000001,"5min":1.857,"15min":1.857},"last":1.091}]},{"osd":7,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.056,"15min":1.1040000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3080000000000001,"5min":1.607,"15min":1.607},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.0429999999999999,"15min":1.093},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2450000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.98299999999999998}]}]},{"osd":5,"up_from":201,"seq":863288426503,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35732,"kb_used_data":2628,"kb_used_omap":35,"kb_used_meta":33052,"kb_avail":94336108,"statfs":{"total":96636764160,"available":96600174592,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36852,"internal_metadata":33845260},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69840,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302000,"statfs":{"total":96636764160,"available":96565248000,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.40600000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.99099999999999999,"15min":0.96299999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.1479999999999999,"5min":8.2089999999999996,"15min":8.2089999999999996},"last":0.95399999999999996}]},{"osd":1,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.93100000000000005,"15min":0.81999999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0549999999999999,"5min":10.932,"15min":10.932},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.84999999999999998,"15min":0.873},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.101,"5min":2.101,"15min":2.101},"last":1.123}]},{"osd":2,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":0.91400000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":5.9359999999999999,"5min":12.968999999999999,"15min":12.968999999999999},"last":1.0760000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.89200000000000002,"15min":0.91700000000000004},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.26},{"interface":"front","average":{"1min":0.754,"5min":0.80700000000000005,"15min":0.82099999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97899999999999998,"5min":1.159,"15min":1.159},"last":0.84099999999999997}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":0.95499999999999996,"15min":0.873},"min":{"1min":0.56899999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":8.2550000000000008,"5min":8.2550000000000008,"15min":8.2550000000000008},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.198,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.1519999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.83899999999999997,"15min":0.86499999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.123,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0960000000000001,"15min":1.03},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1240000000000001,"5min":10.885,"15min":10.885},"last":1.224}]},{"osd":7,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.92500000000000004,"15min":0.93200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.92800000000000005,"15min":0.95899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1830000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.3129999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51788,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320052,"statfs":{"total":96636764160,"available":96583733248,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.34399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.161,"5min":1.581,"15min":1.581},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.38900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.0049999999999999}]},{"osd":1,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.72699999999999998},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.143,"5min":1.577,"15min":1.577},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.073,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.872}]},{"osd":2,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.026,"5min":1.145,"15min":1.145},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2430000000000001,"5min":1.375,"15min":1.375},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92100000000000004,"15min":0.93400000000000005},"min":{"1min":0.57199999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.466,"5min":1.488,"15min":1.488},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.94299999999999995,"15min":0.95699999999999996},"min":{"1min":0.64600000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3340000000000001,"5min":1.512,"15min":1.512},"last":0.97899999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3149999999999999,"5min":1.369,"15min":1.369},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.002,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4299999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.1359999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.008,"15min":1.022},"min":{"1min":0.63800000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.04},{"interface":"front","average":{"1min":0.998,"5min":1.0169999999999999,"15min":1.0249999999999999},"min":{"1min":0.69499999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.0720000000000001}]},{"osd":7,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.0960000000000001,"15min":1.0920000000000001},"min":{"1min":0.73999999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7549999999999999,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.0109999999999999,"15min":0.99199999999999999},"min":{"1min":0.68100000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.1919999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33748,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338092,"statfs":{"total":96636764160,"available":96602206208,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.91600000000000004,"15min":1.018},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2070000000000001,"5min":6.867,"15min":6.867},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.66700000000000004},"min":{"1min":0.45000000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":1.0009999999999999,"5min":1.234,"15min":1.234},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79600000000000004,"15min":0.79000000000000004},"min":{"1min":0.46100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.306,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93200000000000005,"15min":0.94399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.016,"5min":0.999,"15min":1.006},"min":{"1min":0.70799999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.631,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.88}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.97099999999999997,"15min":0.96899999999999997},"min":{"1min":0.58299999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1,"15min":1.004},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.4670000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.018,"15min":1.0109999999999999},"min":{"1min":0.68899999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4690000000000001,"5min":2.109,"15min":2.109},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.03,"5min":1.0680000000000001,"15min":1.0760000000000001},"min":{"1min":0.61499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4330000000000001,"5min":1.748,"15min":1.748},"last":0.746}]},{"osd":7,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0369999999999999,"15min":1.0349999999999999},"min":{"1min":0.745,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.036,"15min":1.038},"min":{"1min":0.627,"5min":0.505,"15min":0.505},"max":{"1min":1.4610000000000001,"5min":1.7,"15min":1.7},"last":0.93400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29648,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342192,"statfs":{"total":96636764160,"available":96606404608,"internally_reserved":0,"allocated":2691072,"data_stored":1296855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.34499999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.0960000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65500000000000003,"15min":0.64500000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.92300000000000004}]},{"osd":2,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.47599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.70599999999999996,"15min":0.68600000000000005},"min":{"1min":0.41599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.79300000000000004}]},{"osd":3,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.751,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.157,"5min":1.171,"15min":1.171},"last":1.034},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.73499999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.0089999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90500000000000003,"15min":0.89700000000000002},"min":{"1min":0.53500000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.474,"5min":1.925,"15min":1.925},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.008,"5min":1.1599999999999999,"15min":1.0049999999999999},"min":{"1min":0.69199999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.319,"5min":13.701000000000001,"15min":13.701000000000001},"last":1.119}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.006,"15min":1.01},"min":{"1min":0.66400000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3400000000000001,"5min":1.48,"15min":1.48},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.98899999999999999,"15min":0.995},"min":{"1min":0.624,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.409,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.1919999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0129999999999999,"15min":1.014},"min":{"1min":0.56200000000000006,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2410000000000001,"5min":1.575,"15min":1.575},"last":1.258},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0509999999999999,"15min":1.046},"min":{"1min":0.66000000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.544}]},{"osd":7,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.633,"15min":1.282},"min":{"1min":0.73399999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.73,"5min":59.652000000000001,"15min":59.652000000000001},"last":1.415},{"interface":"front","average":{"1min":1.028,"5min":1.589,"15min":1.2390000000000001},"min":{"1min":0.64400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.423,"5min":59.679000000000002,"15min":59.679000000000002},"last":1.153}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38416,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333424,"statfs":{"total":96636764160,"available":96597426176,"internally_reserved":0,"allocated":3276800,"data_stored":1887185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.30199999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.548,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.33700000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.629}]},{"osd":2,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.35299999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.1399999999999999,"5min":1.629,"15min":1.629},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78900000000000003,"15min":0.80100000000000005},"min":{"1min":0.439,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2889999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.89100000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.038,"5min":1.635,"15min":1.635},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4710000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.86799999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.52800000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93300000000000005,"15min":0.94899999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3360000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.1020000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.02,"15min":1.034},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1870000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.873,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.254,"5min":1.724,"15min":1.724},"last":1.1180000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.0209999999999999,"15min":1.0429999999999999},"min":{"1min":0.50700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5469999999999999,"5min":1.552,"15min":1.552},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0509999999999999,"15min":1.077},"min":{"1min":0.55400000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.5780000000000001,"5min":2.024,"15min":2.024},"last":1.024}]},{"osd":7,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.079,"15min":1.093},"min":{"1min":0.59599999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5389999999999999,"5min":2,"15min":2},"last":1.073},{"interface":"front","average":{"1min":0.999,"5min":1.0900000000000001,"15min":1.1279999999999999},"min":{"1min":0.69099999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.389,"5min":1.744,"15min":1.744},"last":1.0029999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:05:03.403 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T07:05:03.403 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-13T07:05:03.706 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T07:05:03.716 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T07:05:03.716 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:05:04.008 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:05:04.011 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:05:04.022 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-13T07:05:02.663768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":412,"num_object_clones":0,"num_object_copies":824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":412,"num_whiteouts":404,"num_read":239562,"num_read_kb":1058974,"num_write":24247,"num_write_kb":570384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5266,"num_bytes_recovered":116470768,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20858,"ondisk_log_size":20858,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":371516,"kb_used_data":22156,"kb_used_omap":46,"kb_used_meta":349201,"kb_avail":754603204,"statfs":{"total":773094113280,"available":772713680896,"internally_reserved":0,"allocated":22687744,"data_stored":11560198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48002,"internal_metadata":357581950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-672,"num_bytes_recovered":-590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.366275"},"pg_stats":[{"pgid":"3.1f","version":"202'415","reported_seq":6819,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322192+0000","last_change":"2024-09-13T07:04:48.134119+0000","last_active":"2024-09-13T07:04:57.322192+0000","last_peered":"2024-09-13T07:04:57.322192+0000","last_clean":"2024-09-13T07:04:57.322192+0000","last_became_active":"2024-09-13T07:04:48.131331+0000","last_became_peered":"2024-09-13T07:04:48.131331+0000","last_unstale":"2024-09-13T07:04:57.322192+0000","last_undegraded":"2024-09-13T07:04:57.322192+0000","last_fullsized":"2024-09-13T07:04:57.322192+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:37:36.544267+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098858737000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6370,"num_read_kb":30920,"num_write":459,"num_write_kb":15662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":4960629,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"195'379","reported_seq":4118,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317289+0000","last_change":"2024-09-13T07:04:51.127712+0000","last_active":"2024-09-13T07:04:57.317289+0000","last_peered":"2024-09-13T07:04:57.317289+0000","last_clean":"2024-09-13T07:04:57.317289+0000","last_became_active":"2024-09-13T07:04:51.125903+0000","last_became_peered":"2024-09-13T07:04:51.125903+0000","last_unstale":"2024-09-13T07:04:57.317289+0000","last_undegraded":"2024-09-13T07:04:57.317289+0000","last_fullsized":"2024-09-13T07:04:57.317289+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:39.870655+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001432102,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4275,"num_read_kb":38815,"num_write":373,"num_write_kb":7930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"202'489","reported_seq":2580,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322555+0000","last_change":"2024-09-13T07:04:38.530429+0000","last_active":"2024-09-13T07:04:57.322555+0000","last_peered":"2024-09-13T07:04:57.322555+0000","last_clean":"2024-09-13T07:04:57.322555+0000","last_became_active":"2024-09-13T07:04:38.221697+0000","last_became_peered":"2024-09-13T07:04:38.221697+0000","last_unstale":"2024-09-13T07:04:57.322555+0000","last_undegraded":"2024-09-13T07:04:57.322555+0000","last_fullsized":"2024-09-13T07:04:57.322555+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:55.624774+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.073210949999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3758,"num_read_kb":13816,"num_write":504,"num_write_kb":5767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"202'500","reported_seq":5592,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321964+0000","last_change":"2024-09-13T07:04:38.656329+0000","last_active":"2024-09-13T07:04:57.321964+0000","last_peered":"2024-09-13T07:04:57.321964+0000","last_clean":"2024-09-13T07:04:57.321964+0000","last_became_active":"2024-09-13T07:04:38.225836+0000","last_became_peered":"2024-09-13T07:04:38.225836+0000","last_unstale":"2024-09-13T07:04:57.321964+0000","last_undegraded":"2024-09-13T07:04:57.321964+0000","last_fullsized":"2024-09-13T07:04:57.321964+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:40.102083+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.078226893000000006,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6179,"num_read_kb":24975,"num_write":551,"num_write_kb":10621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"198'381","reported_seq":6393,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322169+0000","last_change":"2024-09-13T07:04:48.130936+0000","last_active":"2024-09-13T07:04:57.322169+0000","last_peered":"2024-09-13T07:04:57.322169+0000","last_clean":"2024-09-13T07:04:57.322169+0000","last_became_active":"2024-09-13T07:04:48.128577+0000","last_became_peered":"2024-09-13T07:04:48.128577+0000","last_unstale":"2024-09-13T07:04:57.322169+0000","last_undegraded":"2024-09-13T07:04:57.322169+0000","last_fullsized":"2024-09-13T07:04:57.322169+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:54.041815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065506189999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6468,"num_read_kb":53020,"num_write":414,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":148740,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"198'1119","reported_seq":8612,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321787+0000","last_change":"2024-09-13T07:04:50.123312+0000","last_active":"2024-09-13T07:04:57.321787+0000","last_peered":"2024-09-13T07:04:57.321787+0000","last_clean":"2024-09-13T07:04:57.321787+0000","last_became_active":"2024-09-13T07:04:50.121731+0000","last_became_peered":"2024-09-13T07:04:50.121731+0000","last_unstale":"2024-09-13T07:04:57.321787+0000","last_undegraded":"2024-09-13T07:04:57.321787+0000","last_fullsized":"2024-09-13T07:04:57.321787+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":1119,"log_dups_size":0,"ondisk_log_size":1119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:03.606743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002988971,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":11548,"num_read_kb":34808,"num_write":1366,"num_write_kb":17528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":3976247,"num_keys_recovered":80,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"197'628","reported_seq":7696,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318581+0000","last_change":"2024-09-13T07:04:50.125951+0000","last_active":"2024-09-13T07:04:57.318581+0000","last_peered":"2024-09-13T07:04:57.318581+0000","last_clean":"2024-09-13T07:04:57.318581+0000","last_became_active":"2024-09-13T07:04:50.123234+0000","last_became_peered":"2024-09-13T07:04:50.123234+0000","last_unstale":"2024-09-13T07:04:57.318581+0000","last_undegraded":"2024-09-13T07:04:57.318581+0000","last_fullsized":"2024-09-13T07:04:57.318581+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:05.238196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001498135,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9140,"num_read_kb":20172,"num_write":660,"num_write_kb":17799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":1288675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"198'697","reported_seq":7807,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316898+0000","last_change":"2024-09-13T07:04:51.128278+0000","last_active":"2024-09-13T07:04:57.316898+0000","last_peered":"2024-09-13T07:04:57.316898+0000","last_clean":"2024-09-13T07:04:57.316898+0000","last_became_active":"2024-09-13T07:04:51.126595+0000","last_became_peered":"2024-09-13T07:04:51.126595+0000","last_unstale":"2024-09-13T07:04:57.316898+0000","last_undegraded":"2024-09-13T07:04:57.316898+0000","last_fullsized":"2024-09-13T07:04:57.316898+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:40.666326+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001501437,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8486,"num_read_kb":30532,"num_write":647,"num_write_kb":16391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":13265380,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"202'893","reported_seq":12728,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322291+0000","last_change":"2024-09-13T07:04:45.142416+0000","last_active":"2024-09-13T07:04:57.322291+0000","last_peered":"2024-09-13T07:04:57.322291+0000","last_clean":"2024-09-13T07:04:57.322291+0000","last_became_active":"2024-09-13T07:04:45.140362+0000","last_became_peered":"2024-09-13T07:04:45.140362+0000","last_unstale":"2024-09-13T07:04:57.322291+0000","last_undegraded":"2024-09-13T07:04:57.322291+0000","last_fullsized":"2024-09-13T07:04:57.322291+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:06:57.919834+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.082577740999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":15171,"num_read_kb":70673,"num_write":1118,"num_write_kb":35612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":8426052,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"198'847","reported_seq":12699,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.323110+0000","last_change":"2024-09-13T07:04:47.128907+0000","last_active":"2024-09-13T07:04:57.323110+0000","last_peered":"2024-09-13T07:04:57.323110+0000","last_clean":"2024-09-13T07:04:57.323110+0000","last_became_active":"2024-09-13T07:04:47.126092+0000","last_became_peered":"2024-09-13T07:04:47.126092+0000","last_unstale":"2024-09-13T07:04:57.323110+0000","last_undegraded":"2024-09-13T07:04:57.323110+0000","last_fullsized":"2024-09-13T07:04:57.323110+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:17:07.217834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075556199999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":14283,"num_read_kb":24697,"num_write":1275,"num_write_kb":8273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1104561,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"198'452","reported_seq":2662,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800574+0000","last_change":"2024-09-13T07:04:57.327607+0000","last_active":"2024-09-13T07:04:58.800574+0000","last_peered":"2024-09-13T07:04:58.800574+0000","last_clean":"2024-09-13T07:04:58.800574+0000","last_became_active":"2024-09-13T07:04:57.321478+0000","last_became_peered":"2024-09-13T07:04:57.321478+0000","last_unstale":"2024-09-13T07:04:58.800574+0000","last_undegraded":"2024-09-13T07:04:58.800574+0000","last_fullsized":"2024-09-13T07:04:58.800574+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:54:35.402390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001895851,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2526,"num_read_kb":26928,"num_write":433,"num_write_kb":12456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"198'516","reported_seq":5743,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321735+0000","last_change":"2024-09-13T07:04:46.128420+0000","last_active":"2024-09-13T07:04:57.321735+0000","last_peered":"2024-09-13T07:04:57.321735+0000","last_clean":"2024-09-13T07:04:57.321735+0000","last_became_active":"2024-09-13T07:04:46.126170+0000","last_became_peered":"2024-09-13T07:04:46.126170+0000","last_unstale":"2024-09-13T07:04:57.321735+0000","last_undegraded":"2024-09-13T07:04:57.321735+0000","last_fullsized":"2024-09-13T07:04:57.321735+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:41.697482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074505329999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5002,"num_read_kb":31007,"num_write":502,"num_write_kb":34041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":6788059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357224+0000","last_change":"2024-09-13T07:04:43.266461+0000","last_active":"2024-09-13T07:04:57.357224+0000","last_peered":"2024-09-13T07:04:57.357224+0000","last_clean":"2024-09-13T07:04:57.357224+0000","last_became_active":"2024-09-13T07:04:43.266020+0000","last_became_peered":"2024-09-13T07:04:43.266020+0000","last_unstale":"2024-09-13T07:04:57.357224+0000","last_undegraded":"2024-09-13T07:04:57.357224+0000","last_fullsized":"2024-09-13T07:04:57.357224+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:26.181412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"199'459","reported_seq":3615,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322391+0000","last_change":"2024-09-13T07:04:49.126459+0000","last_active":"2024-09-13T07:04:57.322391+0000","last_peered":"2024-09-13T07:04:57.322391+0000","last_clean":"2024-09-13T07:04:57.322391+0000","last_became_active":"2024-09-13T07:04:49.125105+0000","last_became_peered":"2024-09-13T07:04:49.125105+0000","last_unstale":"2024-09-13T07:04:57.322391+0000","last_undegraded":"2024-09-13T07:04:57.322391+0000","last_fullsized":"2024-09-13T07:04:57.322391+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:44:15.141007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066392000000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2852,"num_read_kb":36914,"num_write":449,"num_write_kb":17878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4764805,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318965+0000","last_change":"2024-09-13T07:04:46.127776+0000","last_active":"2024-09-13T07:04:57.318965+0000","last_peered":"2024-09-13T07:04:57.318965+0000","last_clean":"2024-09-13T07:04:57.318965+0000","last_became_active":"2024-09-13T07:04:46.127497+0000","last_became_peered":"2024-09-13T07:04:46.127497+0000","last_unstale":"2024-09-13T07:04:57.318965+0000","last_undegraded":"2024-09-13T07:04:57.318965+0000","last_fullsized":"2024-09-13T07:04:57.318965+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:32.313060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"198'599","reported_seq":4770,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317513+0000","last_change":"2024-09-13T07:04:46.128247+0000","last_active":"2024-09-13T07:04:57.317513+0000","last_peered":"2024-09-13T07:04:57.317513+0000","last_clean":"2024-09-13T07:04:57.317513+0000","last_became_active":"2024-09-13T07:04:46.126668+0000","last_became_peered":"2024-09-13T07:04:46.126668+0000","last_unstale":"2024-09-13T07:04:57.317513+0000","last_undegraded":"2024-09-13T07:04:57.317513+0000","last_fullsized":"2024-09-13T07:04:57.317513+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:46:46.209389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001721534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5009,"num_read_kb":48072,"num_write":666,"num_write_kb":15004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":128514,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800745+0000","last_change":"2024-09-13T07:04:57.327360+0000","last_active":"2024-09-13T07:04:58.800745+0000","last_peered":"2024-09-13T07:04:58.800745+0000","last_clean":"2024-09-13T07:04:58.800745+0000","last_became_active":"2024-09-13T07:04:57.325955+0000","last_became_peered":"2024-09-13T07:04:57.325955+0000","last_unstale":"2024-09-13T07:04:58.800745+0000","last_undegraded":"2024-09-13T07:04:58.800745+0000","last_fullsized":"2024-09-13T07:04:58.800745+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:36:35.543692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"202'377","reported_seq":1798,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318846+0000","last_change":"2024-09-13T07:04:49.134895+0000","last_active":"2024-09-13T07:04:57.318846+0000","last_peered":"2024-09-13T07:04:57.318846+0000","last_clean":"2024-09-13T07:04:57.318846+0000","last_became_active":"2024-09-13T07:04:49.132150+0000","last_became_peered":"2024-09-13T07:04:49.132150+0000","last_unstale":"2024-09-13T07:04:57.318846+0000","last_undegraded":"2024-09-13T07:04:57.318846+0000","last_fullsized":"2024-09-13T07:04:57.318846+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:54:57.301919+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.088725134999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1176,"num_read_kb":23362,"num_write":393,"num_write_kb":13842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":2874205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318745+0000","last_change":"2024-09-13T07:04:46.129346+0000","last_active":"2024-09-13T07:04:57.318745+0000","last_peered":"2024-09-13T07:04:57.318745+0000","last_clean":"2024-09-13T07:04:57.318745+0000","last_became_active":"2024-09-13T07:04:46.129069+0000","last_became_peered":"2024-09-13T07:04:46.129069+0000","last_unstale":"2024-09-13T07:04:57.318745+0000","last_undegraded":"2024-09-13T07:04:57.318745+0000","last_fullsized":"2024-09-13T07:04:57.318745+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:36.800953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'708","reported_seq":5274,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322759+0000","last_change":"2024-09-13T07:04:38.491014+0000","last_active":"2024-09-13T07:04:57.322759+0000","last_peered":"2024-09-13T07:04:57.322759+0000","last_clean":"2024-09-13T07:04:57.322759+0000","last_became_active":"2024-09-13T07:04:38.218110+0000","last_became_peered":"2024-09-13T07:04:38.218110+0000","last_unstale":"2024-09-13T07:04:57.322759+0000","last_undegraded":"2024-09-13T07:04:57.322759+0000","last_fullsized":"2024-09-13T07:04:57.322759+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:23:34.445526+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.101155064,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6847,"num_read_kb":24040,"num_write":729,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"199'610","reported_seq":6642,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322366+0000","last_change":"2024-09-13T07:04:34.193235+0000","last_active":"2024-09-13T07:04:57.322366+0000","last_peered":"2024-09-13T07:04:57.322366+0000","last_clean":"2024-09-13T07:04:57.322366+0000","last_became_active":"2024-09-13T07:03:06.744590+0000","last_became_peered":"2024-09-13T07:03:06.744590+0000","last_unstale":"2024-09-13T07:04:57.322366+0000","last_undegraded":"2024-09-13T07:04:57.322366+0000","last_fullsized":"2024-09-13T07:04:57.322366+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:59:25.982967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006933257,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":6314,"num_read_kb":32619,"num_write":703,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":11860709,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":175,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322603+0000","last_change":"2024-09-13T07:04:46.408868+0000","last_active":"2024-09-13T07:04:57.322603+0000","last_peered":"2024-09-13T07:04:57.322603+0000","last_clean":"2024-09-13T07:04:57.322603+0000","last_became_active":"2024-09-13T07:04:46.125291+0000","last_became_peered":"2024-09-13T07:04:46.125291+0000","last_unstale":"2024-09-13T07:04:57.322603+0000","last_undegraded":"2024-09-13T07:04:57.322603+0000","last_fullsized":"2024-09-13T07:04:57.322603+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:50:37.881225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"197'615","reported_seq":14047,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318699+0000","last_change":"2024-09-13T07:04:50.127801+0000","last_active":"2024-09-13T07:04:57.318699+0000","last_peered":"2024-09-13T07:04:57.318699+0000","last_clean":"2024-09-13T07:04:57.318699+0000","last_became_active":"2024-09-13T07:04:50.122952+0000","last_became_peered":"2024-09-13T07:04:50.122952+0000","last_unstale":"2024-09-13T07:04:57.318699+0000","last_undegraded":"2024-09-13T07:04:57.318699+0000","last_fullsized":"2024-09-13T07:04:57.318699+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:19.383007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084358800000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13923,"num_read_kb":43420,"num_write":574,"num_write_kb":22169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":8978985,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.875181+0000","last_change":"2024-09-13T07:02:44.091196+0000","last_active":"2024-09-13T07:04:57.875181+0000","last_peered":"2024-09-13T07:04:57.875181+0000","last_clean":"2024-09-13T07:04:57.875181+0000","last_became_active":"2024-09-13T07:02:44.090715+0000","last_became_peered":"2024-09-13T07:02:44.090715+0000","last_unstale":"2024-09-13T07:04:57.875181+0000","last_undegraded":"2024-09-13T07:04:57.875181+0000","last_fullsized":"2024-09-13T07:04:57.875181+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:08.303433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"202'376","reported_seq":7117,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322070+0000","last_change":"2024-09-13T07:04:54.130710+0000","last_active":"2024-09-13T07:04:57.322070+0000","last_peered":"2024-09-13T07:04:57.322070+0000","last_clean":"2024-09-13T07:04:57.322070+0000","last_became_active":"2024-09-13T07:04:54.129673+0000","last_became_peered":"2024-09-13T07:04:54.129673+0000","last_unstale":"2024-09-13T07:04:57.322070+0000","last_undegraded":"2024-09-13T07:04:57.322070+0000","last_fullsized":"2024-09-13T07:04:57.322070+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:42.016575+0000","scrub_duration":0,"objects_trimmed":26,"snaptrim_duration":0.064131435000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6959,"num_read_kb":35900,"num_write":367,"num_write_kb":8191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":3448599,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800247+0000","last_change":"2024-09-13T07:04:57.320659+0000","last_active":"2024-09-13T07:04:58.800247+0000","last_peered":"2024-09-13T07:04:58.800247+0000","last_clean":"2024-09-13T07:04:58.800247+0000","last_became_active":"2024-09-13T07:04:57.320044+0000","last_became_peered":"2024-09-13T07:04:57.320044+0000","last_unstale":"2024-09-13T07:04:58.800247+0000","last_undegraded":"2024-09-13T07:04:58.800247+0000","last_fullsized":"2024-09-13T07:04:58.800247+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:31.189059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"205'511","reported_seq":4274,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317039+0000","last_change":"2024-09-13T07:04:51.127015+0000","last_active":"2024-09-13T07:04:57.317039+0000","last_peered":"2024-09-13T07:04:57.317039+0000","last_clean":"2024-09-13T07:04:57.317039+0000","last_became_active":"2024-09-13T07:04:51.124840+0000","last_became_peered":"2024-09-13T07:04:51.124840+0000","last_unstale":"2024-09-13T07:04:57.317039+0000","last_undegraded":"2024-09-13T07:04:57.317039+0000","last_fullsized":"2024-09-13T07:04:57.317039+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:38:08.467746+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.071371410999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6314,"num_read_kb":38281,"num_write":477,"num_write_kb":24168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":195,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321960+0000","last_change":"2024-09-13T07:04:38.219981+0000","last_active":"2024-09-13T07:04:57.321960+0000","last_peered":"2024-09-13T07:04:57.321960+0000","last_clean":"2024-09-13T07:04:57.321960+0000","last_became_active":"2024-09-13T07:04:38.218537+0000","last_became_peered":"2024-09-13T07:04:38.218537+0000","last_unstale":"2024-09-13T07:04:57.321960+0000","last_undegraded":"2024-09-13T07:04:57.321960+0000","last_fullsized":"2024-09-13T07:04:57.321960+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:49:58.271696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":343,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800461+0000","last_change":"2024-09-13T07:04:57.325982+0000","last_active":"2024-09-13T07:04:58.800461+0000","last_peered":"2024-09-13T07:04:58.800461+0000","last_clean":"2024-09-13T07:04:58.800461+0000","last_became_active":"2024-09-13T07:04:57.324203+0000","last_became_peered":"2024-09-13T07:04:57.324203+0000","last_unstale":"2024-09-13T07:04:58.800461+0000","last_undegraded":"2024-09-13T07:04:58.800461+0000","last_fullsized":"2024-09-13T07:04:58.800461+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:01.766156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"198'782","reported_seq":5552,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316797+0000","last_change":"2024-09-13T07:04:53.134367+0000","last_active":"2024-09-13T07:04:57.316797+0000","last_peered":"2024-09-13T07:04:57.316797+0000","last_clean":"2024-09-13T07:04:57.316797+0000","last_became_active":"2024-09-13T07:04:53.131823+0000","last_became_peered":"2024-09-13T07:04:53.131823+0000","last_unstale":"2024-09-13T07:04:57.316797+0000","last_undegraded":"2024-09-13T07:04:57.316797+0000","last_fullsized":"2024-09-13T07:04:57.316797+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:32:42.148081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032724400000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5361,"num_read_kb":39438,"num_write":976,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":861185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"192'432","reported_seq":9628,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322616+0000","last_change":"2024-09-13T07:04:34.192006+0000","last_active":"2024-09-13T07:04:57.322616+0000","last_peered":"2024-09-13T07:04:57.322616+0000","last_clean":"2024-09-13T07:04:57.322616+0000","last_became_active":"2024-09-13T07:02:47.511257+0000","last_became_peered":"2024-09-13T07:02:47.511257+0000","last_unstale":"2024-09-13T07:04:57.322616+0000","last_undegraded":"2024-09-13T07:04:57.322616+0000","last_fullsized":"2024-09-13T07:04:57.322616+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:04.038572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059111220000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9849,"num_read_kb":28359,"num_write":451,"num_write_kb":30112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5191019,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"198'996","reported_seq":4549,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357116+0000","last_change":"2024-09-13T07:04:34.657476+0000","last_active":"2024-09-13T07:04:57.357116+0000","last_peered":"2024-09-13T07:04:57.357116+0000","last_clean":"2024-09-13T07:04:57.357116+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:04:57.357116+0000","last_undegraded":"2024-09-13T07:04:57.357116+0000","last_fullsized":"2024-09-13T07:04:57.357116+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":996,"log_dups_size":0,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:11.537644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016753040000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7385,"num_read_kb":27871,"num_write":1272,"num_write_kb":19033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"198'599","reported_seq":7645,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316852+0000","last_change":"2024-09-13T07:04:51.128619+0000","last_active":"2024-09-13T07:04:57.316852+0000","last_peered":"2024-09-13T07:04:57.316852+0000","last_clean":"2024-09-13T07:04:57.316852+0000","last_became_active":"2024-09-13T07:04:51.126975+0000","last_became_peered":"2024-09-13T07:04:51.126975+0000","last_unstale":"2024-09-13T07:04:57.316852+0000","last_undegraded":"2024-09-13T07:04:57.316852+0000","last_fullsized":"2024-09-13T07:04:57.316852+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:29.230850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020303249999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9098,"num_read_kb":32181,"num_write":560,"num_write_kb":27201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":10029829,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"205'1700","reported_seq":13975,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322182+0000","last_change":"2024-09-13T07:04:41.334599+0000","last_active":"2024-09-13T07:04:57.322182+0000","last_peered":"2024-09-13T07:04:57.322182+0000","last_clean":"2024-09-13T07:04:57.322182+0000","last_became_active":"2024-09-13T07:04:41.239873+0000","last_became_peered":"2024-09-13T07:04:41.239873+0000","last_unstale":"2024-09-13T07:04:57.322182+0000","last_undegraded":"2024-09-13T07:04:57.322182+0000","last_fullsized":"2024-09-13T07:04:57.322182+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":1700,"log_dups_size":0,"ondisk_log_size":1700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:18.172446+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.091557893000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21059,"num_read_kb":55383,"num_write":2606,"num_write_kb":25105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"198'662","reported_seq":6936,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800373+0000","last_change":"2024-09-13T07:04:57.326646+0000","last_active":"2024-09-13T07:04:58.800373+0000","last_peered":"2024-09-13T07:04:58.800373+0000","last_clean":"2024-09-13T07:04:58.800373+0000","last_became_active":"2024-09-13T07:04:57.323373+0000","last_became_peered":"2024-09-13T07:04:57.323373+0000","last_unstale":"2024-09-13T07:04:58.800373+0000","last_undegraded":"2024-09-13T07:04:58.800373+0000","last_fullsized":"2024-09-13T07:04:58.800373+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:47:35.380836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002404799,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8026,"num_read_kb":31428,"num_write":564,"num_write_kb":31570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9245169,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"198'561","reported_seq":3683,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800239+0000","last_change":"2024-09-13T07:04:57.329473+0000","last_active":"2024-09-13T07:04:58.800239+0000","last_peered":"2024-09-13T07:04:58.800239+0000","last_clean":"2024-09-13T07:04:58.800239+0000","last_became_active":"2024-09-13T07:04:57.325924+0000","last_became_peered":"2024-09-13T07:04:57.325924+0000","last_unstale":"2024-09-13T07:04:58.800239+0000","last_undegraded":"2024-09-13T07:04:58.800239+0000","last_fullsized":"2024-09-13T07:04:58.800239+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:09.975816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086238349999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4232,"num_read_kb":25664,"num_write":732,"num_write_kb":10506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1357199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"198'909","reported_seq":3819,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800888+0000","last_change":"2024-09-13T07:04:57.330501+0000","last_active":"2024-09-13T07:04:58.800888+0000","last_peered":"2024-09-13T07:04:58.800888+0000","last_clean":"2024-09-13T07:04:58.800888+0000","last_became_active":"2024-09-13T07:04:57.326822+0000","last_became_peered":"2024-09-13T07:04:57.326822+0000","last_unstale":"2024-09-13T07:04:58.800888+0000","last_undegraded":"2024-09-13T07:04:58.800888+0000","last_fullsized":"2024-09-13T07:04:58.800888+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:47:41.007573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075911479999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5743,"num_read_kb":24275,"num_write":1279,"num_write_kb":27074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":68557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"198'419","reported_seq":2601,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357197+0000","last_change":"2024-09-13T07:04:34.657069+0000","last_active":"2024-09-13T07:04:57.357197+0000","last_peered":"2024-09-13T07:04:57.357197+0000","last_clean":"2024-09-13T07:04:57.357197+0000","last_became_active":"2024-09-13T07:03:06.743141+0000","last_became_peered":"2024-09-13T07:03:06.743141+0000","last_unstale":"2024-09-13T07:04:57.357197+0000","last_undegraded":"2024-09-13T07:04:57.357197+0000","last_fullsized":"2024-09-13T07:04:57.357197+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:03:42.072271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012751349999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2106,"num_read_kb":12966,"num_write":446,"num_write_kb":14235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":10481695,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"198'592","reported_seq":8333,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800670+0000","last_change":"2024-09-13T07:04:57.330363+0000","last_active":"2024-09-13T07:04:58.800670+0000","last_peered":"2024-09-13T07:04:58.800670+0000","last_clean":"2024-09-13T07:04:58.800670+0000","last_became_active":"2024-09-13T07:04:57.325398+0000","last_became_peered":"2024-09-13T07:04:57.325398+0000","last_unstale":"2024-09-13T07:04:58.800670+0000","last_undegraded":"2024-09-13T07:04:58.800670+0000","last_fullsized":"2024-09-13T07:04:58.800670+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:57.680683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046356670000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8997,"num_read_kb":22165,"num_write":633,"num_write_kb":13453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":102776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"198'927","reported_seq":9117,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322084+0000","last_change":"2024-09-13T07:04:47.132559+0000","last_active":"2024-09-13T07:04:57.322084+0000","last_peered":"2024-09-13T07:04:57.322084+0000","last_clean":"2024-09-13T07:04:57.322084+0000","last_became_active":"2024-09-13T07:04:47.130438+0000","last_became_peered":"2024-09-13T07:04:47.130438+0000","last_unstale":"2024-09-13T07:04:57.322084+0000","last_undegraded":"2024-09-13T07:04:57.322084+0000","last_fullsized":"2024-09-13T07:04:57.322084+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:47:15.123470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076933,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9860,"num_read_kb":45806,"num_write":1253,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":5924852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"199'588","reported_seq":4689,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800492+0000","last_change":"2024-09-13T07:04:57.329853+0000","last_active":"2024-09-13T07:04:58.800492+0000","last_peered":"2024-09-13T07:04:58.800492+0000","last_clean":"2024-09-13T07:04:58.800492+0000","last_became_active":"2024-09-13T07:04:57.325425+0000","last_became_peered":"2024-09-13T07:04:57.325425+0000","last_unstale":"2024-09-13T07:04:58.800492+0000","last_undegraded":"2024-09-13T07:04:58.800492+0000","last_fullsized":"2024-09-13T07:04:58.800492+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:05.096848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051571560000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5130,"num_read_kb":30367,"num_write":654,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":12828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":408,"num_object_clones":0,"num_object_copies":816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":408,"num_whiteouts":404,"num_read":239446,"num_read_kb":1058874,"num_write":24086,"num_write_kb":567966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5260,"num_bytes_recovered":115290013,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},"log_size":20738,"ondisk_log_size":20738,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78604,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293236,"statfs":{"total":96636764160,"available":96556273664,"internally_reserved":0,"allocated":3276800,"data_stored":1887770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77199819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71399999999999997,"15min":0.70099999999999996},"min":{"1min":0.40500000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1870000000000001,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":0.88200000000000001,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":12.583,"5min":12.583,"15min":12.583},"last":0.80700000000000005}]},{"osd":1,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76000000000000001,"15min":0.73399999999999999},"min":{"1min":0.41299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.552,"5min":2.016,"15min":2.016},"last":0.871},{"interface":"front","average":{"1min":1.5089999999999999,"5min":0.99199999999999999,"15min":0.92800000000000005},"min":{"1min":0.56599999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":12.481,"5min":12.481,"15min":12.481},"last":0.96399999999999997}]},{"osd":2,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.88900000000000001,"15min":0.871},"min":{"1min":0.46200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85699999999999998,"15min":0.873},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.2330000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.91200000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2509999999999999,"5min":2.379,"15min":2.379},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.92700000000000005,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6220000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0489999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.048,"15min":0.96999999999999997},"min":{"1min":0.46300000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":12.622,"5min":12.622,"15min":12.622},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.98799999999999999,"15min":1.0009999999999999},"min":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2729999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.127}]},{"osd":5,"last update":"Fri Sep 13 07:03:03 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0129999999999999,"15min":1.0309999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.427,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.024,"15min":1.038},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3069999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.0880000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.198,"15min":1.121},"min":{"1min":0.64700000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":12.555999999999999,"5min":12.555999999999999,"15min":12.555999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.099,"15min":1.1020000000000001},"min":{"1min":0.72499999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.1890000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33740,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338100,"statfs":{"total":96636764160,"available":96602214400,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.84299999999999997,"15min":0.875},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.262,"5min":1.538,"15min":1.538},"last":0.871}]},{"osd":1,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.327,"5min":1.484,"15min":1.484},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.83799999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78200000000000003,"15min":0.81200000000000006},"min":{"1min":0.51800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.96899999999999997,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.88800000000000001,"15min":0.93000000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2290000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.95499999999999996}]},{"osd":3,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.92200000000000004,"15min":0.94899999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3799999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.94699999999999995,"15min":0.94599999999999995},"min":{"1min":0.55000000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.98099999999999998,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":1.036}]},{"osd":4,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88800000000000001,"15min":0.89300000000000002},"min":{"1min":0.47499999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1539999999999999,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.98699999999999999,"15min":1.0189999999999999},"min":{"1min":0.45100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.29,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.141,"5min":1.579,"15min":1.579},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0349999999999999,"15min":1.0549999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3420000000000001,"5min":1.857,"15min":1.857},"last":1.091}]},{"osd":7,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.056,"15min":1.1040000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3080000000000001,"5min":1.607,"15min":1.607},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.0429999999999999,"15min":1.093},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2450000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.98299999999999998}]}]},{"osd":5,"up_from":201,"seq":863288426503,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35732,"kb_used_data":2628,"kb_used_omap":35,"kb_used_meta":33052,"kb_avail":94336108,"statfs":{"total":96636764160,"available":96600174592,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36852,"internal_metadata":33845260},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69840,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302000,"statfs":{"total":96636764160,"available":96565248000,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.40600000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.99099999999999999,"15min":0.96299999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.1479999999999999,"5min":8.2089999999999996,"15min":8.2089999999999996},"last":0.95399999999999996}]},{"osd":1,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.93100000000000005,"15min":0.81999999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0549999999999999,"5min":10.932,"15min":10.932},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.84999999999999998,"15min":0.873},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.101,"5min":2.101,"15min":2.101},"last":1.123}]},{"osd":2,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":0.91400000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":5.9359999999999999,"5min":12.968999999999999,"15min":12.968999999999999},"last":1.0760000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.89200000000000002,"15min":0.91700000000000004},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.26},{"interface":"front","average":{"1min":0.754,"5min":0.80700000000000005,"15min":0.82099999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97899999999999998,"5min":1.159,"15min":1.159},"last":0.84099999999999997}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":0.95499999999999996,"15min":0.873},"min":{"1min":0.56899999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":8.2550000000000008,"5min":8.2550000000000008,"15min":8.2550000000000008},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.198,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.1519999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.83899999999999997,"15min":0.86499999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.123,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0960000000000001,"15min":1.03},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1240000000000001,"5min":10.885,"15min":10.885},"last":1.224}]},{"osd":7,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.92500000000000004,"15min":0.93200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.92800000000000005,"15min":0.95899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1830000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.3129999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51788,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320052,"statfs":{"total":96636764160,"available":96583733248,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.34399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.161,"5min":1.581,"15min":1.581},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.38900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.0049999999999999}]},{"osd":1,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.72699999999999998},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.143,"5min":1.577,"15min":1.577},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.073,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.872}]},{"osd":2,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.026,"5min":1.145,"15min":1.145},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2430000000000001,"5min":1.375,"15min":1.375},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92100000000000004,"15min":0.93400000000000005},"min":{"1min":0.57199999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.466,"5min":1.488,"15min":1.488},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.94299999999999995,"15min":0.95699999999999996},"min":{"1min":0.64600000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3340000000000001,"5min":1.512,"15min":1.512},"last":0.97899999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3149999999999999,"5min":1.369,"15min":1.369},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.002,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4299999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.1359999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.008,"15min":1.022},"min":{"1min":0.63800000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.04},{"interface":"front","average":{"1min":0.998,"5min":1.0169999999999999,"15min":1.0249999999999999},"min":{"1min":0.69499999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.0720000000000001}]},{"osd":7,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.0960000000000001,"15min":1.0920000000000001},"min":{"1min":0.73999999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7549999999999999,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.0109999999999999,"15min":0.99199999999999999},"min":{"1min":0.68100000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.1919999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33748,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338092,"statfs":{"total":96636764160,"available":96602206208,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.91600000000000004,"15min":1.018},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2070000000000001,"5min":6.867,"15min":6.867},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.66700000000000004},"min":{"1min":0.45000000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":1.0009999999999999,"5min":1.234,"15min":1.234},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79600000000000004,"15min":0.79000000000000004},"min":{"1min":0.46100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.306,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93200000000000005,"15min":0.94399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.016,"5min":0.999,"15min":1.006},"min":{"1min":0.70799999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.631,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.88}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.97099999999999997,"15min":0.96899999999999997},"min":{"1min":0.58299999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1,"15min":1.004},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.4670000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.018,"15min":1.0109999999999999},"min":{"1min":0.68899999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4690000000000001,"5min":2.109,"15min":2.109},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.03,"5min":1.0680000000000001,"15min":1.0760000000000001},"min":{"1min":0.61499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4330000000000001,"5min":1.748,"15min":1.748},"last":0.746}]},{"osd":7,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0369999999999999,"15min":1.0349999999999999},"min":{"1min":0.745,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.036,"15min":1.038},"min":{"1min":0.627,"5min":0.505,"15min":0.505},"max":{"1min":1.4610000000000001,"5min":1.7,"15min":1.7},"last":0.93400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29648,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342192,"statfs":{"total":96636764160,"available":96606404608,"internally_reserved":0,"allocated":2691072,"data_stored":1296855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.34499999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.0960000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65500000000000003,"15min":0.64500000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.92300000000000004}]},{"osd":2,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.47599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.70599999999999996,"15min":0.68600000000000005},"min":{"1min":0.41599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.79300000000000004}]},{"osd":3,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.751,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.157,"5min":1.171,"15min":1.171},"last":1.034},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.73499999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.0089999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90500000000000003,"15min":0.89700000000000002},"min":{"1min":0.53500000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.474,"5min":1.925,"15min":1.925},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.008,"5min":1.1599999999999999,"15min":1.0049999999999999},"min":{"1min":0.69199999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.319,"5min":13.701000000000001,"15min":13.701000000000001},"last":1.119}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.006,"15min":1.01},"min":{"1min":0.66400000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3400000000000001,"5min":1.48,"15min":1.48},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.98899999999999999,"15min":0.995},"min":{"1min":0.624,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.409,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.1919999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0129999999999999,"15min":1.014},"min":{"1min":0.56200000000000006,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2410000000000001,"5min":1.575,"15min":1.575},"last":1.258},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0509999999999999,"15min":1.046},"min":{"1min":0.66000000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.544}]},{"osd":7,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.633,"15min":1.282},"min":{"1min":0.73399999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.73,"5min":59.652000000000001,"15min":59.652000000000001},"last":1.415},{"interface":"front","average":{"1min":1.028,"5min":1.589,"15min":1.2390000000000001},"min":{"1min":0.64400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.423,"5min":59.679000000000002,"15min":59.679000000000002},"last":1.153}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38416,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333424,"statfs":{"total":96636764160,"available":96597426176,"internally_reserved":0,"allocated":3276800,"data_stored":1887185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.30199999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.548,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.33700000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.629}]},{"osd":2,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.35299999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.1399999999999999,"5min":1.629,"15min":1.629},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78900000000000003,"15min":0.80100000000000005},"min":{"1min":0.439,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2889999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.89100000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.038,"5min":1.635,"15min":1.635},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4710000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.86799999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.52800000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93300000000000005,"15min":0.94899999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3360000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.1020000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.02,"15min":1.034},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1870000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.873,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.254,"5min":1.724,"15min":1.724},"last":1.1180000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.0209999999999999,"15min":1.0429999999999999},"min":{"1min":0.50700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5469999999999999,"5min":1.552,"15min":1.552},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0509999999999999,"15min":1.077},"min":{"1min":0.55400000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.5780000000000001,"5min":2.024,"15min":2.024},"last":1.024}]},{"osd":7,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.079,"15min":1.093},"min":{"1min":0.59599999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5389999999999999,"5min":2,"15min":2},"last":1.073},{"interface":"front","average":{"1min":0.999,"5min":1.0900000000000001,"15min":1.1279999999999999},"min":{"1min":0.69099999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.389,"5min":1.744,"15min":1.744},"last":1.0029999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:05:04.025 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:05:04.326 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:05:04.327 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:05:04.340 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-13T07:05:02.663768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":412,"num_object_clones":0,"num_object_copies":824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":412,"num_whiteouts":404,"num_read":239562,"num_read_kb":1058974,"num_write":24247,"num_write_kb":570384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5266,"num_bytes_recovered":116470768,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20858,"ondisk_log_size":20858,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":371516,"kb_used_data":22156,"kb_used_omap":46,"kb_used_meta":349201,"kb_avail":754603204,"statfs":{"total":773094113280,"available":772713680896,"internally_reserved":0,"allocated":22687744,"data_stored":11560198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48002,"internal_metadata":357581950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-672,"num_bytes_recovered":-590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.366275"},"pg_stats":[{"pgid":"3.1f","version":"202'415","reported_seq":6819,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322192+0000","last_change":"2024-09-13T07:04:48.134119+0000","last_active":"2024-09-13T07:04:57.322192+0000","last_peered":"2024-09-13T07:04:57.322192+0000","last_clean":"2024-09-13T07:04:57.322192+0000","last_became_active":"2024-09-13T07:04:48.131331+0000","last_became_peered":"2024-09-13T07:04:48.131331+0000","last_unstale":"2024-09-13T07:04:57.322192+0000","last_undegraded":"2024-09-13T07:04:57.322192+0000","last_fullsized":"2024-09-13T07:04:57.322192+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:37:36.544267+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098858737000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6370,"num_read_kb":30920,"num_write":459,"num_write_kb":15662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":4960629,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"195'379","reported_seq":4118,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317289+0000","last_change":"2024-09-13T07:04:51.127712+0000","last_active":"2024-09-13T07:04:57.317289+0000","last_peered":"2024-09-13T07:04:57.317289+0000","last_clean":"2024-09-13T07:04:57.317289+0000","last_became_active":"2024-09-13T07:04:51.125903+0000","last_became_peered":"2024-09-13T07:04:51.125903+0000","last_unstale":"2024-09-13T07:04:57.317289+0000","last_undegraded":"2024-09-13T07:04:57.317289+0000","last_fullsized":"2024-09-13T07:04:57.317289+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:39.870655+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001432102,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4275,"num_read_kb":38815,"num_write":373,"num_write_kb":7930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"202'489","reported_seq":2580,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322555+0000","last_change":"2024-09-13T07:04:38.530429+0000","last_active":"2024-09-13T07:04:57.322555+0000","last_peered":"2024-09-13T07:04:57.322555+0000","last_clean":"2024-09-13T07:04:57.322555+0000","last_became_active":"2024-09-13T07:04:38.221697+0000","last_became_peered":"2024-09-13T07:04:38.221697+0000","last_unstale":"2024-09-13T07:04:57.322555+0000","last_undegraded":"2024-09-13T07:04:57.322555+0000","last_fullsized":"2024-09-13T07:04:57.322555+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:55.624774+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.073210949999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3758,"num_read_kb":13816,"num_write":504,"num_write_kb":5767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"202'500","reported_seq":5592,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321964+0000","last_change":"2024-09-13T07:04:38.656329+0000","last_active":"2024-09-13T07:04:57.321964+0000","last_peered":"2024-09-13T07:04:57.321964+0000","last_clean":"2024-09-13T07:04:57.321964+0000","last_became_active":"2024-09-13T07:04:38.225836+0000","last_became_peered":"2024-09-13T07:04:38.225836+0000","last_unstale":"2024-09-13T07:04:57.321964+0000","last_undegraded":"2024-09-13T07:04:57.321964+0000","last_fullsized":"2024-09-13T07:04:57.321964+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:40.102083+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.078226893000000006,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6179,"num_read_kb":24975,"num_write":551,"num_write_kb":10621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"198'381","reported_seq":6393,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322169+0000","last_change":"2024-09-13T07:04:48.130936+0000","last_active":"2024-09-13T07:04:57.322169+0000","last_peered":"2024-09-13T07:04:57.322169+0000","last_clean":"2024-09-13T07:04:57.322169+0000","last_became_active":"2024-09-13T07:04:48.128577+0000","last_became_peered":"2024-09-13T07:04:48.128577+0000","last_unstale":"2024-09-13T07:04:57.322169+0000","last_undegraded":"2024-09-13T07:04:57.322169+0000","last_fullsized":"2024-09-13T07:04:57.322169+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:54.041815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065506189999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6468,"num_read_kb":53020,"num_write":414,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":148740,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"198'1119","reported_seq":8612,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321787+0000","last_change":"2024-09-13T07:04:50.123312+0000","last_active":"2024-09-13T07:04:57.321787+0000","last_peered":"2024-09-13T07:04:57.321787+0000","last_clean":"2024-09-13T07:04:57.321787+0000","last_became_active":"2024-09-13T07:04:50.121731+0000","last_became_peered":"2024-09-13T07:04:50.121731+0000","last_unstale":"2024-09-13T07:04:57.321787+0000","last_undegraded":"2024-09-13T07:04:57.321787+0000","last_fullsized":"2024-09-13T07:04:57.321787+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":1119,"log_dups_size":0,"ondisk_log_size":1119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:03.606743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002988971,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":11548,"num_read_kb":34808,"num_write":1366,"num_write_kb":17528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":3976247,"num_keys_recovered":80,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"197'628","reported_seq":7696,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318581+0000","last_change":"2024-09-13T07:04:50.125951+0000","last_active":"2024-09-13T07:04:57.318581+0000","last_peered":"2024-09-13T07:04:57.318581+0000","last_clean":"2024-09-13T07:04:57.318581+0000","last_became_active":"2024-09-13T07:04:50.123234+0000","last_became_peered":"2024-09-13T07:04:50.123234+0000","last_unstale":"2024-09-13T07:04:57.318581+0000","last_undegraded":"2024-09-13T07:04:57.318581+0000","last_fullsized":"2024-09-13T07:04:57.318581+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:05.238196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001498135,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9140,"num_read_kb":20172,"num_write":660,"num_write_kb":17799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":1288675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"198'697","reported_seq":7807,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316898+0000","last_change":"2024-09-13T07:04:51.128278+0000","last_active":"2024-09-13T07:04:57.316898+0000","last_peered":"2024-09-13T07:04:57.316898+0000","last_clean":"2024-09-13T07:04:57.316898+0000","last_became_active":"2024-09-13T07:04:51.126595+0000","last_became_peered":"2024-09-13T07:04:51.126595+0000","last_unstale":"2024-09-13T07:04:57.316898+0000","last_undegraded":"2024-09-13T07:04:57.316898+0000","last_fullsized":"2024-09-13T07:04:57.316898+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:40.666326+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001501437,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8486,"num_read_kb":30532,"num_write":647,"num_write_kb":16391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":13265380,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"202'893","reported_seq":12728,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322291+0000","last_change":"2024-09-13T07:04:45.142416+0000","last_active":"2024-09-13T07:04:57.322291+0000","last_peered":"2024-09-13T07:04:57.322291+0000","last_clean":"2024-09-13T07:04:57.322291+0000","last_became_active":"2024-09-13T07:04:45.140362+0000","last_became_peered":"2024-09-13T07:04:45.140362+0000","last_unstale":"2024-09-13T07:04:57.322291+0000","last_undegraded":"2024-09-13T07:04:57.322291+0000","last_fullsized":"2024-09-13T07:04:57.322291+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:06:57.919834+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.082577740999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":15171,"num_read_kb":70673,"num_write":1118,"num_write_kb":35612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":8426052,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"198'847","reported_seq":12699,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.323110+0000","last_change":"2024-09-13T07:04:47.128907+0000","last_active":"2024-09-13T07:04:57.323110+0000","last_peered":"2024-09-13T07:04:57.323110+0000","last_clean":"2024-09-13T07:04:57.323110+0000","last_became_active":"2024-09-13T07:04:47.126092+0000","last_became_peered":"2024-09-13T07:04:47.126092+0000","last_unstale":"2024-09-13T07:04:57.323110+0000","last_undegraded":"2024-09-13T07:04:57.323110+0000","last_fullsized":"2024-09-13T07:04:57.323110+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:17:07.217834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075556199999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":14283,"num_read_kb":24697,"num_write":1275,"num_write_kb":8273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1104561,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"198'452","reported_seq":2662,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800574+0000","last_change":"2024-09-13T07:04:57.327607+0000","last_active":"2024-09-13T07:04:58.800574+0000","last_peered":"2024-09-13T07:04:58.800574+0000","last_clean":"2024-09-13T07:04:58.800574+0000","last_became_active":"2024-09-13T07:04:57.321478+0000","last_became_peered":"2024-09-13T07:04:57.321478+0000","last_unstale":"2024-09-13T07:04:58.800574+0000","last_undegraded":"2024-09-13T07:04:58.800574+0000","last_fullsized":"2024-09-13T07:04:58.800574+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:54:35.402390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001895851,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2526,"num_read_kb":26928,"num_write":433,"num_write_kb":12456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"198'516","reported_seq":5743,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321735+0000","last_change":"2024-09-13T07:04:46.128420+0000","last_active":"2024-09-13T07:04:57.321735+0000","last_peered":"2024-09-13T07:04:57.321735+0000","last_clean":"2024-09-13T07:04:57.321735+0000","last_became_active":"2024-09-13T07:04:46.126170+0000","last_became_peered":"2024-09-13T07:04:46.126170+0000","last_unstale":"2024-09-13T07:04:57.321735+0000","last_undegraded":"2024-09-13T07:04:57.321735+0000","last_fullsized":"2024-09-13T07:04:57.321735+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:41.697482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074505329999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5002,"num_read_kb":31007,"num_write":502,"num_write_kb":34041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":6788059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357224+0000","last_change":"2024-09-13T07:04:43.266461+0000","last_active":"2024-09-13T07:04:57.357224+0000","last_peered":"2024-09-13T07:04:57.357224+0000","last_clean":"2024-09-13T07:04:57.357224+0000","last_became_active":"2024-09-13T07:04:43.266020+0000","last_became_peered":"2024-09-13T07:04:43.266020+0000","last_unstale":"2024-09-13T07:04:57.357224+0000","last_undegraded":"2024-09-13T07:04:57.357224+0000","last_fullsized":"2024-09-13T07:04:57.357224+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:26.181412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"199'459","reported_seq":3615,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322391+0000","last_change":"2024-09-13T07:04:49.126459+0000","last_active":"2024-09-13T07:04:57.322391+0000","last_peered":"2024-09-13T07:04:57.322391+0000","last_clean":"2024-09-13T07:04:57.322391+0000","last_became_active":"2024-09-13T07:04:49.125105+0000","last_became_peered":"2024-09-13T07:04:49.125105+0000","last_unstale":"2024-09-13T07:04:57.322391+0000","last_undegraded":"2024-09-13T07:04:57.322391+0000","last_fullsized":"2024-09-13T07:04:57.322391+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:44:15.141007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066392000000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2852,"num_read_kb":36914,"num_write":449,"num_write_kb":17878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4764805,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318965+0000","last_change":"2024-09-13T07:04:46.127776+0000","last_active":"2024-09-13T07:04:57.318965+0000","last_peered":"2024-09-13T07:04:57.318965+0000","last_clean":"2024-09-13T07:04:57.318965+0000","last_became_active":"2024-09-13T07:04:46.127497+0000","last_became_peered":"2024-09-13T07:04:46.127497+0000","last_unstale":"2024-09-13T07:04:57.318965+0000","last_undegraded":"2024-09-13T07:04:57.318965+0000","last_fullsized":"2024-09-13T07:04:57.318965+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:32.313060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"198'599","reported_seq":4770,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317513+0000","last_change":"2024-09-13T07:04:46.128247+0000","last_active":"2024-09-13T07:04:57.317513+0000","last_peered":"2024-09-13T07:04:57.317513+0000","last_clean":"2024-09-13T07:04:57.317513+0000","last_became_active":"2024-09-13T07:04:46.126668+0000","last_became_peered":"2024-09-13T07:04:46.126668+0000","last_unstale":"2024-09-13T07:04:57.317513+0000","last_undegraded":"2024-09-13T07:04:57.317513+0000","last_fullsized":"2024-09-13T07:04:57.317513+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:46:46.209389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001721534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5009,"num_read_kb":48072,"num_write":666,"num_write_kb":15004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":128514,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800745+0000","last_change":"2024-09-13T07:04:57.327360+0000","last_active":"2024-09-13T07:04:58.800745+0000","last_peered":"2024-09-13T07:04:58.800745+0000","last_clean":"2024-09-13T07:04:58.800745+0000","last_became_active":"2024-09-13T07:04:57.325955+0000","last_became_peered":"2024-09-13T07:04:57.325955+0000","last_unstale":"2024-09-13T07:04:58.800745+0000","last_undegraded":"2024-09-13T07:04:58.800745+0000","last_fullsized":"2024-09-13T07:04:58.800745+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:36:35.543692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"202'377","reported_seq":1798,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318846+0000","last_change":"2024-09-13T07:04:49.134895+0000","last_active":"2024-09-13T07:04:57.318846+0000","last_peered":"2024-09-13T07:04:57.318846+0000","last_clean":"2024-09-13T07:04:57.318846+0000","last_became_active":"2024-09-13T07:04:49.132150+0000","last_became_peered":"2024-09-13T07:04:49.132150+0000","last_unstale":"2024-09-13T07:04:57.318846+0000","last_undegraded":"2024-09-13T07:04:57.318846+0000","last_fullsized":"2024-09-13T07:04:57.318846+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:54:57.301919+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.088725134999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1176,"num_read_kb":23362,"num_write":393,"num_write_kb":13842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":2874205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318745+0000","last_change":"2024-09-13T07:04:46.129346+0000","last_active":"2024-09-13T07:04:57.318745+0000","last_peered":"2024-09-13T07:04:57.318745+0000","last_clean":"2024-09-13T07:04:57.318745+0000","last_became_active":"2024-09-13T07:04:46.129069+0000","last_became_peered":"2024-09-13T07:04:46.129069+0000","last_unstale":"2024-09-13T07:04:57.318745+0000","last_undegraded":"2024-09-13T07:04:57.318745+0000","last_fullsized":"2024-09-13T07:04:57.318745+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:36.800953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'708","reported_seq":5274,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322759+0000","last_change":"2024-09-13T07:04:38.491014+0000","last_active":"2024-09-13T07:04:57.322759+0000","last_peered":"2024-09-13T07:04:57.322759+0000","last_clean":"2024-09-13T07:04:57.322759+0000","last_became_active":"2024-09-13T07:04:38.218110+0000","last_became_peered":"2024-09-13T07:04:38.218110+0000","last_unstale":"2024-09-13T07:04:57.322759+0000","last_undegraded":"2024-09-13T07:04:57.322759+0000","last_fullsized":"2024-09-13T07:04:57.322759+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:23:34.445526+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.101155064,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6847,"num_read_kb":24040,"num_write":729,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"199'610","reported_seq":6642,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322366+0000","last_change":"2024-09-13T07:04:34.193235+0000","last_active":"2024-09-13T07:04:57.322366+0000","last_peered":"2024-09-13T07:04:57.322366+0000","last_clean":"2024-09-13T07:04:57.322366+0000","last_became_active":"2024-09-13T07:03:06.744590+0000","last_became_peered":"2024-09-13T07:03:06.744590+0000","last_unstale":"2024-09-13T07:04:57.322366+0000","last_undegraded":"2024-09-13T07:04:57.322366+0000","last_fullsized":"2024-09-13T07:04:57.322366+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:59:25.982967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006933257,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":6314,"num_read_kb":32619,"num_write":703,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":11860709,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":175,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322603+0000","last_change":"2024-09-13T07:04:46.408868+0000","last_active":"2024-09-13T07:04:57.322603+0000","last_peered":"2024-09-13T07:04:57.322603+0000","last_clean":"2024-09-13T07:04:57.322603+0000","last_became_active":"2024-09-13T07:04:46.125291+0000","last_became_peered":"2024-09-13T07:04:46.125291+0000","last_unstale":"2024-09-13T07:04:57.322603+0000","last_undegraded":"2024-09-13T07:04:57.322603+0000","last_fullsized":"2024-09-13T07:04:57.322603+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:50:37.881225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"197'615","reported_seq":14047,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318699+0000","last_change":"2024-09-13T07:04:50.127801+0000","last_active":"2024-09-13T07:04:57.318699+0000","last_peered":"2024-09-13T07:04:57.318699+0000","last_clean":"2024-09-13T07:04:57.318699+0000","last_became_active":"2024-09-13T07:04:50.122952+0000","last_became_peered":"2024-09-13T07:04:50.122952+0000","last_unstale":"2024-09-13T07:04:57.318699+0000","last_undegraded":"2024-09-13T07:04:57.318699+0000","last_fullsized":"2024-09-13T07:04:57.318699+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:19.383007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084358800000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13923,"num_read_kb":43420,"num_write":574,"num_write_kb":22169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":8978985,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.875181+0000","last_change":"2024-09-13T07:02:44.091196+0000","last_active":"2024-09-13T07:04:57.875181+0000","last_peered":"2024-09-13T07:04:57.875181+0000","last_clean":"2024-09-13T07:04:57.875181+0000","last_became_active":"2024-09-13T07:02:44.090715+0000","last_became_peered":"2024-09-13T07:02:44.090715+0000","last_unstale":"2024-09-13T07:04:57.875181+0000","last_undegraded":"2024-09-13T07:04:57.875181+0000","last_fullsized":"2024-09-13T07:04:57.875181+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:08.303433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"202'376","reported_seq":7117,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322070+0000","last_change":"2024-09-13T07:04:54.130710+0000","last_active":"2024-09-13T07:04:57.322070+0000","last_peered":"2024-09-13T07:04:57.322070+0000","last_clean":"2024-09-13T07:04:57.322070+0000","last_became_active":"2024-09-13T07:04:54.129673+0000","last_became_peered":"2024-09-13T07:04:54.129673+0000","last_unstale":"2024-09-13T07:04:57.322070+0000","last_undegraded":"2024-09-13T07:04:57.322070+0000","last_fullsized":"2024-09-13T07:04:57.322070+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:42.016575+0000","scrub_duration":0,"objects_trimmed":26,"snaptrim_duration":0.064131435000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6959,"num_read_kb":35900,"num_write":367,"num_write_kb":8191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":3448599,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800247+0000","last_change":"2024-09-13T07:04:57.320659+0000","last_active":"2024-09-13T07:04:58.800247+0000","last_peered":"2024-09-13T07:04:58.800247+0000","last_clean":"2024-09-13T07:04:58.800247+0000","last_became_active":"2024-09-13T07:04:57.320044+0000","last_became_peered":"2024-09-13T07:04:57.320044+0000","last_unstale":"2024-09-13T07:04:58.800247+0000","last_undegraded":"2024-09-13T07:04:58.800247+0000","last_fullsized":"2024-09-13T07:04:58.800247+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:31.189059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"205'511","reported_seq":4274,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317039+0000","last_change":"2024-09-13T07:04:51.127015+0000","last_active":"2024-09-13T07:04:57.317039+0000","last_peered":"2024-09-13T07:04:57.317039+0000","last_clean":"2024-09-13T07:04:57.317039+0000","last_became_active":"2024-09-13T07:04:51.124840+0000","last_became_peered":"2024-09-13T07:04:51.124840+0000","last_unstale":"2024-09-13T07:04:57.317039+0000","last_undegraded":"2024-09-13T07:04:57.317039+0000","last_fullsized":"2024-09-13T07:04:57.317039+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:38:08.467746+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.071371410999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6314,"num_read_kb":38281,"num_write":477,"num_write_kb":24168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":195,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321960+0000","last_change":"2024-09-13T07:04:38.219981+0000","last_active":"2024-09-13T07:04:57.321960+0000","last_peered":"2024-09-13T07:04:57.321960+0000","last_clean":"2024-09-13T07:04:57.321960+0000","last_became_active":"2024-09-13T07:04:38.218537+0000","last_became_peered":"2024-09-13T07:04:38.218537+0000","last_unstale":"2024-09-13T07:04:57.321960+0000","last_undegraded":"2024-09-13T07:04:57.321960+0000","last_fullsized":"2024-09-13T07:04:57.321960+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:49:58.271696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":343,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800461+0000","last_change":"2024-09-13T07:04:57.325982+0000","last_active":"2024-09-13T07:04:58.800461+0000","last_peered":"2024-09-13T07:04:58.800461+0000","last_clean":"2024-09-13T07:04:58.800461+0000","last_became_active":"2024-09-13T07:04:57.324203+0000","last_became_peered":"2024-09-13T07:04:57.324203+0000","last_unstale":"2024-09-13T07:04:58.800461+0000","last_undegraded":"2024-09-13T07:04:58.800461+0000","last_fullsized":"2024-09-13T07:04:58.800461+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:01.766156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"198'782","reported_seq":5552,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316797+0000","last_change":"2024-09-13T07:04:53.134367+0000","last_active":"2024-09-13T07:04:57.316797+0000","last_peered":"2024-09-13T07:04:57.316797+0000","last_clean":"2024-09-13T07:04:57.316797+0000","last_became_active":"2024-09-13T07:04:53.131823+0000","last_became_peered":"2024-09-13T07:04:53.131823+0000","last_unstale":"2024-09-13T07:04:57.316797+0000","last_undegraded":"2024-09-13T07:04:57.316797+0000","last_fullsized":"2024-09-13T07:04:57.316797+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:32:42.148081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032724400000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5361,"num_read_kb":39438,"num_write":976,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":861185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"192'432","reported_seq":9628,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322616+0000","last_change":"2024-09-13T07:04:34.192006+0000","last_active":"2024-09-13T07:04:57.322616+0000","last_peered":"2024-09-13T07:04:57.322616+0000","last_clean":"2024-09-13T07:04:57.322616+0000","last_became_active":"2024-09-13T07:02:47.511257+0000","last_became_peered":"2024-09-13T07:02:47.511257+0000","last_unstale":"2024-09-13T07:04:57.322616+0000","last_undegraded":"2024-09-13T07:04:57.322616+0000","last_fullsized":"2024-09-13T07:04:57.322616+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:04.038572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059111220000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9849,"num_read_kb":28359,"num_write":451,"num_write_kb":30112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5191019,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"198'996","reported_seq":4549,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357116+0000","last_change":"2024-09-13T07:04:34.657476+0000","last_active":"2024-09-13T07:04:57.357116+0000","last_peered":"2024-09-13T07:04:57.357116+0000","last_clean":"2024-09-13T07:04:57.357116+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:04:57.357116+0000","last_undegraded":"2024-09-13T07:04:57.357116+0000","last_fullsized":"2024-09-13T07:04:57.357116+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":996,"log_dups_size":0,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:11.537644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016753040000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7385,"num_read_kb":27871,"num_write":1272,"num_write_kb":19033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"198'599","reported_seq":7645,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316852+0000","last_change":"2024-09-13T07:04:51.128619+0000","last_active":"2024-09-13T07:04:57.316852+0000","last_peered":"2024-09-13T07:04:57.316852+0000","last_clean":"2024-09-13T07:04:57.316852+0000","last_became_active":"2024-09-13T07:04:51.126975+0000","last_became_peered":"2024-09-13T07:04:51.126975+0000","last_unstale":"2024-09-13T07:04:57.316852+0000","last_undegraded":"2024-09-13T07:04:57.316852+0000","last_fullsized":"2024-09-13T07:04:57.316852+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:29.230850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020303249999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9098,"num_read_kb":32181,"num_write":560,"num_write_kb":27201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":10029829,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"205'1700","reported_seq":13975,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322182+0000","last_change":"2024-09-13T07:04:41.334599+0000","last_active":"2024-09-13T07:04:57.322182+0000","last_peered":"2024-09-13T07:04:57.322182+0000","last_clean":"2024-09-13T07:04:57.322182+0000","last_became_active":"2024-09-13T07:04:41.239873+0000","last_became_peered":"2024-09-13T07:04:41.239873+0000","last_unstale":"2024-09-13T07:04:57.322182+0000","last_undegraded":"2024-09-13T07:04:57.322182+0000","last_fullsized":"2024-09-13T07:04:57.322182+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":1700,"log_dups_size":0,"ondisk_log_size":1700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:18.172446+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.091557893000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21059,"num_read_kb":55383,"num_write":2606,"num_write_kb":25105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"198'662","reported_seq":6936,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800373+0000","last_change":"2024-09-13T07:04:57.326646+0000","last_active":"2024-09-13T07:04:58.800373+0000","last_peered":"2024-09-13T07:04:58.800373+0000","last_clean":"2024-09-13T07:04:58.800373+0000","last_became_active":"2024-09-13T07:04:57.323373+0000","last_became_peered":"2024-09-13T07:04:57.323373+0000","last_unstale":"2024-09-13T07:04:58.800373+0000","last_undegraded":"2024-09-13T07:04:58.800373+0000","last_fullsized":"2024-09-13T07:04:58.800373+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:47:35.380836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002404799,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8026,"num_read_kb":31428,"num_write":564,"num_write_kb":31570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9245169,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"198'561","reported_seq":3683,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800239+0000","last_change":"2024-09-13T07:04:57.329473+0000","last_active":"2024-09-13T07:04:58.800239+0000","last_peered":"2024-09-13T07:04:58.800239+0000","last_clean":"2024-09-13T07:04:58.800239+0000","last_became_active":"2024-09-13T07:04:57.325924+0000","last_became_peered":"2024-09-13T07:04:57.325924+0000","last_unstale":"2024-09-13T07:04:58.800239+0000","last_undegraded":"2024-09-13T07:04:58.800239+0000","last_fullsized":"2024-09-13T07:04:58.800239+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:09.975816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086238349999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4232,"num_read_kb":25664,"num_write":732,"num_write_kb":10506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1357199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"198'909","reported_seq":3819,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800888+0000","last_change":"2024-09-13T07:04:57.330501+0000","last_active":"2024-09-13T07:04:58.800888+0000","last_peered":"2024-09-13T07:04:58.800888+0000","last_clean":"2024-09-13T07:04:58.800888+0000","last_became_active":"2024-09-13T07:04:57.326822+0000","last_became_peered":"2024-09-13T07:04:57.326822+0000","last_unstale":"2024-09-13T07:04:58.800888+0000","last_undegraded":"2024-09-13T07:04:58.800888+0000","last_fullsized":"2024-09-13T07:04:58.800888+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:47:41.007573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075911479999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5743,"num_read_kb":24275,"num_write":1279,"num_write_kb":27074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":68557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"198'419","reported_seq":2601,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357197+0000","last_change":"2024-09-13T07:04:34.657069+0000","last_active":"2024-09-13T07:04:57.357197+0000","last_peered":"2024-09-13T07:04:57.357197+0000","last_clean":"2024-09-13T07:04:57.357197+0000","last_became_active":"2024-09-13T07:03:06.743141+0000","last_became_peered":"2024-09-13T07:03:06.743141+0000","last_unstale":"2024-09-13T07:04:57.357197+0000","last_undegraded":"2024-09-13T07:04:57.357197+0000","last_fullsized":"2024-09-13T07:04:57.357197+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:03:42.072271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012751349999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2106,"num_read_kb":12966,"num_write":446,"num_write_kb":14235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":10481695,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"198'592","reported_seq":8333,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800670+0000","last_change":"2024-09-13T07:04:57.330363+0000","last_active":"2024-09-13T07:04:58.800670+0000","last_peered":"2024-09-13T07:04:58.800670+0000","last_clean":"2024-09-13T07:04:58.800670+0000","last_became_active":"2024-09-13T07:04:57.325398+0000","last_became_peered":"2024-09-13T07:04:57.325398+0000","last_unstale":"2024-09-13T07:04:58.800670+0000","last_undegraded":"2024-09-13T07:04:58.800670+0000","last_fullsized":"2024-09-13T07:04:58.800670+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:57.680683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046356670000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8997,"num_read_kb":22165,"num_write":633,"num_write_kb":13453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":102776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"198'927","reported_seq":9117,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322084+0000","last_change":"2024-09-13T07:04:47.132559+0000","last_active":"2024-09-13T07:04:57.322084+0000","last_peered":"2024-09-13T07:04:57.322084+0000","last_clean":"2024-09-13T07:04:57.322084+0000","last_became_active":"2024-09-13T07:04:47.130438+0000","last_became_peered":"2024-09-13T07:04:47.130438+0000","last_unstale":"2024-09-13T07:04:57.322084+0000","last_undegraded":"2024-09-13T07:04:57.322084+0000","last_fullsized":"2024-09-13T07:04:57.322084+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:47:15.123470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076933,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9860,"num_read_kb":45806,"num_write":1253,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":5924852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"199'588","reported_seq":4689,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800492+0000","last_change":"2024-09-13T07:04:57.329853+0000","last_active":"2024-09-13T07:04:58.800492+0000","last_peered":"2024-09-13T07:04:58.800492+0000","last_clean":"2024-09-13T07:04:58.800492+0000","last_became_active":"2024-09-13T07:04:57.325425+0000","last_became_peered":"2024-09-13T07:04:57.325425+0000","last_unstale":"2024-09-13T07:04:58.800492+0000","last_undegraded":"2024-09-13T07:04:58.800492+0000","last_fullsized":"2024-09-13T07:04:58.800492+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:05.096848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051571560000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5130,"num_read_kb":30367,"num_write":654,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":12828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":408,"num_object_clones":0,"num_object_copies":816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":408,"num_whiteouts":404,"num_read":239446,"num_read_kb":1058874,"num_write":24086,"num_write_kb":567966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5260,"num_bytes_recovered":115290013,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},"log_size":20738,"ondisk_log_size":20738,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78604,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293236,"statfs":{"total":96636764160,"available":96556273664,"internally_reserved":0,"allocated":3276800,"data_stored":1887770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77199819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71399999999999997,"15min":0.70099999999999996},"min":{"1min":0.40500000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1870000000000001,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":0.88200000000000001,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":12.583,"5min":12.583,"15min":12.583},"last":0.80700000000000005}]},{"osd":1,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76000000000000001,"15min":0.73399999999999999},"min":{"1min":0.41299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.552,"5min":2.016,"15min":2.016},"last":0.871},{"interface":"front","average":{"1min":1.5089999999999999,"5min":0.99199999999999999,"15min":0.92800000000000005},"min":{"1min":0.56599999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":12.481,"5min":12.481,"15min":12.481},"last":0.96399999999999997}]},{"osd":2,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.88900000000000001,"15min":0.871},"min":{"1min":0.46200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85699999999999998,"15min":0.873},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.2330000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.91200000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2509999999999999,"5min":2.379,"15min":2.379},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.92700000000000005,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6220000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0489999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.048,"15min":0.96999999999999997},"min":{"1min":0.46300000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":12.622,"5min":12.622,"15min":12.622},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.98799999999999999,"15min":1.0009999999999999},"min":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2729999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.127}]},{"osd":5,"last update":"Fri Sep 13 07:03:03 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0129999999999999,"15min":1.0309999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.427,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.024,"15min":1.038},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3069999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.0880000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.198,"15min":1.121},"min":{"1min":0.64700000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":12.555999999999999,"5min":12.555999999999999,"15min":12.555999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.099,"15min":1.1020000000000001},"min":{"1min":0.72499999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.1890000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33740,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338100,"statfs":{"total":96636764160,"available":96602214400,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.84299999999999997,"15min":0.875},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.262,"5min":1.538,"15min":1.538},"last":0.871}]},{"osd":1,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.327,"5min":1.484,"15min":1.484},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.83799999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78200000000000003,"15min":0.81200000000000006},"min":{"1min":0.51800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.96899999999999997,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.88800000000000001,"15min":0.93000000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2290000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.95499999999999996}]},{"osd":3,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.92200000000000004,"15min":0.94899999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3799999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.94699999999999995,"15min":0.94599999999999995},"min":{"1min":0.55000000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.98099999999999998,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":1.036}]},{"osd":4,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88800000000000001,"15min":0.89300000000000002},"min":{"1min":0.47499999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1539999999999999,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.98699999999999999,"15min":1.0189999999999999},"min":{"1min":0.45100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.29,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.141,"5min":1.579,"15min":1.579},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0349999999999999,"15min":1.0549999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3420000000000001,"5min":1.857,"15min":1.857},"last":1.091}]},{"osd":7,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.056,"15min":1.1040000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3080000000000001,"5min":1.607,"15min":1.607},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.0429999999999999,"15min":1.093},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2450000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.98299999999999998}]}]},{"osd":5,"up_from":201,"seq":863288426503,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35732,"kb_used_data":2628,"kb_used_omap":35,"kb_used_meta":33052,"kb_avail":94336108,"statfs":{"total":96636764160,"available":96600174592,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36852,"internal_metadata":33845260},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69840,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302000,"statfs":{"total":96636764160,"available":96565248000,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.40600000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.99099999999999999,"15min":0.96299999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.1479999999999999,"5min":8.2089999999999996,"15min":8.2089999999999996},"last":0.95399999999999996}]},{"osd":1,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.93100000000000005,"15min":0.81999999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0549999999999999,"5min":10.932,"15min":10.932},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.84999999999999998,"15min":0.873},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.101,"5min":2.101,"15min":2.101},"last":1.123}]},{"osd":2,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":0.91400000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":5.9359999999999999,"5min":12.968999999999999,"15min":12.968999999999999},"last":1.0760000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.89200000000000002,"15min":0.91700000000000004},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.26},{"interface":"front","average":{"1min":0.754,"5min":0.80700000000000005,"15min":0.82099999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97899999999999998,"5min":1.159,"15min":1.159},"last":0.84099999999999997}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":0.95499999999999996,"15min":0.873},"min":{"1min":0.56899999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":8.2550000000000008,"5min":8.2550000000000008,"15min":8.2550000000000008},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.198,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.1519999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.83899999999999997,"15min":0.86499999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.123,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0960000000000001,"15min":1.03},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1240000000000001,"5min":10.885,"15min":10.885},"last":1.224}]},{"osd":7,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.92500000000000004,"15min":0.93200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.92800000000000005,"15min":0.95899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1830000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.3129999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51788,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320052,"statfs":{"total":96636764160,"available":96583733248,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.34399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.161,"5min":1.581,"15min":1.581},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.38900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.0049999999999999}]},{"osd":1,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.72699999999999998},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.143,"5min":1.577,"15min":1.577},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.073,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.872}]},{"osd":2,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.026,"5min":1.145,"15min":1.145},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2430000000000001,"5min":1.375,"15min":1.375},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92100000000000004,"15min":0.93400000000000005},"min":{"1min":0.57199999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.466,"5min":1.488,"15min":1.488},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.94299999999999995,"15min":0.95699999999999996},"min":{"1min":0.64600000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3340000000000001,"5min":1.512,"15min":1.512},"last":0.97899999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3149999999999999,"5min":1.369,"15min":1.369},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.002,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4299999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.1359999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.008,"15min":1.022},"min":{"1min":0.63800000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.04},{"interface":"front","average":{"1min":0.998,"5min":1.0169999999999999,"15min":1.0249999999999999},"min":{"1min":0.69499999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.0720000000000001}]},{"osd":7,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.0960000000000001,"15min":1.0920000000000001},"min":{"1min":0.73999999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7549999999999999,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.0109999999999999,"15min":0.99199999999999999},"min":{"1min":0.68100000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.1919999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33748,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338092,"statfs":{"total":96636764160,"available":96602206208,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.91600000000000004,"15min":1.018},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2070000000000001,"5min":6.867,"15min":6.867},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.66700000000000004},"min":{"1min":0.45000000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":1.0009999999999999,"5min":1.234,"15min":1.234},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79600000000000004,"15min":0.79000000000000004},"min":{"1min":0.46100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.306,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93200000000000005,"15min":0.94399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.016,"5min":0.999,"15min":1.006},"min":{"1min":0.70799999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.631,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.88}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.97099999999999997,"15min":0.96899999999999997},"min":{"1min":0.58299999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1,"15min":1.004},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.4670000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.018,"15min":1.0109999999999999},"min":{"1min":0.68899999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4690000000000001,"5min":2.109,"15min":2.109},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.03,"5min":1.0680000000000001,"15min":1.0760000000000001},"min":{"1min":0.61499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4330000000000001,"5min":1.748,"15min":1.748},"last":0.746}]},{"osd":7,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0369999999999999,"15min":1.0349999999999999},"min":{"1min":0.745,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.036,"15min":1.038},"min":{"1min":0.627,"5min":0.505,"15min":0.505},"max":{"1min":1.4610000000000001,"5min":1.7,"15min":1.7},"last":0.93400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29648,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342192,"statfs":{"total":96636764160,"available":96606404608,"internally_reserved":0,"allocated":2691072,"data_stored":1296855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.34499999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.0960000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65500000000000003,"15min":0.64500000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.92300000000000004}]},{"osd":2,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.47599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.70599999999999996,"15min":0.68600000000000005},"min":{"1min":0.41599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.79300000000000004}]},{"osd":3,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.751,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.157,"5min":1.171,"15min":1.171},"last":1.034},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.73499999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.0089999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90500000000000003,"15min":0.89700000000000002},"min":{"1min":0.53500000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.474,"5min":1.925,"15min":1.925},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.008,"5min":1.1599999999999999,"15min":1.0049999999999999},"min":{"1min":0.69199999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.319,"5min":13.701000000000001,"15min":13.701000000000001},"last":1.119}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.006,"15min":1.01},"min":{"1min":0.66400000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3400000000000001,"5min":1.48,"15min":1.48},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.98899999999999999,"15min":0.995},"min":{"1min":0.624,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.409,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.1919999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0129999999999999,"15min":1.014},"min":{"1min":0.56200000000000006,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2410000000000001,"5min":1.575,"15min":1.575},"last":1.258},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0509999999999999,"15min":1.046},"min":{"1min":0.66000000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.544}]},{"osd":7,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.633,"15min":1.282},"min":{"1min":0.73399999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.73,"5min":59.652000000000001,"15min":59.652000000000001},"last":1.415},{"interface":"front","average":{"1min":1.028,"5min":1.589,"15min":1.2390000000000001},"min":{"1min":0.64400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.423,"5min":59.679000000000002,"15min":59.679000000000002},"last":1.153}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38416,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333424,"statfs":{"total":96636764160,"available":96597426176,"internally_reserved":0,"allocated":3276800,"data_stored":1887185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.30199999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.548,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.33700000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.629}]},{"osd":2,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.35299999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.1399999999999999,"5min":1.629,"15min":1.629},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78900000000000003,"15min":0.80100000000000005},"min":{"1min":0.439,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2889999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.89100000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.038,"5min":1.635,"15min":1.635},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4710000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.86799999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.52800000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93300000000000005,"15min":0.94899999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3360000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.1020000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.02,"15min":1.034},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1870000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.873,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.254,"5min":1.724,"15min":1.724},"last":1.1180000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.0209999999999999,"15min":1.0429999999999999},"min":{"1min":0.50700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5469999999999999,"5min":1.552,"15min":1.552},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0509999999999999,"15min":1.077},"min":{"1min":0.55400000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.5780000000000001,"5min":2.024,"15min":2.024},"last":1.024}]},{"osd":7,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.079,"15min":1.093},"min":{"1min":0.59599999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5389999999999999,"5min":2,"15min":2},"last":1.073},{"interface":"front","average":{"1min":0.999,"5min":1.0900000000000001,"15min":1.1279999999999999},"min":{"1min":0.69099999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.389,"5min":1.744,"15min":1.744},"last":1.0029999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:05:04.343 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T07:05:04.343 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:05:04.661 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:05:04.664 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:05:04.677 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-13T07:05:02.663768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":412,"num_object_clones":0,"num_object_copies":824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":412,"num_whiteouts":404,"num_read":239562,"num_read_kb":1058974,"num_write":24247,"num_write_kb":570384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5266,"num_bytes_recovered":116470768,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20858,"ondisk_log_size":20858,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":371516,"kb_used_data":22156,"kb_used_omap":46,"kb_used_meta":349201,"kb_avail":754603204,"statfs":{"total":773094113280,"available":772713680896,"internally_reserved":0,"allocated":22687744,"data_stored":11560198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48002,"internal_metadata":357581950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-672,"num_bytes_recovered":-590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.366275"},"pg_stats":[{"pgid":"3.1f","version":"202'415","reported_seq":6819,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322192+0000","last_change":"2024-09-13T07:04:48.134119+0000","last_active":"2024-09-13T07:04:57.322192+0000","last_peered":"2024-09-13T07:04:57.322192+0000","last_clean":"2024-09-13T07:04:57.322192+0000","last_became_active":"2024-09-13T07:04:48.131331+0000","last_became_peered":"2024-09-13T07:04:48.131331+0000","last_unstale":"2024-09-13T07:04:57.322192+0000","last_undegraded":"2024-09-13T07:04:57.322192+0000","last_fullsized":"2024-09-13T07:04:57.322192+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:37:36.544267+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098858737000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6370,"num_read_kb":30920,"num_write":459,"num_write_kb":15662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":4960629,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"195'379","reported_seq":4118,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317289+0000","last_change":"2024-09-13T07:04:51.127712+0000","last_active":"2024-09-13T07:04:57.317289+0000","last_peered":"2024-09-13T07:04:57.317289+0000","last_clean":"2024-09-13T07:04:57.317289+0000","last_became_active":"2024-09-13T07:04:51.125903+0000","last_became_peered":"2024-09-13T07:04:51.125903+0000","last_unstale":"2024-09-13T07:04:57.317289+0000","last_undegraded":"2024-09-13T07:04:57.317289+0000","last_fullsized":"2024-09-13T07:04:57.317289+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:39.870655+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001432102,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4275,"num_read_kb":38815,"num_write":373,"num_write_kb":7930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"202'489","reported_seq":2580,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322555+0000","last_change":"2024-09-13T07:04:38.530429+0000","last_active":"2024-09-13T07:04:57.322555+0000","last_peered":"2024-09-13T07:04:57.322555+0000","last_clean":"2024-09-13T07:04:57.322555+0000","last_became_active":"2024-09-13T07:04:38.221697+0000","last_became_peered":"2024-09-13T07:04:38.221697+0000","last_unstale":"2024-09-13T07:04:57.322555+0000","last_undegraded":"2024-09-13T07:04:57.322555+0000","last_fullsized":"2024-09-13T07:04:57.322555+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:55.624774+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.073210949999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3758,"num_read_kb":13816,"num_write":504,"num_write_kb":5767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"202'500","reported_seq":5592,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321964+0000","last_change":"2024-09-13T07:04:38.656329+0000","last_active":"2024-09-13T07:04:57.321964+0000","last_peered":"2024-09-13T07:04:57.321964+0000","last_clean":"2024-09-13T07:04:57.321964+0000","last_became_active":"2024-09-13T07:04:38.225836+0000","last_became_peered":"2024-09-13T07:04:38.225836+0000","last_unstale":"2024-09-13T07:04:57.321964+0000","last_undegraded":"2024-09-13T07:04:57.321964+0000","last_fullsized":"2024-09-13T07:04:57.321964+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:40.102083+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.078226893000000006,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6179,"num_read_kb":24975,"num_write":551,"num_write_kb":10621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"198'381","reported_seq":6393,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322169+0000","last_change":"2024-09-13T07:04:48.130936+0000","last_active":"2024-09-13T07:04:57.322169+0000","last_peered":"2024-09-13T07:04:57.322169+0000","last_clean":"2024-09-13T07:04:57.322169+0000","last_became_active":"2024-09-13T07:04:48.128577+0000","last_became_peered":"2024-09-13T07:04:48.128577+0000","last_unstale":"2024-09-13T07:04:57.322169+0000","last_undegraded":"2024-09-13T07:04:57.322169+0000","last_fullsized":"2024-09-13T07:04:57.322169+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:54.041815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065506189999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6468,"num_read_kb":53020,"num_write":414,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":148740,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"198'1119","reported_seq":8612,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321787+0000","last_change":"2024-09-13T07:04:50.123312+0000","last_active":"2024-09-13T07:04:57.321787+0000","last_peered":"2024-09-13T07:04:57.321787+0000","last_clean":"2024-09-13T07:04:57.321787+0000","last_became_active":"2024-09-13T07:04:50.121731+0000","last_became_peered":"2024-09-13T07:04:50.121731+0000","last_unstale":"2024-09-13T07:04:57.321787+0000","last_undegraded":"2024-09-13T07:04:57.321787+0000","last_fullsized":"2024-09-13T07:04:57.321787+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":1119,"log_dups_size":0,"ondisk_log_size":1119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:03.606743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002988971,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":11548,"num_read_kb":34808,"num_write":1366,"num_write_kb":17528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":3976247,"num_keys_recovered":80,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"197'628","reported_seq":7696,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318581+0000","last_change":"2024-09-13T07:04:50.125951+0000","last_active":"2024-09-13T07:04:57.318581+0000","last_peered":"2024-09-13T07:04:57.318581+0000","last_clean":"2024-09-13T07:04:57.318581+0000","last_became_active":"2024-09-13T07:04:50.123234+0000","last_became_peered":"2024-09-13T07:04:50.123234+0000","last_unstale":"2024-09-13T07:04:57.318581+0000","last_undegraded":"2024-09-13T07:04:57.318581+0000","last_fullsized":"2024-09-13T07:04:57.318581+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:05.238196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001498135,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9140,"num_read_kb":20172,"num_write":660,"num_write_kb":17799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":1288675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"198'697","reported_seq":7807,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316898+0000","last_change":"2024-09-13T07:04:51.128278+0000","last_active":"2024-09-13T07:04:57.316898+0000","last_peered":"2024-09-13T07:04:57.316898+0000","last_clean":"2024-09-13T07:04:57.316898+0000","last_became_active":"2024-09-13T07:04:51.126595+0000","last_became_peered":"2024-09-13T07:04:51.126595+0000","last_unstale":"2024-09-13T07:04:57.316898+0000","last_undegraded":"2024-09-13T07:04:57.316898+0000","last_fullsized":"2024-09-13T07:04:57.316898+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:40.666326+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001501437,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8486,"num_read_kb":30532,"num_write":647,"num_write_kb":16391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":13265380,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"202'893","reported_seq":12728,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322291+0000","last_change":"2024-09-13T07:04:45.142416+0000","last_active":"2024-09-13T07:04:57.322291+0000","last_peered":"2024-09-13T07:04:57.322291+0000","last_clean":"2024-09-13T07:04:57.322291+0000","last_became_active":"2024-09-13T07:04:45.140362+0000","last_became_peered":"2024-09-13T07:04:45.140362+0000","last_unstale":"2024-09-13T07:04:57.322291+0000","last_undegraded":"2024-09-13T07:04:57.322291+0000","last_fullsized":"2024-09-13T07:04:57.322291+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:06:57.919834+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.082577740999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":15171,"num_read_kb":70673,"num_write":1118,"num_write_kb":35612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":8426052,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"198'847","reported_seq":12699,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.323110+0000","last_change":"2024-09-13T07:04:47.128907+0000","last_active":"2024-09-13T07:04:57.323110+0000","last_peered":"2024-09-13T07:04:57.323110+0000","last_clean":"2024-09-13T07:04:57.323110+0000","last_became_active":"2024-09-13T07:04:47.126092+0000","last_became_peered":"2024-09-13T07:04:47.126092+0000","last_unstale":"2024-09-13T07:04:57.323110+0000","last_undegraded":"2024-09-13T07:04:57.323110+0000","last_fullsized":"2024-09-13T07:04:57.323110+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:17:07.217834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075556199999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":14283,"num_read_kb":24697,"num_write":1275,"num_write_kb":8273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1104561,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"198'452","reported_seq":2662,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800574+0000","last_change":"2024-09-13T07:04:57.327607+0000","last_active":"2024-09-13T07:04:58.800574+0000","last_peered":"2024-09-13T07:04:58.800574+0000","last_clean":"2024-09-13T07:04:58.800574+0000","last_became_active":"2024-09-13T07:04:57.321478+0000","last_became_peered":"2024-09-13T07:04:57.321478+0000","last_unstale":"2024-09-13T07:04:58.800574+0000","last_undegraded":"2024-09-13T07:04:58.800574+0000","last_fullsized":"2024-09-13T07:04:58.800574+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:54:35.402390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001895851,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2526,"num_read_kb":26928,"num_write":433,"num_write_kb":12456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"198'516","reported_seq":5743,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321735+0000","last_change":"2024-09-13T07:04:46.128420+0000","last_active":"2024-09-13T07:04:57.321735+0000","last_peered":"2024-09-13T07:04:57.321735+0000","last_clean":"2024-09-13T07:04:57.321735+0000","last_became_active":"2024-09-13T07:04:46.126170+0000","last_became_peered":"2024-09-13T07:04:46.126170+0000","last_unstale":"2024-09-13T07:04:57.321735+0000","last_undegraded":"2024-09-13T07:04:57.321735+0000","last_fullsized":"2024-09-13T07:04:57.321735+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:41.697482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074505329999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5002,"num_read_kb":31007,"num_write":502,"num_write_kb":34041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":6788059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357224+0000","last_change":"2024-09-13T07:04:43.266461+0000","last_active":"2024-09-13T07:04:57.357224+0000","last_peered":"2024-09-13T07:04:57.357224+0000","last_clean":"2024-09-13T07:04:57.357224+0000","last_became_active":"2024-09-13T07:04:43.266020+0000","last_became_peered":"2024-09-13T07:04:43.266020+0000","last_unstale":"2024-09-13T07:04:57.357224+0000","last_undegraded":"2024-09-13T07:04:57.357224+0000","last_fullsized":"2024-09-13T07:04:57.357224+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:26.181412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"199'459","reported_seq":3615,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322391+0000","last_change":"2024-09-13T07:04:49.126459+0000","last_active":"2024-09-13T07:04:57.322391+0000","last_peered":"2024-09-13T07:04:57.322391+0000","last_clean":"2024-09-13T07:04:57.322391+0000","last_became_active":"2024-09-13T07:04:49.125105+0000","last_became_peered":"2024-09-13T07:04:49.125105+0000","last_unstale":"2024-09-13T07:04:57.322391+0000","last_undegraded":"2024-09-13T07:04:57.322391+0000","last_fullsized":"2024-09-13T07:04:57.322391+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:44:15.141007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066392000000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2852,"num_read_kb":36914,"num_write":449,"num_write_kb":17878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4764805,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318965+0000","last_change":"2024-09-13T07:04:46.127776+0000","last_active":"2024-09-13T07:04:57.318965+0000","last_peered":"2024-09-13T07:04:57.318965+0000","last_clean":"2024-09-13T07:04:57.318965+0000","last_became_active":"2024-09-13T07:04:46.127497+0000","last_became_peered":"2024-09-13T07:04:46.127497+0000","last_unstale":"2024-09-13T07:04:57.318965+0000","last_undegraded":"2024-09-13T07:04:57.318965+0000","last_fullsized":"2024-09-13T07:04:57.318965+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:32.313060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"198'599","reported_seq":4770,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317513+0000","last_change":"2024-09-13T07:04:46.128247+0000","last_active":"2024-09-13T07:04:57.317513+0000","last_peered":"2024-09-13T07:04:57.317513+0000","last_clean":"2024-09-13T07:04:57.317513+0000","last_became_active":"2024-09-13T07:04:46.126668+0000","last_became_peered":"2024-09-13T07:04:46.126668+0000","last_unstale":"2024-09-13T07:04:57.317513+0000","last_undegraded":"2024-09-13T07:04:57.317513+0000","last_fullsized":"2024-09-13T07:04:57.317513+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:46:46.209389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001721534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5009,"num_read_kb":48072,"num_write":666,"num_write_kb":15004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":128514,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800745+0000","last_change":"2024-09-13T07:04:57.327360+0000","last_active":"2024-09-13T07:04:58.800745+0000","last_peered":"2024-09-13T07:04:58.800745+0000","last_clean":"2024-09-13T07:04:58.800745+0000","last_became_active":"2024-09-13T07:04:57.325955+0000","last_became_peered":"2024-09-13T07:04:57.325955+0000","last_unstale":"2024-09-13T07:04:58.800745+0000","last_undegraded":"2024-09-13T07:04:58.800745+0000","last_fullsized":"2024-09-13T07:04:58.800745+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:36:35.543692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"202'377","reported_seq":1798,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318846+0000","last_change":"2024-09-13T07:04:49.134895+0000","last_active":"2024-09-13T07:04:57.318846+0000","last_peered":"2024-09-13T07:04:57.318846+0000","last_clean":"2024-09-13T07:04:57.318846+0000","last_became_active":"2024-09-13T07:04:49.132150+0000","last_became_peered":"2024-09-13T07:04:49.132150+0000","last_unstale":"2024-09-13T07:04:57.318846+0000","last_undegraded":"2024-09-13T07:04:57.318846+0000","last_fullsized":"2024-09-13T07:04:57.318846+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:54:57.301919+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.088725134999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1176,"num_read_kb":23362,"num_write":393,"num_write_kb":13842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":2874205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318745+0000","last_change":"2024-09-13T07:04:46.129346+0000","last_active":"2024-09-13T07:04:57.318745+0000","last_peered":"2024-09-13T07:04:57.318745+0000","last_clean":"2024-09-13T07:04:57.318745+0000","last_became_active":"2024-09-13T07:04:46.129069+0000","last_became_peered":"2024-09-13T07:04:46.129069+0000","last_unstale":"2024-09-13T07:04:57.318745+0000","last_undegraded":"2024-09-13T07:04:57.318745+0000","last_fullsized":"2024-09-13T07:04:57.318745+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:36.800953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'708","reported_seq":5274,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322759+0000","last_change":"2024-09-13T07:04:38.491014+0000","last_active":"2024-09-13T07:04:57.322759+0000","last_peered":"2024-09-13T07:04:57.322759+0000","last_clean":"2024-09-13T07:04:57.322759+0000","last_became_active":"2024-09-13T07:04:38.218110+0000","last_became_peered":"2024-09-13T07:04:38.218110+0000","last_unstale":"2024-09-13T07:04:57.322759+0000","last_undegraded":"2024-09-13T07:04:57.322759+0000","last_fullsized":"2024-09-13T07:04:57.322759+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:23:34.445526+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.101155064,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6847,"num_read_kb":24040,"num_write":729,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"199'610","reported_seq":6642,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322366+0000","last_change":"2024-09-13T07:04:34.193235+0000","last_active":"2024-09-13T07:04:57.322366+0000","last_peered":"2024-09-13T07:04:57.322366+0000","last_clean":"2024-09-13T07:04:57.322366+0000","last_became_active":"2024-09-13T07:03:06.744590+0000","last_became_peered":"2024-09-13T07:03:06.744590+0000","last_unstale":"2024-09-13T07:04:57.322366+0000","last_undegraded":"2024-09-13T07:04:57.322366+0000","last_fullsized":"2024-09-13T07:04:57.322366+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:59:25.982967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006933257,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":6314,"num_read_kb":32619,"num_write":703,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":11860709,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":175,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322603+0000","last_change":"2024-09-13T07:04:46.408868+0000","last_active":"2024-09-13T07:04:57.322603+0000","last_peered":"2024-09-13T07:04:57.322603+0000","last_clean":"2024-09-13T07:04:57.322603+0000","last_became_active":"2024-09-13T07:04:46.125291+0000","last_became_peered":"2024-09-13T07:04:46.125291+0000","last_unstale":"2024-09-13T07:04:57.322603+0000","last_undegraded":"2024-09-13T07:04:57.322603+0000","last_fullsized":"2024-09-13T07:04:57.322603+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:50:37.881225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"197'615","reported_seq":14047,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318699+0000","last_change":"2024-09-13T07:04:50.127801+0000","last_active":"2024-09-13T07:04:57.318699+0000","last_peered":"2024-09-13T07:04:57.318699+0000","last_clean":"2024-09-13T07:04:57.318699+0000","last_became_active":"2024-09-13T07:04:50.122952+0000","last_became_peered":"2024-09-13T07:04:50.122952+0000","last_unstale":"2024-09-13T07:04:57.318699+0000","last_undegraded":"2024-09-13T07:04:57.318699+0000","last_fullsized":"2024-09-13T07:04:57.318699+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:19.383007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084358800000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13923,"num_read_kb":43420,"num_write":574,"num_write_kb":22169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":8978985,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.875181+0000","last_change":"2024-09-13T07:02:44.091196+0000","last_active":"2024-09-13T07:04:57.875181+0000","last_peered":"2024-09-13T07:04:57.875181+0000","last_clean":"2024-09-13T07:04:57.875181+0000","last_became_active":"2024-09-13T07:02:44.090715+0000","last_became_peered":"2024-09-13T07:02:44.090715+0000","last_unstale":"2024-09-13T07:04:57.875181+0000","last_undegraded":"2024-09-13T07:04:57.875181+0000","last_fullsized":"2024-09-13T07:04:57.875181+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:08.303433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"202'376","reported_seq":7117,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322070+0000","last_change":"2024-09-13T07:04:54.130710+0000","last_active":"2024-09-13T07:04:57.322070+0000","last_peered":"2024-09-13T07:04:57.322070+0000","last_clean":"2024-09-13T07:04:57.322070+0000","last_became_active":"2024-09-13T07:04:54.129673+0000","last_became_peered":"2024-09-13T07:04:54.129673+0000","last_unstale":"2024-09-13T07:04:57.322070+0000","last_undegraded":"2024-09-13T07:04:57.322070+0000","last_fullsized":"2024-09-13T07:04:57.322070+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:42.016575+0000","scrub_duration":0,"objects_trimmed":26,"snaptrim_duration":0.064131435000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6959,"num_read_kb":35900,"num_write":367,"num_write_kb":8191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":3448599,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800247+0000","last_change":"2024-09-13T07:04:57.320659+0000","last_active":"2024-09-13T07:04:58.800247+0000","last_peered":"2024-09-13T07:04:58.800247+0000","last_clean":"2024-09-13T07:04:58.800247+0000","last_became_active":"2024-09-13T07:04:57.320044+0000","last_became_peered":"2024-09-13T07:04:57.320044+0000","last_unstale":"2024-09-13T07:04:58.800247+0000","last_undegraded":"2024-09-13T07:04:58.800247+0000","last_fullsized":"2024-09-13T07:04:58.800247+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:31.189059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"205'511","reported_seq":4274,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317039+0000","last_change":"2024-09-13T07:04:51.127015+0000","last_active":"2024-09-13T07:04:57.317039+0000","last_peered":"2024-09-13T07:04:57.317039+0000","last_clean":"2024-09-13T07:04:57.317039+0000","last_became_active":"2024-09-13T07:04:51.124840+0000","last_became_peered":"2024-09-13T07:04:51.124840+0000","last_unstale":"2024-09-13T07:04:57.317039+0000","last_undegraded":"2024-09-13T07:04:57.317039+0000","last_fullsized":"2024-09-13T07:04:57.317039+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:38:08.467746+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.071371410999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6314,"num_read_kb":38281,"num_write":477,"num_write_kb":24168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":195,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321960+0000","last_change":"2024-09-13T07:04:38.219981+0000","last_active":"2024-09-13T07:04:57.321960+0000","last_peered":"2024-09-13T07:04:57.321960+0000","last_clean":"2024-09-13T07:04:57.321960+0000","last_became_active":"2024-09-13T07:04:38.218537+0000","last_became_peered":"2024-09-13T07:04:38.218537+0000","last_unstale":"2024-09-13T07:04:57.321960+0000","last_undegraded":"2024-09-13T07:04:57.321960+0000","last_fullsized":"2024-09-13T07:04:57.321960+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:49:58.271696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":343,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800461+0000","last_change":"2024-09-13T07:04:57.325982+0000","last_active":"2024-09-13T07:04:58.800461+0000","last_peered":"2024-09-13T07:04:58.800461+0000","last_clean":"2024-09-13T07:04:58.800461+0000","last_became_active":"2024-09-13T07:04:57.324203+0000","last_became_peered":"2024-09-13T07:04:57.324203+0000","last_unstale":"2024-09-13T07:04:58.800461+0000","last_undegraded":"2024-09-13T07:04:58.800461+0000","last_fullsized":"2024-09-13T07:04:58.800461+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:01.766156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"198'782","reported_seq":5552,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316797+0000","last_change":"2024-09-13T07:04:53.134367+0000","last_active":"2024-09-13T07:04:57.316797+0000","last_peered":"2024-09-13T07:04:57.316797+0000","last_clean":"2024-09-13T07:04:57.316797+0000","last_became_active":"2024-09-13T07:04:53.131823+0000","last_became_peered":"2024-09-13T07:04:53.131823+0000","last_unstale":"2024-09-13T07:04:57.316797+0000","last_undegraded":"2024-09-13T07:04:57.316797+0000","last_fullsized":"2024-09-13T07:04:57.316797+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:32:42.148081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032724400000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5361,"num_read_kb":39438,"num_write":976,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":861185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"192'432","reported_seq":9628,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322616+0000","last_change":"2024-09-13T07:04:34.192006+0000","last_active":"2024-09-13T07:04:57.322616+0000","last_peered":"2024-09-13T07:04:57.322616+0000","last_clean":"2024-09-13T07:04:57.322616+0000","last_became_active":"2024-09-13T07:02:47.511257+0000","last_became_peered":"2024-09-13T07:02:47.511257+0000","last_unstale":"2024-09-13T07:04:57.322616+0000","last_undegraded":"2024-09-13T07:04:57.322616+0000","last_fullsized":"2024-09-13T07:04:57.322616+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:04.038572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059111220000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9849,"num_read_kb":28359,"num_write":451,"num_write_kb":30112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5191019,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"198'996","reported_seq":4549,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357116+0000","last_change":"2024-09-13T07:04:34.657476+0000","last_active":"2024-09-13T07:04:57.357116+0000","last_peered":"2024-09-13T07:04:57.357116+0000","last_clean":"2024-09-13T07:04:57.357116+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:04:57.357116+0000","last_undegraded":"2024-09-13T07:04:57.357116+0000","last_fullsized":"2024-09-13T07:04:57.357116+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":996,"log_dups_size":0,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:11.537644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016753040000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7385,"num_read_kb":27871,"num_write":1272,"num_write_kb":19033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"198'599","reported_seq":7645,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316852+0000","last_change":"2024-09-13T07:04:51.128619+0000","last_active":"2024-09-13T07:04:57.316852+0000","last_peered":"2024-09-13T07:04:57.316852+0000","last_clean":"2024-09-13T07:04:57.316852+0000","last_became_active":"2024-09-13T07:04:51.126975+0000","last_became_peered":"2024-09-13T07:04:51.126975+0000","last_unstale":"2024-09-13T07:04:57.316852+0000","last_undegraded":"2024-09-13T07:04:57.316852+0000","last_fullsized":"2024-09-13T07:04:57.316852+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:29.230850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020303249999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9098,"num_read_kb":32181,"num_write":560,"num_write_kb":27201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":10029829,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"205'1700","reported_seq":13975,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322182+0000","last_change":"2024-09-13T07:04:41.334599+0000","last_active":"2024-09-13T07:04:57.322182+0000","last_peered":"2024-09-13T07:04:57.322182+0000","last_clean":"2024-09-13T07:04:57.322182+0000","last_became_active":"2024-09-13T07:04:41.239873+0000","last_became_peered":"2024-09-13T07:04:41.239873+0000","last_unstale":"2024-09-13T07:04:57.322182+0000","last_undegraded":"2024-09-13T07:04:57.322182+0000","last_fullsized":"2024-09-13T07:04:57.322182+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":1700,"log_dups_size":0,"ondisk_log_size":1700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:18.172446+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.091557893000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21059,"num_read_kb":55383,"num_write":2606,"num_write_kb":25105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"198'662","reported_seq":6936,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800373+0000","last_change":"2024-09-13T07:04:57.326646+0000","last_active":"2024-09-13T07:04:58.800373+0000","last_peered":"2024-09-13T07:04:58.800373+0000","last_clean":"2024-09-13T07:04:58.800373+0000","last_became_active":"2024-09-13T07:04:57.323373+0000","last_became_peered":"2024-09-13T07:04:57.323373+0000","last_unstale":"2024-09-13T07:04:58.800373+0000","last_undegraded":"2024-09-13T07:04:58.800373+0000","last_fullsized":"2024-09-13T07:04:58.800373+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:47:35.380836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002404799,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8026,"num_read_kb":31428,"num_write":564,"num_write_kb":31570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9245169,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"198'561","reported_seq":3683,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800239+0000","last_change":"2024-09-13T07:04:57.329473+0000","last_active":"2024-09-13T07:04:58.800239+0000","last_peered":"2024-09-13T07:04:58.800239+0000","last_clean":"2024-09-13T07:04:58.800239+0000","last_became_active":"2024-09-13T07:04:57.325924+0000","last_became_peered":"2024-09-13T07:04:57.325924+0000","last_unstale":"2024-09-13T07:04:58.800239+0000","last_undegraded":"2024-09-13T07:04:58.800239+0000","last_fullsized":"2024-09-13T07:04:58.800239+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:09.975816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086238349999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4232,"num_read_kb":25664,"num_write":732,"num_write_kb":10506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1357199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"198'909","reported_seq":3819,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800888+0000","last_change":"2024-09-13T07:04:57.330501+0000","last_active":"2024-09-13T07:04:58.800888+0000","last_peered":"2024-09-13T07:04:58.800888+0000","last_clean":"2024-09-13T07:04:58.800888+0000","last_became_active":"2024-09-13T07:04:57.326822+0000","last_became_peered":"2024-09-13T07:04:57.326822+0000","last_unstale":"2024-09-13T07:04:58.800888+0000","last_undegraded":"2024-09-13T07:04:58.800888+0000","last_fullsized":"2024-09-13T07:04:58.800888+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:47:41.007573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075911479999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5743,"num_read_kb":24275,"num_write":1279,"num_write_kb":27074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":68557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"198'419","reported_seq":2601,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357197+0000","last_change":"2024-09-13T07:04:34.657069+0000","last_active":"2024-09-13T07:04:57.357197+0000","last_peered":"2024-09-13T07:04:57.357197+0000","last_clean":"2024-09-13T07:04:57.357197+0000","last_became_active":"2024-09-13T07:03:06.743141+0000","last_became_peered":"2024-09-13T07:03:06.743141+0000","last_unstale":"2024-09-13T07:04:57.357197+0000","last_undegraded":"2024-09-13T07:04:57.357197+0000","last_fullsized":"2024-09-13T07:04:57.357197+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:03:42.072271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012751349999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2106,"num_read_kb":12966,"num_write":446,"num_write_kb":14235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":10481695,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"198'592","reported_seq":8333,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800670+0000","last_change":"2024-09-13T07:04:57.330363+0000","last_active":"2024-09-13T07:04:58.800670+0000","last_peered":"2024-09-13T07:04:58.800670+0000","last_clean":"2024-09-13T07:04:58.800670+0000","last_became_active":"2024-09-13T07:04:57.325398+0000","last_became_peered":"2024-09-13T07:04:57.325398+0000","last_unstale":"2024-09-13T07:04:58.800670+0000","last_undegraded":"2024-09-13T07:04:58.800670+0000","last_fullsized":"2024-09-13T07:04:58.800670+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:57.680683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046356670000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8997,"num_read_kb":22165,"num_write":633,"num_write_kb":13453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":102776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"198'927","reported_seq":9117,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322084+0000","last_change":"2024-09-13T07:04:47.132559+0000","last_active":"2024-09-13T07:04:57.322084+0000","last_peered":"2024-09-13T07:04:57.322084+0000","last_clean":"2024-09-13T07:04:57.322084+0000","last_became_active":"2024-09-13T07:04:47.130438+0000","last_became_peered":"2024-09-13T07:04:47.130438+0000","last_unstale":"2024-09-13T07:04:57.322084+0000","last_undegraded":"2024-09-13T07:04:57.322084+0000","last_fullsized":"2024-09-13T07:04:57.322084+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:47:15.123470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076933,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9860,"num_read_kb":45806,"num_write":1253,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":5924852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"199'588","reported_seq":4689,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800492+0000","last_change":"2024-09-13T07:04:57.329853+0000","last_active":"2024-09-13T07:04:58.800492+0000","last_peered":"2024-09-13T07:04:58.800492+0000","last_clean":"2024-09-13T07:04:58.800492+0000","last_became_active":"2024-09-13T07:04:57.325425+0000","last_became_peered":"2024-09-13T07:04:57.325425+0000","last_unstale":"2024-09-13T07:04:58.800492+0000","last_undegraded":"2024-09-13T07:04:58.800492+0000","last_fullsized":"2024-09-13T07:04:58.800492+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:05.096848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051571560000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5130,"num_read_kb":30367,"num_write":654,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":12828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":408,"num_object_clones":0,"num_object_copies":816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":408,"num_whiteouts":404,"num_read":239446,"num_read_kb":1058874,"num_write":24086,"num_write_kb":567966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5260,"num_bytes_recovered":115290013,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},"log_size":20738,"ondisk_log_size":20738,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78604,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293236,"statfs":{"total":96636764160,"available":96556273664,"internally_reserved":0,"allocated":3276800,"data_stored":1887770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77199819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71399999999999997,"15min":0.70099999999999996},"min":{"1min":0.40500000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1870000000000001,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":0.88200000000000001,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":12.583,"5min":12.583,"15min":12.583},"last":0.80700000000000005}]},{"osd":1,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76000000000000001,"15min":0.73399999999999999},"min":{"1min":0.41299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.552,"5min":2.016,"15min":2.016},"last":0.871},{"interface":"front","average":{"1min":1.5089999999999999,"5min":0.99199999999999999,"15min":0.92800000000000005},"min":{"1min":0.56599999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":12.481,"5min":12.481,"15min":12.481},"last":0.96399999999999997}]},{"osd":2,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.88900000000000001,"15min":0.871},"min":{"1min":0.46200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85699999999999998,"15min":0.873},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.2330000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.91200000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2509999999999999,"5min":2.379,"15min":2.379},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.92700000000000005,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6220000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0489999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.048,"15min":0.96999999999999997},"min":{"1min":0.46300000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":12.622,"5min":12.622,"15min":12.622},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.98799999999999999,"15min":1.0009999999999999},"min":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2729999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.127}]},{"osd":5,"last update":"Fri Sep 13 07:03:03 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0129999999999999,"15min":1.0309999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.427,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.024,"15min":1.038},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3069999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.0880000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.198,"15min":1.121},"min":{"1min":0.64700000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":12.555999999999999,"5min":12.555999999999999,"15min":12.555999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.099,"15min":1.1020000000000001},"min":{"1min":0.72499999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.1890000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33740,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338100,"statfs":{"total":96636764160,"available":96602214400,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.84299999999999997,"15min":0.875},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.262,"5min":1.538,"15min":1.538},"last":0.871}]},{"osd":1,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.327,"5min":1.484,"15min":1.484},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.83799999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78200000000000003,"15min":0.81200000000000006},"min":{"1min":0.51800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.96899999999999997,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.88800000000000001,"15min":0.93000000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2290000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.95499999999999996}]},{"osd":3,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.92200000000000004,"15min":0.94899999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3799999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.94699999999999995,"15min":0.94599999999999995},"min":{"1min":0.55000000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.98099999999999998,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":1.036}]},{"osd":4,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88800000000000001,"15min":0.89300000000000002},"min":{"1min":0.47499999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1539999999999999,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.98699999999999999,"15min":1.0189999999999999},"min":{"1min":0.45100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.29,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.141,"5min":1.579,"15min":1.579},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0349999999999999,"15min":1.0549999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3420000000000001,"5min":1.857,"15min":1.857},"last":1.091}]},{"osd":7,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.056,"15min":1.1040000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3080000000000001,"5min":1.607,"15min":1.607},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.0429999999999999,"15min":1.093},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2450000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.98299999999999998}]}]},{"osd":5,"up_from":201,"seq":863288426503,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35732,"kb_used_data":2628,"kb_used_omap":35,"kb_used_meta":33052,"kb_avail":94336108,"statfs":{"total":96636764160,"available":96600174592,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36852,"internal_metadata":33845260},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69840,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302000,"statfs":{"total":96636764160,"available":96565248000,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.40600000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.99099999999999999,"15min":0.96299999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.1479999999999999,"5min":8.2089999999999996,"15min":8.2089999999999996},"last":0.95399999999999996}]},{"osd":1,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.93100000000000005,"15min":0.81999999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0549999999999999,"5min":10.932,"15min":10.932},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.84999999999999998,"15min":0.873},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.101,"5min":2.101,"15min":2.101},"last":1.123}]},{"osd":2,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":0.91400000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":5.9359999999999999,"5min":12.968999999999999,"15min":12.968999999999999},"last":1.0760000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.89200000000000002,"15min":0.91700000000000004},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.26},{"interface":"front","average":{"1min":0.754,"5min":0.80700000000000005,"15min":0.82099999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97899999999999998,"5min":1.159,"15min":1.159},"last":0.84099999999999997}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":0.95499999999999996,"15min":0.873},"min":{"1min":0.56899999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":8.2550000000000008,"5min":8.2550000000000008,"15min":8.2550000000000008},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.198,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.1519999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.83899999999999997,"15min":0.86499999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.123,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0960000000000001,"15min":1.03},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1240000000000001,"5min":10.885,"15min":10.885},"last":1.224}]},{"osd":7,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.92500000000000004,"15min":0.93200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.92800000000000005,"15min":0.95899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1830000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.3129999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51788,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320052,"statfs":{"total":96636764160,"available":96583733248,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.34399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.161,"5min":1.581,"15min":1.581},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.38900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.0049999999999999}]},{"osd":1,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.72699999999999998},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.143,"5min":1.577,"15min":1.577},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.073,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.872}]},{"osd":2,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.026,"5min":1.145,"15min":1.145},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2430000000000001,"5min":1.375,"15min":1.375},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92100000000000004,"15min":0.93400000000000005},"min":{"1min":0.57199999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.466,"5min":1.488,"15min":1.488},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.94299999999999995,"15min":0.95699999999999996},"min":{"1min":0.64600000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3340000000000001,"5min":1.512,"15min":1.512},"last":0.97899999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3149999999999999,"5min":1.369,"15min":1.369},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.002,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4299999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.1359999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.008,"15min":1.022},"min":{"1min":0.63800000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.04},{"interface":"front","average":{"1min":0.998,"5min":1.0169999999999999,"15min":1.0249999999999999},"min":{"1min":0.69499999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.0720000000000001}]},{"osd":7,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.0960000000000001,"15min":1.0920000000000001},"min":{"1min":0.73999999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7549999999999999,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.0109999999999999,"15min":0.99199999999999999},"min":{"1min":0.68100000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.1919999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33748,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338092,"statfs":{"total":96636764160,"available":96602206208,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.91600000000000004,"15min":1.018},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2070000000000001,"5min":6.867,"15min":6.867},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.66700000000000004},"min":{"1min":0.45000000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70699999999999996,"15min":0.71499999999999997},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":1.0009999999999999,"5min":1.234,"15min":1.234},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79600000000000004,"15min":0.79000000000000004},"min":{"1min":0.46100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.306,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93200000000000005,"15min":0.94399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.016,"5min":0.999,"15min":1.006},"min":{"1min":0.70799999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.631,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.88}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.97099999999999997,"15min":0.96899999999999997},"min":{"1min":0.58299999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1,"15min":1.004},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.4670000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.018,"15min":1.0109999999999999},"min":{"1min":0.68899999999999995,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4690000000000001,"5min":2.109,"15min":2.109},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.03,"5min":1.0680000000000001,"15min":1.0760000000000001},"min":{"1min":0.61499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4330000000000001,"5min":1.748,"15min":1.748},"last":0.746}]},{"osd":7,"last update":"Fri Sep 13 07:04:03 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0369999999999999,"15min":1.0349999999999999},"min":{"1min":0.745,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.036,"15min":1.038},"min":{"1min":0.627,"5min":0.505,"15min":0.505},"max":{"1min":1.4610000000000001,"5min":1.7,"15min":1.7},"last":0.93400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29648,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342192,"statfs":{"total":96636764160,"available":96606404608,"internally_reserved":0,"allocated":2691072,"data_stored":1296855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.34499999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.0960000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65500000000000003,"15min":0.64500000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.92300000000000004}]},{"osd":2,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.47599999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.70599999999999996,"15min":0.68600000000000005},"min":{"1min":0.41599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.79300000000000004}]},{"osd":3,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.751,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.157,"5min":1.171,"15min":1.171},"last":1.034},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.73499999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.0089999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90500000000000003,"15min":0.89700000000000002},"min":{"1min":0.53500000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.474,"5min":1.925,"15min":1.925},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.008,"5min":1.1599999999999999,"15min":1.0049999999999999},"min":{"1min":0.69199999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.319,"5min":13.701000000000001,"15min":13.701000000000001},"last":1.119}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.006,"15min":1.01},"min":{"1min":0.66400000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3400000000000001,"5min":1.48,"15min":1.48},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.98899999999999999,"15min":0.995},"min":{"1min":0.624,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.409,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.1919999999999999}]},{"osd":6,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0129999999999999,"15min":1.014},"min":{"1min":0.56200000000000006,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2410000000000001,"5min":1.575,"15min":1.575},"last":1.258},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0509999999999999,"15min":1.046},"min":{"1min":0.66000000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.544}]},{"osd":7,"last update":"Fri Sep 13 07:04:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.633,"15min":1.282},"min":{"1min":0.73399999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.73,"5min":59.652000000000001,"15min":59.652000000000001},"last":1.415},{"interface":"front","average":{"1min":1.028,"5min":1.589,"15min":1.2390000000000001},"min":{"1min":0.64400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.423,"5min":59.679000000000002,"15min":59.679000000000002},"last":1.153}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38416,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333424,"statfs":{"total":96636764160,"available":96597426176,"internally_reserved":0,"allocated":3276800,"data_stored":1887185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.30199999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.548,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.33700000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.629}]},{"osd":2,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.35299999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.1399999999999999,"5min":1.629,"15min":1.629},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78900000000000003,"15min":0.80100000000000005},"min":{"1min":0.439,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2889999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.89100000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.038,"5min":1.635,"15min":1.635},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4710000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.86799999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.89900000000000002,"15min":0.93600000000000005},"min":{"1min":0.52800000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93300000000000005,"15min":0.94899999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3360000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.1020000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.02,"15min":1.034},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1870000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.873,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.254,"5min":1.724,"15min":1.724},"last":1.1180000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.0209999999999999,"15min":1.0429999999999999},"min":{"1min":0.50700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5469999999999999,"5min":1.552,"15min":1.552},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0509999999999999,"15min":1.077},"min":{"1min":0.55400000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.5780000000000001,"5min":2.024,"15min":2.024},"last":1.024}]},{"osd":7,"last update":"Fri Sep 13 07:04:04 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.079,"15min":1.093},"min":{"1min":0.59599999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5389999999999999,"5min":2,"15min":2},"last":1.073},{"interface":"front","average":{"1min":0.999,"5min":1.0900000000000001,"15min":1.1279999999999999},"min":{"1min":0.69099999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.389,"5min":1.744,"15min":1.744},"last":1.0029999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:05:04.680 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T07:05:04.982 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:05:04.982 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":222,"fsid":"25bb30d1-9541-4e40-8659-18d83bb1bd8a","created":"2024-09-13T07:00:49.323883+0000","modified":"2024-09-13T07:04:58.313385+0000","last_up_change":"2024-09-13T07:04:37.195137+0000","last_in_change":"2024-09-13T07:04:42.234060+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T07:01:00.675365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T07:01:07.404263+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T07:01:17.321011+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":103,"snap_epoch":199,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21736eee-00cd-4eef-9725-eca7776957b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6817","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6819","nonce":3731386737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6821","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6823","nonce":3731386737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6829","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6831","nonce":3731386737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6825","nonce":3731386737},{"type":"v1","addr":"172.21.15.118:6827","nonce":3731386737}]},"public_addr":"172.21.15.118:6819/3731386737","cluster_addr":"172.21.15.118:6823/3731386737","heartbeat_back_addr":"172.21.15.118:6831/3731386737","heartbeat_front_addr":"172.21.15.118:6827/3731386737","state":["exists","up"]},{"osd":1,"uuid":"05818db2-841d-406d-be17-8fbdab4fff69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":215,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6818","nonce":876310814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6822","nonce":876310814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6830","nonce":876310814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":876310814},{"type":"v1","addr":"172.21.15.118:6826","nonce":876310814}]},"public_addr":"172.21.15.118:6818/876310814","cluster_addr":"172.21.15.118:6822/876310814","heartbeat_back_addr":"172.21.15.118:6830/876310814","heartbeat_front_addr":"172.21.15.118:6826/876310814","state":["exists","up"]},{"osd":2,"uuid":"2387761c-42d2-481b-b5b5-b16d3a943b22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6809","nonce":690188762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6811","nonce":690188762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6815","nonce":690188762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":690188762},{"type":"v1","addr":"172.21.15.118:6813","nonce":690188762}]},"public_addr":"172.21.15.118:6809/690188762","cluster_addr":"172.21.15.118:6811/690188762","heartbeat_back_addr":"172.21.15.118:6815/690188762","heartbeat_front_addr":"172.21.15.118:6813/690188762","state":["exists","up"]},{"osd":3,"uuid":"56500e90-f76b-47c8-a649-893400359c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6801","nonce":2198175481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6803","nonce":2198175481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6807","nonce":2198175481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":2198175481},{"type":"v1","addr":"172.21.15.118:6805","nonce":2198175481}]},"public_addr":"172.21.15.118:6801/2198175481","cluster_addr":"172.21.15.118:6803/2198175481","heartbeat_back_addr":"172.21.15.118:6807/2198175481","heartbeat_front_addr":"172.21.15.118:6805/2198175481","state":["exists","up"]},{"osd":4,"uuid":"f594a612-aa07-4565-a09e-08368cf397b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6813","nonce":1835601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6819","nonce":1835601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6830","nonce":1835601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1835601651},{"type":"v1","addr":"172.21.15.164:6825","nonce":1835601651}]},"public_addr":"172.21.15.164:6813/1835601651","cluster_addr":"172.21.15.164:6819/1835601651","heartbeat_back_addr":"172.21.15.164:6830/1835601651","heartbeat_front_addr":"172.21.15.164:6825/1835601651","state":["exists","up"]},{"osd":5,"uuid":"1013b81f-e6b4-430b-938d-88759d43b881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":147,"up_from":201,"up_thru":210,"down_at":148,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2019781558},{"type":"v1","addr":"172.21.15.164:6801","nonce":2019781558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2019781558},{"type":"v1","addr":"172.21.15.164:6803","nonce":2019781558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2019781558},{"type":"v1","addr":"172.21.15.164:6807","nonce":2019781558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2019781558},{"type":"v1","addr":"172.21.15.164:6805","nonce":2019781558}]},"public_addr":"172.21.15.164:6801/2019781558","cluster_addr":"172.21.15.164:6803/2019781558","heartbeat_back_addr":"172.21.15.164:6807/2019781558","heartbeat_front_addr":"172.21.15.164:6805/2019781558","state":["exists","up"]},{"osd":6,"uuid":"367f5f5b-5919-4534-b88c-229f327e4950","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":206,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6812","nonce":131309660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6815","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6818","nonce":131309660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6831","nonce":131309660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":131309660},{"type":"v1","addr":"172.21.15.164:6824","nonce":131309660}]},"public_addr":"172.21.15.164:6812/131309660","cluster_addr":"172.21.15.164:6818/131309660","heartbeat_back_addr":"172.21.15.164:6831/131309660","heartbeat_front_addr":"172.21.15.164:6824/131309660","state":["exists","up"]},{"osd":7,"uuid":"90d9e486-e1ae-4305-80a6-6f483e534643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6811","nonce":652173699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6816","nonce":652173699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6829","nonce":652173699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":652173699},{"type":"v1","addr":"172.21.15.164:6823","nonce":652173699}]},"public_addr":"172.21.15.164:6811/652173699","cluster_addr":"172.21.15.164:6816/652173699","heartbeat_back_addr":"172.21.15.164:6829/652173699","heartbeat_front_addr":"172.21.15.164:6823/652173699","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.059233+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.083130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.848896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.847456+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.914135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:56.834284+0000","dead_epoch":148},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.003715+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T07:00:57.033514+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T07:05:05.998 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T07:05:05.998 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:05:06.162 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:05:06.162 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:05:06.162 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:05:06.175 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T07:05:06.485 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T07:05:06.500 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T07:05:06.500 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:05:06.649 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:05:06.649 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:05:06.649 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:05:06.662 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T07:05:06.970 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T07:05:06.981 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T07:05:06.981 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:05:07.129 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:05:07.129 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:05:07.129 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:05:07.143 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T07:05:07.449 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T07:05:07.460 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T07:05:07.460 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:05:07.606 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:05:07.606 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:05:07.606 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:05:07.619 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T07:05:07.927 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T07:05:07.940 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T07:05:07.940 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:05:08.097 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:05:08.097 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:05:08.097 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:05:08.110 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T07:05:08.417 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T07:05:08.428 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T07:05:08.428 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:05:08.577 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:05:08.577 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-13T07:05:08.577 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:05:08.591 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T07:05:08.905 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T07:05:08.916 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T07:05:08.916 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:05:09.066 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:05:09.066 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:05:09.067 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:05:09.080 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T07:05:09.386 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T07:05:09.403 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T07:05:09.403 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T07:05:09.536 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-09-13T07:05:09.537 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T07:05:09.537 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-09-13T07:05:09.550 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T07:05:09.852 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T07:05:09.863 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:05:10.174 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:05:10.176 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:05:10.188 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-13T07:05:08.665528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":412,"num_object_clones":0,"num_object_copies":824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":412,"num_whiteouts":404,"num_read":239562,"num_read_kb":1058974,"num_write":24247,"num_write_kb":570384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5266,"num_bytes_recovered":116470768,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20858,"ondisk_log_size":20858,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":371516,"kb_used_data":22156,"kb_used_omap":46,"kb_used_meta":349201,"kb_avail":754603204,"statfs":{"total":773094113280,"available":772713680896,"internally_reserved":0,"allocated":22687744,"data_stored":11560198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48002,"internal_metadata":357581950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-672,"num_bytes_recovered":-590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.348867"},"pg_stats":[{"pgid":"3.1f","version":"202'415","reported_seq":6819,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322192+0000","last_change":"2024-09-13T07:04:48.134119+0000","last_active":"2024-09-13T07:04:57.322192+0000","last_peered":"2024-09-13T07:04:57.322192+0000","last_clean":"2024-09-13T07:04:57.322192+0000","last_became_active":"2024-09-13T07:04:48.131331+0000","last_became_peered":"2024-09-13T07:04:48.131331+0000","last_unstale":"2024-09-13T07:04:57.322192+0000","last_undegraded":"2024-09-13T07:04:57.322192+0000","last_fullsized":"2024-09-13T07:04:57.322192+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:37:36.544267+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098858737000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6370,"num_read_kb":30920,"num_write":459,"num_write_kb":15662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":4960629,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"195'379","reported_seq":4118,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317289+0000","last_change":"2024-09-13T07:04:51.127712+0000","last_active":"2024-09-13T07:04:57.317289+0000","last_peered":"2024-09-13T07:04:57.317289+0000","last_clean":"2024-09-13T07:04:57.317289+0000","last_became_active":"2024-09-13T07:04:51.125903+0000","last_became_peered":"2024-09-13T07:04:51.125903+0000","last_unstale":"2024-09-13T07:04:57.317289+0000","last_undegraded":"2024-09-13T07:04:57.317289+0000","last_fullsized":"2024-09-13T07:04:57.317289+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:39.870655+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001432102,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4275,"num_read_kb":38815,"num_write":373,"num_write_kb":7930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"202'489","reported_seq":2580,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322555+0000","last_change":"2024-09-13T07:04:38.530429+0000","last_active":"2024-09-13T07:04:57.322555+0000","last_peered":"2024-09-13T07:04:57.322555+0000","last_clean":"2024-09-13T07:04:57.322555+0000","last_became_active":"2024-09-13T07:04:38.221697+0000","last_became_peered":"2024-09-13T07:04:38.221697+0000","last_unstale":"2024-09-13T07:04:57.322555+0000","last_undegraded":"2024-09-13T07:04:57.322555+0000","last_fullsized":"2024-09-13T07:04:57.322555+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:55.624774+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.073210949999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3758,"num_read_kb":13816,"num_write":504,"num_write_kb":5767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"202'500","reported_seq":5592,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321964+0000","last_change":"2024-09-13T07:04:38.656329+0000","last_active":"2024-09-13T07:04:57.321964+0000","last_peered":"2024-09-13T07:04:57.321964+0000","last_clean":"2024-09-13T07:04:57.321964+0000","last_became_active":"2024-09-13T07:04:38.225836+0000","last_became_peered":"2024-09-13T07:04:38.225836+0000","last_unstale":"2024-09-13T07:04:57.321964+0000","last_undegraded":"2024-09-13T07:04:57.321964+0000","last_fullsized":"2024-09-13T07:04:57.321964+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:40.102083+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.078226893000000006,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6179,"num_read_kb":24975,"num_write":551,"num_write_kb":10621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"198'381","reported_seq":6393,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322169+0000","last_change":"2024-09-13T07:04:48.130936+0000","last_active":"2024-09-13T07:04:57.322169+0000","last_peered":"2024-09-13T07:04:57.322169+0000","last_clean":"2024-09-13T07:04:57.322169+0000","last_became_active":"2024-09-13T07:04:48.128577+0000","last_became_peered":"2024-09-13T07:04:48.128577+0000","last_unstale":"2024-09-13T07:04:57.322169+0000","last_undegraded":"2024-09-13T07:04:57.322169+0000","last_fullsized":"2024-09-13T07:04:57.322169+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:54.041815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065506189999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6468,"num_read_kb":53020,"num_write":414,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":148740,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"198'1119","reported_seq":8612,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321787+0000","last_change":"2024-09-13T07:04:50.123312+0000","last_active":"2024-09-13T07:04:57.321787+0000","last_peered":"2024-09-13T07:04:57.321787+0000","last_clean":"2024-09-13T07:04:57.321787+0000","last_became_active":"2024-09-13T07:04:50.121731+0000","last_became_peered":"2024-09-13T07:04:50.121731+0000","last_unstale":"2024-09-13T07:04:57.321787+0000","last_undegraded":"2024-09-13T07:04:57.321787+0000","last_fullsized":"2024-09-13T07:04:57.321787+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":1119,"log_dups_size":0,"ondisk_log_size":1119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:03.606743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002988971,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":11548,"num_read_kb":34808,"num_write":1366,"num_write_kb":17528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":3976247,"num_keys_recovered":80,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"197'628","reported_seq":7696,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318581+0000","last_change":"2024-09-13T07:04:50.125951+0000","last_active":"2024-09-13T07:04:57.318581+0000","last_peered":"2024-09-13T07:04:57.318581+0000","last_clean":"2024-09-13T07:04:57.318581+0000","last_became_active":"2024-09-13T07:04:50.123234+0000","last_became_peered":"2024-09-13T07:04:50.123234+0000","last_unstale":"2024-09-13T07:04:57.318581+0000","last_undegraded":"2024-09-13T07:04:57.318581+0000","last_fullsized":"2024-09-13T07:04:57.318581+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:05.238196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001498135,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9140,"num_read_kb":20172,"num_write":660,"num_write_kb":17799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":1288675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"198'697","reported_seq":7807,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316898+0000","last_change":"2024-09-13T07:04:51.128278+0000","last_active":"2024-09-13T07:04:57.316898+0000","last_peered":"2024-09-13T07:04:57.316898+0000","last_clean":"2024-09-13T07:04:57.316898+0000","last_became_active":"2024-09-13T07:04:51.126595+0000","last_became_peered":"2024-09-13T07:04:51.126595+0000","last_unstale":"2024-09-13T07:04:57.316898+0000","last_undegraded":"2024-09-13T07:04:57.316898+0000","last_fullsized":"2024-09-13T07:04:57.316898+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:51:40.666326+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001501437,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8486,"num_read_kb":30532,"num_write":647,"num_write_kb":16391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":13265380,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"202'893","reported_seq":12728,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322291+0000","last_change":"2024-09-13T07:04:45.142416+0000","last_active":"2024-09-13T07:04:57.322291+0000","last_peered":"2024-09-13T07:04:57.322291+0000","last_clean":"2024-09-13T07:04:57.322291+0000","last_became_active":"2024-09-13T07:04:45.140362+0000","last_became_peered":"2024-09-13T07:04:45.140362+0000","last_unstale":"2024-09-13T07:04:57.322291+0000","last_undegraded":"2024-09-13T07:04:57.322291+0000","last_fullsized":"2024-09-13T07:04:57.322291+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:06:57.919834+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.082577740999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":15171,"num_read_kb":70673,"num_write":1118,"num_write_kb":35612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":8426052,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"198'847","reported_seq":12699,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.323110+0000","last_change":"2024-09-13T07:04:47.128907+0000","last_active":"2024-09-13T07:04:57.323110+0000","last_peered":"2024-09-13T07:04:57.323110+0000","last_clean":"2024-09-13T07:04:57.323110+0000","last_became_active":"2024-09-13T07:04:47.126092+0000","last_became_peered":"2024-09-13T07:04:47.126092+0000","last_unstale":"2024-09-13T07:04:57.323110+0000","last_undegraded":"2024-09-13T07:04:57.323110+0000","last_fullsized":"2024-09-13T07:04:57.323110+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:17:07.217834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075556199999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":14283,"num_read_kb":24697,"num_write":1275,"num_write_kb":8273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1104561,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"198'452","reported_seq":2662,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800574+0000","last_change":"2024-09-13T07:04:57.327607+0000","last_active":"2024-09-13T07:04:58.800574+0000","last_peered":"2024-09-13T07:04:58.800574+0000","last_clean":"2024-09-13T07:04:58.800574+0000","last_became_active":"2024-09-13T07:04:57.321478+0000","last_became_peered":"2024-09-13T07:04:57.321478+0000","last_unstale":"2024-09-13T07:04:58.800574+0000","last_undegraded":"2024-09-13T07:04:58.800574+0000","last_fullsized":"2024-09-13T07:04:58.800574+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:54:35.402390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001895851,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2526,"num_read_kb":26928,"num_write":433,"num_write_kb":12456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"198'516","reported_seq":5743,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321735+0000","last_change":"2024-09-13T07:04:46.128420+0000","last_active":"2024-09-13T07:04:57.321735+0000","last_peered":"2024-09-13T07:04:57.321735+0000","last_clean":"2024-09-13T07:04:57.321735+0000","last_became_active":"2024-09-13T07:04:46.126170+0000","last_became_peered":"2024-09-13T07:04:46.126170+0000","last_unstale":"2024-09-13T07:04:57.321735+0000","last_undegraded":"2024-09-13T07:04:57.321735+0000","last_fullsized":"2024-09-13T07:04:57.321735+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:41.697482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074505329999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5002,"num_read_kb":31007,"num_write":502,"num_write_kb":34041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":6788059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357224+0000","last_change":"2024-09-13T07:04:43.266461+0000","last_active":"2024-09-13T07:04:57.357224+0000","last_peered":"2024-09-13T07:04:57.357224+0000","last_clean":"2024-09-13T07:04:57.357224+0000","last_became_active":"2024-09-13T07:04:43.266020+0000","last_became_peered":"2024-09-13T07:04:43.266020+0000","last_unstale":"2024-09-13T07:04:57.357224+0000","last_undegraded":"2024-09-13T07:04:57.357224+0000","last_fullsized":"2024-09-13T07:04:57.357224+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:07:26.181412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"199'459","reported_seq":3615,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322391+0000","last_change":"2024-09-13T07:04:49.126459+0000","last_active":"2024-09-13T07:04:57.322391+0000","last_peered":"2024-09-13T07:04:57.322391+0000","last_clean":"2024-09-13T07:04:57.322391+0000","last_became_active":"2024-09-13T07:04:49.125105+0000","last_became_peered":"2024-09-13T07:04:49.125105+0000","last_unstale":"2024-09-13T07:04:57.322391+0000","last_undegraded":"2024-09-13T07:04:57.322391+0000","last_fullsized":"2024-09-13T07:04:57.322391+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:44:15.141007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066392000000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2852,"num_read_kb":36914,"num_write":449,"num_write_kb":17878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4764805,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318965+0000","last_change":"2024-09-13T07:04:46.127776+0000","last_active":"2024-09-13T07:04:57.318965+0000","last_peered":"2024-09-13T07:04:57.318965+0000","last_clean":"2024-09-13T07:04:57.318965+0000","last_became_active":"2024-09-13T07:04:46.127497+0000","last_became_peered":"2024-09-13T07:04:46.127497+0000","last_unstale":"2024-09-13T07:04:57.318965+0000","last_undegraded":"2024-09-13T07:04:57.318965+0000","last_fullsized":"2024-09-13T07:04:57.318965+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:32.313060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"198'599","reported_seq":4770,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317513+0000","last_change":"2024-09-13T07:04:46.128247+0000","last_active":"2024-09-13T07:04:57.317513+0000","last_peered":"2024-09-13T07:04:57.317513+0000","last_clean":"2024-09-13T07:04:57.317513+0000","last_became_active":"2024-09-13T07:04:46.126668+0000","last_became_peered":"2024-09-13T07:04:46.126668+0000","last_unstale":"2024-09-13T07:04:57.317513+0000","last_undegraded":"2024-09-13T07:04:57.317513+0000","last_fullsized":"2024-09-13T07:04:57.317513+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:46:46.209389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001721534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5009,"num_read_kb":48072,"num_write":666,"num_write_kb":15004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":128514,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800745+0000","last_change":"2024-09-13T07:04:57.327360+0000","last_active":"2024-09-13T07:04:58.800745+0000","last_peered":"2024-09-13T07:04:58.800745+0000","last_clean":"2024-09-13T07:04:58.800745+0000","last_became_active":"2024-09-13T07:04:57.325955+0000","last_became_peered":"2024-09-13T07:04:57.325955+0000","last_unstale":"2024-09-13T07:04:58.800745+0000","last_undegraded":"2024-09-13T07:04:58.800745+0000","last_fullsized":"2024-09-13T07:04:58.800745+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:36:35.543692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"202'377","reported_seq":1798,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318846+0000","last_change":"2024-09-13T07:04:49.134895+0000","last_active":"2024-09-13T07:04:57.318846+0000","last_peered":"2024-09-13T07:04:57.318846+0000","last_clean":"2024-09-13T07:04:57.318846+0000","last_became_active":"2024-09-13T07:04:49.132150+0000","last_became_peered":"2024-09-13T07:04:49.132150+0000","last_unstale":"2024-09-13T07:04:57.318846+0000","last_undegraded":"2024-09-13T07:04:57.318846+0000","last_fullsized":"2024-09-13T07:04:57.318846+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:54:57.301919+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.088725134999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1176,"num_read_kb":23362,"num_write":393,"num_write_kb":13842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":2874205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318745+0000","last_change":"2024-09-13T07:04:46.129346+0000","last_active":"2024-09-13T07:04:57.318745+0000","last_peered":"2024-09-13T07:04:57.318745+0000","last_clean":"2024-09-13T07:04:57.318745+0000","last_became_active":"2024-09-13T07:04:46.129069+0000","last_became_peered":"2024-09-13T07:04:46.129069+0000","last_unstale":"2024-09-13T07:04:57.318745+0000","last_undegraded":"2024-09-13T07:04:57.318745+0000","last_fullsized":"2024-09-13T07:04:57.318745+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:36.800953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'708","reported_seq":5274,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322759+0000","last_change":"2024-09-13T07:04:38.491014+0000","last_active":"2024-09-13T07:04:57.322759+0000","last_peered":"2024-09-13T07:04:57.322759+0000","last_clean":"2024-09-13T07:04:57.322759+0000","last_became_active":"2024-09-13T07:04:38.218110+0000","last_became_peered":"2024-09-13T07:04:38.218110+0000","last_unstale":"2024-09-13T07:04:57.322759+0000","last_undegraded":"2024-09-13T07:04:57.322759+0000","last_fullsized":"2024-09-13T07:04:57.322759+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:23:34.445526+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.101155064,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6847,"num_read_kb":24040,"num_write":729,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"199'610","reported_seq":6642,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322366+0000","last_change":"2024-09-13T07:04:34.193235+0000","last_active":"2024-09-13T07:04:57.322366+0000","last_peered":"2024-09-13T07:04:57.322366+0000","last_clean":"2024-09-13T07:04:57.322366+0000","last_became_active":"2024-09-13T07:03:06.744590+0000","last_became_peered":"2024-09-13T07:03:06.744590+0000","last_unstale":"2024-09-13T07:04:57.322366+0000","last_undegraded":"2024-09-13T07:04:57.322366+0000","last_fullsized":"2024-09-13T07:04:57.322366+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:59:25.982967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006933257,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":6314,"num_read_kb":32619,"num_write":703,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":11860709,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":175,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322603+0000","last_change":"2024-09-13T07:04:46.408868+0000","last_active":"2024-09-13T07:04:57.322603+0000","last_peered":"2024-09-13T07:04:57.322603+0000","last_clean":"2024-09-13T07:04:57.322603+0000","last_became_active":"2024-09-13T07:04:46.125291+0000","last_became_peered":"2024-09-13T07:04:46.125291+0000","last_unstale":"2024-09-13T07:04:57.322603+0000","last_undegraded":"2024-09-13T07:04:57.322603+0000","last_fullsized":"2024-09-13T07:04:57.322603+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:50:37.881225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"197'615","reported_seq":14047,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.318699+0000","last_change":"2024-09-13T07:04:50.127801+0000","last_active":"2024-09-13T07:04:57.318699+0000","last_peered":"2024-09-13T07:04:57.318699+0000","last_clean":"2024-09-13T07:04:57.318699+0000","last_became_active":"2024-09-13T07:04:50.122952+0000","last_became_peered":"2024-09-13T07:04:50.122952+0000","last_unstale":"2024-09-13T07:04:57.318699+0000","last_undegraded":"2024-09-13T07:04:57.318699+0000","last_fullsized":"2024-09-13T07:04:57.318699+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:46:19.383007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084358800000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13923,"num_read_kb":43420,"num_write":574,"num_write_kb":22169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":8978985,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.875181+0000","last_change":"2024-09-13T07:02:44.091196+0000","last_active":"2024-09-13T07:04:57.875181+0000","last_peered":"2024-09-13T07:04:57.875181+0000","last_clean":"2024-09-13T07:04:57.875181+0000","last_became_active":"2024-09-13T07:02:44.090715+0000","last_became_peered":"2024-09-13T07:02:44.090715+0000","last_unstale":"2024-09-13T07:04:57.875181+0000","last_undegraded":"2024-09-13T07:04:57.875181+0000","last_fullsized":"2024-09-13T07:04:57.875181+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:06:08.303433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"202'376","reported_seq":7117,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322070+0000","last_change":"2024-09-13T07:04:54.130710+0000","last_active":"2024-09-13T07:04:57.322070+0000","last_peered":"2024-09-13T07:04:57.322070+0000","last_clean":"2024-09-13T07:04:57.322070+0000","last_became_active":"2024-09-13T07:04:54.129673+0000","last_became_peered":"2024-09-13T07:04:54.129673+0000","last_unstale":"2024-09-13T07:04:57.322070+0000","last_undegraded":"2024-09-13T07:04:57.322070+0000","last_fullsized":"2024-09-13T07:04:57.322070+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:42.016575+0000","scrub_duration":0,"objects_trimmed":26,"snaptrim_duration":0.064131435000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6959,"num_read_kb":35900,"num_write":367,"num_write_kb":8191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":3448599,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800247+0000","last_change":"2024-09-13T07:04:57.320659+0000","last_active":"2024-09-13T07:04:58.800247+0000","last_peered":"2024-09-13T07:04:58.800247+0000","last_clean":"2024-09-13T07:04:58.800247+0000","last_became_active":"2024-09-13T07:04:57.320044+0000","last_became_peered":"2024-09-13T07:04:57.320044+0000","last_unstale":"2024-09-13T07:04:58.800247+0000","last_undegraded":"2024-09-13T07:04:58.800247+0000","last_fullsized":"2024-09-13T07:04:58.800247+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:31.189059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"205'511","reported_seq":4274,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.317039+0000","last_change":"2024-09-13T07:04:51.127015+0000","last_active":"2024-09-13T07:04:57.317039+0000","last_peered":"2024-09-13T07:04:57.317039+0000","last_clean":"2024-09-13T07:04:57.317039+0000","last_became_active":"2024-09-13T07:04:51.124840+0000","last_became_peered":"2024-09-13T07:04:51.124840+0000","last_unstale":"2024-09-13T07:04:57.317039+0000","last_undegraded":"2024-09-13T07:04:57.317039+0000","last_fullsized":"2024-09-13T07:04:57.317039+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:38:08.467746+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.071371410999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6314,"num_read_kb":38281,"num_write":477,"num_write_kb":24168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":195,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.321960+0000","last_change":"2024-09-13T07:04:38.219981+0000","last_active":"2024-09-13T07:04:57.321960+0000","last_peered":"2024-09-13T07:04:57.321960+0000","last_clean":"2024-09-13T07:04:57.321960+0000","last_became_active":"2024-09-13T07:04:38.218537+0000","last_became_peered":"2024-09-13T07:04:38.218537+0000","last_unstale":"2024-09-13T07:04:57.321960+0000","last_undegraded":"2024-09-13T07:04:57.321960+0000","last_fullsized":"2024-09-13T07:04:57.321960+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:08.184208+0000","last_clean_scrub_stamp":"2024-09-13T07:01:08.184208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:49:58.271696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":343,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800461+0000","last_change":"2024-09-13T07:04:57.325982+0000","last_active":"2024-09-13T07:04:58.800461+0000","last_peered":"2024-09-13T07:04:58.800461+0000","last_clean":"2024-09-13T07:04:58.800461+0000","last_became_active":"2024-09-13T07:04:57.324203+0000","last_became_peered":"2024-09-13T07:04:57.324203+0000","last_unstale":"2024-09-13T07:04:58.800461+0000","last_undegraded":"2024-09-13T07:04:58.800461+0000","last_fullsized":"2024-09-13T07:04:58.800461+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:01.149679+0000","last_clean_scrub_stamp":"2024-09-13T07:01:01.149679+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:01.766156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"198'782","reported_seq":5552,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316797+0000","last_change":"2024-09-13T07:04:53.134367+0000","last_active":"2024-09-13T07:04:57.316797+0000","last_peered":"2024-09-13T07:04:57.316797+0000","last_clean":"2024-09-13T07:04:57.316797+0000","last_became_active":"2024-09-13T07:04:53.131823+0000","last_became_peered":"2024-09-13T07:04:53.131823+0000","last_unstale":"2024-09-13T07:04:57.316797+0000","last_undegraded":"2024-09-13T07:04:57.316797+0000","last_fullsized":"2024-09-13T07:04:57.316797+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:32:42.148081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032724400000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5361,"num_read_kb":39438,"num_write":976,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":861185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"192'432","reported_seq":9628,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322616+0000","last_change":"2024-09-13T07:04:34.192006+0000","last_active":"2024-09-13T07:04:57.322616+0000","last_peered":"2024-09-13T07:04:57.322616+0000","last_clean":"2024-09-13T07:04:57.322616+0000","last_became_active":"2024-09-13T07:02:47.511257+0000","last_became_peered":"2024-09-13T07:02:47.511257+0000","last_unstale":"2024-09-13T07:04:57.322616+0000","last_undegraded":"2024-09-13T07:04:57.322616+0000","last_fullsized":"2024-09-13T07:04:57.322616+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:04.038572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059111220000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9849,"num_read_kb":28359,"num_write":451,"num_write_kb":30112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5191019,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"198'996","reported_seq":4549,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357116+0000","last_change":"2024-09-13T07:04:34.657476+0000","last_active":"2024-09-13T07:04:57.357116+0000","last_peered":"2024-09-13T07:04:57.357116+0000","last_clean":"2024-09-13T07:04:57.357116+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:04:57.357116+0000","last_undegraded":"2024-09-13T07:04:57.357116+0000","last_fullsized":"2024-09-13T07:04:57.357116+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":996,"log_dups_size":0,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:56:11.537644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016753040000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7385,"num_read_kb":27871,"num_write":1272,"num_write_kb":19033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"198'599","reported_seq":7645,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.316852+0000","last_change":"2024-09-13T07:04:51.128619+0000","last_active":"2024-09-13T07:04:57.316852+0000","last_peered":"2024-09-13T07:04:57.316852+0000","last_clean":"2024-09-13T07:04:57.316852+0000","last_became_active":"2024-09-13T07:04:51.126975+0000","last_became_peered":"2024-09-13T07:04:51.126975+0000","last_unstale":"2024-09-13T07:04:57.316852+0000","last_undegraded":"2024-09-13T07:04:57.316852+0000","last_fullsized":"2024-09-13T07:04:57.316852+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:38:29.230850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020303249999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9098,"num_read_kb":32181,"num_write":560,"num_write_kb":27201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":10029829,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"205'1700","reported_seq":13975,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322182+0000","last_change":"2024-09-13T07:04:41.334599+0000","last_active":"2024-09-13T07:04:57.322182+0000","last_peered":"2024-09-13T07:04:57.322182+0000","last_clean":"2024-09-13T07:04:57.322182+0000","last_became_active":"2024-09-13T07:04:41.239873+0000","last_became_peered":"2024-09-13T07:04:41.239873+0000","last_unstale":"2024-09-13T07:04:57.322182+0000","last_undegraded":"2024-09-13T07:04:57.322182+0000","last_fullsized":"2024-09-13T07:04:57.322182+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":1700,"log_dups_size":0,"ondisk_log_size":1700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:18.172446+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.091557893000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21059,"num_read_kb":55383,"num_write":2606,"num_write_kb":25105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"198'662","reported_seq":6936,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800373+0000","last_change":"2024-09-13T07:04:57.326646+0000","last_active":"2024-09-13T07:04:58.800373+0000","last_peered":"2024-09-13T07:04:58.800373+0000","last_clean":"2024-09-13T07:04:58.800373+0000","last_became_active":"2024-09-13T07:04:57.323373+0000","last_became_peered":"2024-09-13T07:04:57.323373+0000","last_unstale":"2024-09-13T07:04:58.800373+0000","last_undegraded":"2024-09-13T07:04:58.800373+0000","last_fullsized":"2024-09-13T07:04:58.800373+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:47:35.380836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002404799,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8026,"num_read_kb":31428,"num_write":564,"num_write_kb":31570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9245169,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"198'561","reported_seq":3683,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800239+0000","last_change":"2024-09-13T07:04:57.329473+0000","last_active":"2024-09-13T07:04:58.800239+0000","last_peered":"2024-09-13T07:04:58.800239+0000","last_clean":"2024-09-13T07:04:58.800239+0000","last_became_active":"2024-09-13T07:04:57.325924+0000","last_became_peered":"2024-09-13T07:04:57.325924+0000","last_unstale":"2024-09-13T07:04:58.800239+0000","last_undegraded":"2024-09-13T07:04:58.800239+0000","last_fullsized":"2024-09-13T07:04:58.800239+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:09.975816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086238349999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4232,"num_read_kb":25664,"num_write":732,"num_write_kb":10506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1357199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"198'909","reported_seq":3819,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800888+0000","last_change":"2024-09-13T07:04:57.330501+0000","last_active":"2024-09-13T07:04:58.800888+0000","last_peered":"2024-09-13T07:04:58.800888+0000","last_clean":"2024-09-13T07:04:58.800888+0000","last_became_active":"2024-09-13T07:04:57.326822+0000","last_became_peered":"2024-09-13T07:04:57.326822+0000","last_unstale":"2024-09-13T07:04:58.800888+0000","last_undegraded":"2024-09-13T07:04:58.800888+0000","last_fullsized":"2024-09-13T07:04:58.800888+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:47:41.007573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075911479999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5743,"num_read_kb":24275,"num_write":1279,"num_write_kb":27074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":68557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"198'419","reported_seq":2601,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.357197+0000","last_change":"2024-09-13T07:04:34.657069+0000","last_active":"2024-09-13T07:04:57.357197+0000","last_peered":"2024-09-13T07:04:57.357197+0000","last_clean":"2024-09-13T07:04:57.357197+0000","last_became_active":"2024-09-13T07:03:06.743141+0000","last_became_peered":"2024-09-13T07:03:06.743141+0000","last_unstale":"2024-09-13T07:04:57.357197+0000","last_undegraded":"2024-09-13T07:04:57.357197+0000","last_fullsized":"2024-09-13T07:04:57.357197+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:03:42.072271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012751349999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2106,"num_read_kb":12966,"num_write":446,"num_write_kb":14235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":10481695,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"198'592","reported_seq":8333,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800670+0000","last_change":"2024-09-13T07:04:57.330363+0000","last_active":"2024-09-13T07:04:58.800670+0000","last_peered":"2024-09-13T07:04:58.800670+0000","last_clean":"2024-09-13T07:04:58.800670+0000","last_became_active":"2024-09-13T07:04:57.325398+0000","last_became_peered":"2024-09-13T07:04:57.325398+0000","last_unstale":"2024-09-13T07:04:58.800670+0000","last_undegraded":"2024-09-13T07:04:58.800670+0000","last_fullsized":"2024-09-13T07:04:58.800670+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:57.680683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046356670000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8997,"num_read_kb":22165,"num_write":633,"num_write_kb":13453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":102776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"198'927","reported_seq":9117,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-13T07:04:57.322084+0000","last_change":"2024-09-13T07:04:47.132559+0000","last_active":"2024-09-13T07:04:57.322084+0000","last_peered":"2024-09-13T07:04:57.322084+0000","last_clean":"2024-09-13T07:04:57.322084+0000","last_became_active":"2024-09-13T07:04:47.130438+0000","last_became_peered":"2024-09-13T07:04:47.130438+0000","last_unstale":"2024-09-13T07:04:57.322084+0000","last_undegraded":"2024-09-13T07:04:57.322084+0000","last_fullsized":"2024-09-13T07:04:57.322084+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:47:15.123470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076933,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9860,"num_read_kb":45806,"num_write":1253,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":5924852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"199'588","reported_seq":4689,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.800492+0000","last_change":"2024-09-13T07:04:57.329853+0000","last_active":"2024-09-13T07:04:58.800492+0000","last_peered":"2024-09-13T07:04:58.800492+0000","last_clean":"2024-09-13T07:04:58.800492+0000","last_became_active":"2024-09-13T07:04:57.325425+0000","last_became_peered":"2024-09-13T07:04:57.325425+0000","last_unstale":"2024-09-13T07:04:58.800492+0000","last_undegraded":"2024-09-13T07:04:58.800492+0000","last_fullsized":"2024-09-13T07:04:58.800492+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:05.096848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051571560000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5130,"num_read_kb":30367,"num_write":654,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":12828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":408,"num_object_clones":0,"num_object_copies":816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":408,"num_whiteouts":404,"num_read":239446,"num_read_kb":1058874,"num_write":24086,"num_write_kb":567966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5260,"num_bytes_recovered":115290013,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},"log_size":20738,"ondisk_log_size":20738,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78604,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293236,"statfs":{"total":96636764160,"available":96556273664,"internally_reserved":0,"allocated":3276800,"data_stored":1887770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77199819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71399999999999997,"15min":0.70099999999999996},"min":{"1min":0.40500000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1870000000000001,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.5129999999999999,"5min":0.88200000000000001,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":12.583,"5min":12.583,"15min":12.583},"last":1.0720000000000001}]},{"osd":1,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76000000000000001,"15min":0.73399999999999999},"min":{"1min":0.41299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.552,"5min":2.016,"15min":2.016},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.5089999999999999,"5min":0.99199999999999999,"15min":0.92800000000000005},"min":{"1min":0.56599999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":12.481,"5min":12.481,"15min":12.481},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.88900000000000001,"15min":0.871},"min":{"1min":0.46200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":1.044},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85699999999999998,"15min":0.873},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.012}]},{"osd":3,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.91500000000000004,"15min":0.91200000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2509999999999999,"5min":2.379,"15min":2.379},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.92700000000000005,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6220000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.88800000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.048,"15min":0.96999999999999997},"min":{"1min":0.46300000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":12.622,"5min":12.622,"15min":12.622},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.98799999999999999,"15min":1.0009999999999999},"min":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2729999999999999,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.208}]},{"osd":5,"last update":"Fri Sep 13 07:03:03 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0129999999999999,"15min":1.0309999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.427,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.024,"15min":1.038},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3069999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.133}]},{"osd":6,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.198,"15min":1.121},"min":{"1min":0.64700000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":12.555999999999999,"5min":12.555999999999999,"15min":12.555999999999999},"last":1.157},{"interface":"front","average":{"1min":1.079,"5min":1.099,"15min":1.1020000000000001},"min":{"1min":0.72499999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.2509999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33740,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338100,"statfs":{"total":96636764160,"available":96602214400,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.84299999999999997,"15min":0.875},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.262,"5min":1.538,"15min":1.538},"last":0.871}]},{"osd":1,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.327,"5min":1.484,"15min":1.484},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.83799999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78200000000000003,"15min":0.81200000000000006},"min":{"1min":0.51800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.96899999999999997,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.88800000000000001,"15min":0.93000000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2290000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.95499999999999996}]},{"osd":3,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.92200000000000004,"15min":0.94899999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3799999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.94699999999999995,"15min":0.94599999999999995},"min":{"1min":0.55000000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.98099999999999998,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":1.036}]},{"osd":4,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88800000000000001,"15min":0.89300000000000002},"min":{"1min":0.47499999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1539999999999999,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.98699999999999999,"15min":1.0189999999999999},"min":{"1min":0.45100000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.29,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.141,"5min":1.579,"15min":1.579},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0349999999999999,"15min":1.0549999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3420000000000001,"5min":1.857,"15min":1.857},"last":1.091}]},{"osd":7,"last update":"Fri Sep 13 07:04:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.056,"15min":1.1040000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3080000000000001,"5min":1.607,"15min":1.607},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.0429999999999999,"15min":1.093},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2450000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.98299999999999998}]}]},{"osd":5,"up_from":201,"seq":863288426504,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35732,"kb_used_data":2628,"kb_used_omap":35,"kb_used_meta":33052,"kb_avail":94336108,"statfs":{"total":96636764160,"available":96600174592,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36852,"internal_metadata":33845260},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69840,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302000,"statfs":{"total":96636764160,"available":96565248000,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.40600000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.99099999999999999,"15min":0.96299999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.1479999999999999,"5min":8.2089999999999996,"15min":8.2089999999999996},"last":0.63100000000000001}]},{"osd":1,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.93100000000000005,"15min":0.81999999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0549999999999999,"5min":10.932,"15min":10.932},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.84999999999999998,"15min":0.873},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.101,"5min":2.101,"15min":2.101},"last":0.76900000000000002}]},{"osd":2,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":0.91400000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":5.9359999999999999,"5min":12.968999999999999,"15min":12.968999999999999},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.89200000000000002,"15min":0.91700000000000004},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.80700000000000005,"15min":0.82099999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97899999999999998,"5min":1.159,"15min":1.159},"last":0.92400000000000004}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":0.95499999999999996,"15min":0.873},"min":{"1min":0.56899999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":8.2550000000000008,"5min":8.2550000000000008,"15min":8.2550000000000008},"last":1.004},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.198,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.89900000000000002}]},{"osd":6,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.83899999999999997,"15min":0.86499999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.123,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0960000000000001,"15min":1.03},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1240000000000001,"5min":10.885,"15min":10.885},"last":0.97999999999999998}]},{"osd":7,"last update":"Fri Sep 13 07:04:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.92500000000000004,"15min":0.93200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.92800000000000005,"15min":0.95899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1830000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0289999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51788,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320052,"statfs":{"total":96636764160,"available":96583733248,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.34399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.161,"5min":1.581,"15min":1.581},"last":0.438},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.38900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.72699999999999998},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.143,"5min":1.577,"15min":1.577},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.073,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.026,"5min":1.145,"15min":1.145},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2430000000000001,"5min":1.375,"15min":1.375},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92100000000000004,"15min":0.93400000000000005},"min":{"1min":0.57199999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.466,"5min":1.488,"15min":1.488},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.94299999999999995,"15min":0.95699999999999996},"min":{"1min":0.64600000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3340000000000001,"5min":1.512,"15min":1.512},"last":1.0489999999999999}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3149999999999999,"5min":1.369,"15min":1.369},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.002,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4299999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.21}]},{"osd":6,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.008,"15min":1.022},"min":{"1min":0.63800000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.02},{"interface":"front","average":{"1min":0.998,"5min":1.0169999999999999,"15min":1.0249999999999999},"min":{"1min":0.69499999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.98299999999999998}]},{"osd":7,"last update":"Fri Sep 13 07:04:07 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.0960000000000001,"15min":1.0920000000000001},"min":{"1min":0.73999999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7549999999999999,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.0109999999999999,"15min":0.99199999999999999},"min":{"1min":0.68100000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.0800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33748,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338092,"statfs":{"total":96636764160,"available":96602206208,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.63800000000000001,"15min":0.64300000000000002},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.80100000000000005,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65500000000000003,"15min":0.63700000000000001},"min":{"1min":0.39700000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.81599999999999995,"15min":0.98399999999999999},"min":{"1min":0.38900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84899999999999998,"5min":6.867,"15min":6.867},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88400000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.82999999999999996}]},{"osd":3,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.69399999999999995,"15min":0.70999999999999996},"min":{"1min":0.51300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.92900000000000005,"5min":1.234,"15min":1.234},"last":0.63},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79700000000000004,"15min":0.79100000000000004},"min":{"1min":0.44900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.79800000000000004}]},{"osd":4,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91800000000000004,"15min":0.93999999999999995},"min":{"1min":0.64600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.98899999999999999,"15min":1.0029999999999999},"min":{"1min":0.57299999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.667,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.149}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.97099999999999997,"15min":0.96899999999999997},"min":{"1min":0.58299999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.995},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1,"15min":1.004},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.4670000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.071}]},{"osd":6,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.01,"15min":1.008},"min":{"1min":0.70899999999999996,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.611,"5min":2.109,"15min":2.109},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.004,"5min":1.0529999999999999,"15min":1.071},"min":{"1min":0.70199999999999996,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.714,"5min":1.748,"15min":1.748},"last":1.0269999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0369999999999999,"15min":1.0349999999999999},"min":{"1min":0.80100000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.214},{"interface":"front","average":{"1min":1.024,"5min":1.0329999999999999,"15min":1.0369999999999999},"min":{"1min":0.75900000000000001,"5min":0.505,"15min":0.505},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.1060000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29648,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342192,"statfs":{"total":96636764160,"available":96606404608,"internally_reserved":0,"allocated":2691072,"data_stored":1296855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.58699999999999997},"min":{"1min":0.33100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.034,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66200000000000003,"15min":0.64700000000000002},"min":{"1min":0.33400000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.1240000000000001,"5min":1.77,"15min":1.77},"last":0.63200000000000001}]},{"osd":2,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60399999999999998,"15min":0.58399999999999996},"min":{"1min":0.45800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81799999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.29399999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0489999999999999,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.71299999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76400000000000001,"15min":0.73399999999999999},"min":{"1min":0.45200000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1180000000000001,"5min":1.171,"15min":1.171},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.76400000000000001,"15min":0.73699999999999999},"min":{"1min":0.42999999999999999,"5min":0.316,"15min":0.316},"max":{"1min":1.0089999999999999,"5min":2.141,"15min":2.141},"last":0.80100000000000005}]},{"osd":4,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.91500000000000004,"15min":0.90000000000000002},"min":{"1min":0.501,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.599,"5min":1.925,"15min":1.925},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1719999999999999,"15min":1.0089999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3049999999999999,"5min":13.701000000000001,"15min":13.701000000000001},"last":1.2589999999999999}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.006,"15min":1.01},"min":{"1min":0.66400000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3400000000000001,"5min":1.48,"15min":1.48},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.98899999999999999,"15min":0.995},"min":{"1min":0.624,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.409,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.155}]},{"osd":6,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.014},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.377,"5min":1.575,"15min":1.575},"last":1.214},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0580000000000001,"15min":1.048},"min":{"1min":0.63200000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.544,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.647,"15min":1.2869999999999999},"min":{"1min":0.76700000000000002,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.2999999999999998,"5min":59.652000000000001,"15min":59.652000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.5920000000000001,"15min":1.24},"min":{"1min":0.70099999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.7869999999999999,"5min":59.679000000000002,"15min":59.679000000000002},"last":0.90700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38416,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333424,"statfs":{"total":96636764160,"available":96597426176,"internally_reserved":0,"allocated":3276800,"data_stored":1887185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.7000000000000002,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.34699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.268,"5min":1.738,"15min":1.738},"last":1.268}]},{"osd":2,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.71899999999999997,"15min":0.68700000000000006},"min":{"1min":0.30199999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.752,"15min":0.78800000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0069999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.63500000000000001}]},{"osd":3,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.71399999999999997,"15min":0.71299999999999997},"min":{"1min":0.38300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89800000000000002,"5min":1.635,"15min":1.635},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.002,"5min":0.871,"15min":0.84799999999999998},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":3.738,"5min":3.738,"15min":3.738},"last":0.95599999999999996}]},{"osd":4,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.88100000000000001,"15min":0.93000000000000005},"min":{"1min":0.53200000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.077,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.073,"5min":0.95599999999999996,"15min":0.95699999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":3.6040000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":1.1919999999999999}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.02,"15min":1.034},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1870000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.873,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.254,"5min":1.724,"15min":1.724},"last":0.91400000000000003}]},{"osd":6,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.028,"15min":1.0449999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":4.2359999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.03,"15min":1.0700000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5609999999999999,"5min":2.024,"15min":2.024},"last":1.095}]},{"osd":7,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.0429999999999999,"15min":1.081},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.23,"5min":2,"15min":2},"last":1.04},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0409999999999999,"15min":1.1120000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.226,"5min":1.744,"15min":1.744},"last":1.0009999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:05:10.195 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.195 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.195 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.195 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.195 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.195 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.196 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.196 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.196 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.196 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.196 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.196 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.196 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-13T07:01:08.184208+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.196 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.196 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T07:01:08.184208+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.197 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.197 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T07:01:08.184208+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.197 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.197 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T07:01:08.184208+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.197 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.197 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.197 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T07:01:08.184208+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.197 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.198 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-13T07:01:08.184208+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.198 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.198 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T07:01:08.184208+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.198 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.198 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T07:01:08.184208+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.198 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T07:01:01.149679+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.198 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.199 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.199 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.199 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.199 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.199 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.199 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.199 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.199 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.199 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.200 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.200 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:10.200 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T07:05:30.201 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:05:30.520 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:05:30.523 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:05:30.535 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-13T07:05:28.671811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":412,"num_object_clones":0,"num_object_copies":824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":412,"num_whiteouts":404,"num_read":239562,"num_read_kb":1058974,"num_write":24247,"num_write_kb":570384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5266,"num_bytes_recovered":116470768,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21244,"ondisk_log_size":21244,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":375564,"kb_used_data":22156,"kb_used_omap":46,"kb_used_meta":353297,"kb_avail":754599156,"statfs":{"total":773094113280,"available":772709535744,"internally_reserved":0,"allocated":22687744,"data_stored":11560198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48002,"internal_metadata":361776254},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003907"},"pg_stats":[{"pgid":"3.1f","version":"222'428","reported_seq":6844,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:19.439494+0000","last_change":"2024-09-13T07:05:19.439642+0000","last_active":"2024-09-13T07:05:19.439494+0000","last_peered":"2024-09-13T07:05:19.439494+0000","last_clean":"2024-09-13T07:05:19.439494+0000","last_became_active":"2024-09-13T07:04:48.131331+0000","last_became_peered":"2024-09-13T07:04:48.131331+0000","last_unstale":"2024-09-13T07:05:19.439494+0000","last_undegraded":"2024-09-13T07:05:19.439494+0000","last_fullsized":"2024-09-13T07:05:19.439494+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'428","last_scrub_stamp":"2024-09-13T07:05:19.439473+0000","last_deep_scrub":"222'428","last_deep_scrub_stamp":"2024-09-13T07:05:19.439473+0000","last_clean_scrub_stamp":"2024-09-13T07:05:19.439473+0000","objects_scrubbed":13,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:58.178088+0000","scrub_duration":0.019357694000000002,"objects_trimmed":30,"snaptrim_duration":0.098858737000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6370,"num_read_kb":30920,"num_write":459,"num_write_kb":15662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":4960629,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"222'394","reported_seq":4145,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:10.464066+0000","last_change":"2024-09-13T07:05:10.464191+0000","last_active":"2024-09-13T07:05:10.464066+0000","last_peered":"2024-09-13T07:05:10.464066+0000","last_clean":"2024-09-13T07:05:10.464066+0000","last_became_active":"2024-09-13T07:04:51.125903+0000","last_became_peered":"2024-09-13T07:04:51.125903+0000","last_unstale":"2024-09-13T07:05:10.464066+0000","last_undegraded":"2024-09-13T07:05:10.464066+0000","last_fullsized":"2024-09-13T07:05:10.464066+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'394","last_scrub_stamp":"2024-09-13T07:05:10.464043+0000","last_deep_scrub":"222'394","last_deep_scrub_stamp":"2024-09-13T07:05:10.464043+0000","last_clean_scrub_stamp":"2024-09-13T07:05:10.464043+0000","objects_scrubbed":15,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:37:33.981404+0000","scrub_duration":0.018767491000000001,"objects_trimmed":2,"snaptrim_duration":0.001432102,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4275,"num_read_kb":38815,"num_write":373,"num_write_kb":7930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"202'489","reported_seq":2581,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.320403+0000","last_change":"2024-09-13T07:04:38.530429+0000","last_active":"2024-09-13T07:04:58.320403+0000","last_peered":"2024-09-13T07:04:58.320403+0000","last_clean":"2024-09-13T07:04:58.320403+0000","last_became_active":"2024-09-13T07:04:38.221697+0000","last_became_peered":"2024-09-13T07:04:38.221697+0000","last_unstale":"2024-09-13T07:04:58.320403+0000","last_undegraded":"2024-09-13T07:04:58.320403+0000","last_fullsized":"2024-09-13T07:04:58.320403+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.073210949999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3758,"num_read_kb":13816,"num_write":504,"num_write_kb":5767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"222'514","reported_seq":5618,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:24.889576+0000","last_change":"2024-09-13T07:05:24.889718+0000","last_active":"2024-09-13T07:05:24.889576+0000","last_peered":"2024-09-13T07:05:24.889576+0000","last_clean":"2024-09-13T07:05:24.889576+0000","last_became_active":"2024-09-13T07:04:38.225836+0000","last_became_peered":"2024-09-13T07:04:38.225836+0000","last_unstale":"2024-09-13T07:05:24.889576+0000","last_undegraded":"2024-09-13T07:05:24.889576+0000","last_fullsized":"2024-09-13T07:05:24.889576+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'514","last_scrub_stamp":"2024-09-13T07:05:24.889555+0000","last_deep_scrub":"222'514","last_deep_scrub_stamp":"2024-09-13T07:05:24.889555+0000","last_clean_scrub_stamp":"2024-09-13T07:05:24.889555+0000","objects_scrubbed":14,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:35.513305+0000","scrub_duration":0.021224962,"objects_trimmed":34,"snaptrim_duration":0.078226893000000006,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6179,"num_read_kb":24975,"num_write":551,"num_write_kb":10621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"222'392","reported_seq":6416,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:17.397897+0000","last_change":"2024-09-13T07:05:17.398063+0000","last_active":"2024-09-13T07:05:17.397897+0000","last_peered":"2024-09-13T07:05:17.397897+0000","last_clean":"2024-09-13T07:05:17.397897+0000","last_became_active":"2024-09-13T07:04:48.128577+0000","last_became_peered":"2024-09-13T07:04:48.128577+0000","last_unstale":"2024-09-13T07:05:17.397897+0000","last_undegraded":"2024-09-13T07:05:17.397897+0000","last_fullsized":"2024-09-13T07:05:17.397897+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'392","last_scrub_stamp":"2024-09-13T07:05:17.397875+0000","last_deep_scrub":"222'392","last_deep_scrub_stamp":"2024-09-13T07:05:17.397875+0000","last_clean_scrub_stamp":"2024-09-13T07:05:17.397875+0000","objects_scrubbed":11,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:23:04.467810+0000","scrub_duration":0.017356244,"objects_trimmed":0,"snaptrim_duration":0.0065506189999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6468,"num_read_kb":53020,"num_write":414,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":148740,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"222'1131","reported_seq":8636,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:12.389358+0000","last_change":"2024-09-13T07:05:12.389483+0000","last_active":"2024-09-13T07:05:12.389358+0000","last_peered":"2024-09-13T07:05:12.389358+0000","last_clean":"2024-09-13T07:05:12.389358+0000","last_became_active":"2024-09-13T07:04:50.121731+0000","last_became_peered":"2024-09-13T07:04:50.121731+0000","last_unstale":"2024-09-13T07:05:12.389358+0000","last_undegraded":"2024-09-13T07:05:12.389358+0000","last_fullsized":"2024-09-13T07:05:12.389358+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'1131","last_scrub_stamp":"2024-09-13T07:05:12.389336+0000","last_deep_scrub":"222'1131","last_deep_scrub_stamp":"2024-09-13T07:05:12.389336+0000","last_clean_scrub_stamp":"2024-09-13T07:05:12.389336+0000","objects_scrubbed":12,"log_size":1131,"log_dups_size":0,"ondisk_log_size":1131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:41.586354+0000","scrub_duration":0.019611713999999999,"objects_trimmed":0,"snaptrim_duration":0.002988971,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":11548,"num_read_kb":34808,"num_write":1366,"num_write_kb":17528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":3976247,"num_keys_recovered":80,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"222'639","reported_seq":7719,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:16.950966+0000","last_change":"2024-09-13T07:05:16.951134+0000","last_active":"2024-09-13T07:05:16.950966+0000","last_peered":"2024-09-13T07:05:16.950966+0000","last_clean":"2024-09-13T07:05:16.950966+0000","last_became_active":"2024-09-13T07:04:50.123234+0000","last_became_peered":"2024-09-13T07:04:50.123234+0000","last_unstale":"2024-09-13T07:05:16.950966+0000","last_undegraded":"2024-09-13T07:05:16.950966+0000","last_fullsized":"2024-09-13T07:05:16.950966+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'639","last_scrub_stamp":"2024-09-13T07:05:16.950943+0000","last_deep_scrub":"222'639","last_deep_scrub_stamp":"2024-09-13T07:05:16.950943+0000","last_clean_scrub_stamp":"2024-09-13T07:05:16.950943+0000","objects_scrubbed":11,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:59:06.483229+0000","scrub_duration":0.017176568999999999,"objects_trimmed":0,"snaptrim_duration":0.001498135,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9140,"num_read_kb":20172,"num_write":660,"num_write_kb":17799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":1288675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"222'716","reported_seq":7838,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:18.598082+0000","last_change":"2024-09-13T07:05:18.598296+0000","last_active":"2024-09-13T07:05:18.598082+0000","last_peered":"2024-09-13T07:05:18.598082+0000","last_clean":"2024-09-13T07:05:18.598082+0000","last_became_active":"2024-09-13T07:04:51.126595+0000","last_became_peered":"2024-09-13T07:04:51.126595+0000","last_unstale":"2024-09-13T07:05:18.598082+0000","last_undegraded":"2024-09-13T07:05:18.598082+0000","last_fullsized":"2024-09-13T07:05:18.598082+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'716","last_scrub_stamp":"2024-09-13T07:05:18.598053+0000","last_deep_scrub":"222'716","last_deep_scrub_stamp":"2024-09-13T07:05:18.598053+0000","last_clean_scrub_stamp":"2024-09-13T07:05:18.598053+0000","objects_scrubbed":19,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:53:50.938518+0000","scrub_duration":0.024486260999999999,"objects_trimmed":2,"snaptrim_duration":0.001501437,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8486,"num_read_kb":30532,"num_write":647,"num_write_kb":16391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":13265380,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"222'906","reported_seq":12753,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:11.533272+0000","last_change":"2024-09-13T07:05:11.533393+0000","last_active":"2024-09-13T07:05:11.533272+0000","last_peered":"2024-09-13T07:05:11.533272+0000","last_clean":"2024-09-13T07:05:11.533272+0000","last_became_active":"2024-09-13T07:04:45.140362+0000","last_became_peered":"2024-09-13T07:04:45.140362+0000","last_unstale":"2024-09-13T07:05:11.533272+0000","last_undegraded":"2024-09-13T07:05:11.533272+0000","last_fullsized":"2024-09-13T07:05:11.533272+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'906","last_scrub_stamp":"2024-09-13T07:05:11.533245+0000","last_deep_scrub":"222'906","last_deep_scrub_stamp":"2024-09-13T07:05:11.533245+0000","last_clean_scrub_stamp":"2024-09-13T07:05:11.533245+0000","objects_scrubbed":13,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:41.832464+0000","scrub_duration":0.022800498999999998,"objects_trimmed":34,"snaptrim_duration":0.082577740999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":15171,"num_read_kb":70673,"num_write":1118,"num_write_kb":35612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":8426052,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"198'847","reported_seq":12700,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:04:58.320678+0000","last_change":"2024-09-13T07:04:47.128907+0000","last_active":"2024-09-13T07:04:58.320678+0000","last_peered":"2024-09-13T07:04:58.320678+0000","last_clean":"2024-09-13T07:04:58.320678+0000","last_became_active":"2024-09-13T07:04:47.126092+0000","last_became_peered":"2024-09-13T07:04:47.126092+0000","last_unstale":"2024-09-13T07:04:58.320678+0000","last_undegraded":"2024-09-13T07:04:58.320678+0000","last_fullsized":"2024-09-13T07:04:58.320678+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:01:18.255898+0000","last_clean_scrub_stamp":"2024-09-13T07:01:18.255898+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075556199999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":14283,"num_read_kb":24697,"num_write":1275,"num_write_kb":8273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1104561,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"222'457","reported_seq":2679,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:18.321703+0000","last_change":"2024-09-13T07:05:18.321841+0000","last_active":"2024-09-13T07:05:18.321703+0000","last_peered":"2024-09-13T07:05:18.321703+0000","last_clean":"2024-09-13T07:05:18.321703+0000","last_became_active":"2024-09-13T07:04:57.321478+0000","last_became_peered":"2024-09-13T07:04:57.321478+0000","last_unstale":"2024-09-13T07:05:18.321703+0000","last_undegraded":"2024-09-13T07:05:18.321703+0000","last_fullsized":"2024-09-13T07:05:18.321703+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'457","last_scrub_stamp":"2024-09-13T07:05:18.321677+0000","last_deep_scrub":"222'457","last_deep_scrub_stamp":"2024-09-13T07:05:18.321677+0000","last_clean_scrub_stamp":"2024-09-13T07:05:18.321677+0000","objects_scrubbed":5,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:40.758192+0000","scrub_duration":0.011351843,"objects_trimmed":0,"snaptrim_duration":0.001895851,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2526,"num_read_kb":26928,"num_write":433,"num_write_kb":12456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"222'537","reported_seq":5776,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:19.299462+0000","last_change":"2024-09-13T07:05:19.299643+0000","last_active":"2024-09-13T07:05:19.299462+0000","last_peered":"2024-09-13T07:05:19.299462+0000","last_clean":"2024-09-13T07:05:19.299462+0000","last_became_active":"2024-09-13T07:04:46.126170+0000","last_became_peered":"2024-09-13T07:04:46.126170+0000","last_unstale":"2024-09-13T07:05:19.299462+0000","last_undegraded":"2024-09-13T07:05:19.299462+0000","last_fullsized":"2024-09-13T07:05:19.299462+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'537","last_scrub_stamp":"2024-09-13T07:05:19.299439+0000","last_deep_scrub":"222'537","last_deep_scrub_stamp":"2024-09-13T07:05:19.299439+0000","last_clean_scrub_stamp":"2024-09-13T07:05:19.299439+0000","objects_scrubbed":21,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:03.051537+0000","scrub_duration":0.026987895000000001,"objects_trimmed":0,"snaptrim_duration":0.0074505329999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5002,"num_read_kb":31007,"num_write":502,"num_write_kb":34041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":6788059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":245,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:10.285064+0000","last_change":"2024-09-13T07:05:10.285288+0000","last_active":"2024-09-13T07:05:10.285064+0000","last_peered":"2024-09-13T07:05:10.285064+0000","last_clean":"2024-09-13T07:05:10.285064+0000","last_became_active":"2024-09-13T07:04:43.266020+0000","last_became_peered":"2024-09-13T07:04:43.266020+0000","last_unstale":"2024-09-13T07:05:10.285064+0000","last_undegraded":"2024-09-13T07:05:10.285064+0000","last_fullsized":"2024-09-13T07:05:10.285064+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:05:10.285018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:05:10.285018+0000","last_clean_scrub_stamp":"2024-09-13T07:05:10.285018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:23.358286+0000","scrub_duration":0.002927439,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"222'474","reported_seq":3643,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:16.444642+0000","last_change":"2024-09-13T07:05:16.444877+0000","last_active":"2024-09-13T07:05:16.444642+0000","last_peered":"2024-09-13T07:05:16.444642+0000","last_clean":"2024-09-13T07:05:16.444642+0000","last_became_active":"2024-09-13T07:04:49.125105+0000","last_became_peered":"2024-09-13T07:04:49.125105+0000","last_unstale":"2024-09-13T07:05:16.444642+0000","last_undegraded":"2024-09-13T07:05:16.444642+0000","last_fullsized":"2024-09-13T07:05:16.444642+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'474","last_scrub_stamp":"2024-09-13T07:05:16.444589+0000","last_deep_scrub":"222'474","last_deep_scrub_stamp":"2024-09-13T07:05:16.444589+0000","last_clean_scrub_stamp":"2024-09-13T07:05:16.444589+0000","objects_scrubbed":15,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:00:18.701262+0000","scrub_duration":0.021064025,"objects_trimmed":0,"snaptrim_duration":0.00066392000000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2852,"num_read_kb":36914,"num_write":449,"num_write_kb":17878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4764805,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:11.930609+0000","last_change":"2024-09-13T07:05:11.930844+0000","last_active":"2024-09-13T07:05:11.930609+0000","last_peered":"2024-09-13T07:05:11.930609+0000","last_clean":"2024-09-13T07:05:11.930609+0000","last_became_active":"2024-09-13T07:04:46.127497+0000","last_became_peered":"2024-09-13T07:04:46.127497+0000","last_unstale":"2024-09-13T07:05:11.930609+0000","last_undegraded":"2024-09-13T07:05:11.930609+0000","last_fullsized":"2024-09-13T07:05:11.930609+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:05:11.930565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:05:11.930565+0000","last_clean_scrub_stamp":"2024-09-13T07:05:11.930565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:16.038101+0000","scrub_duration":0.0034661929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"222'618","reported_seq":4801,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:09.502658+0000","last_change":"2024-09-13T07:05:09.502785+0000","last_active":"2024-09-13T07:05:09.502658+0000","last_peered":"2024-09-13T07:05:09.502658+0000","last_clean":"2024-09-13T07:05:09.502658+0000","last_became_active":"2024-09-13T07:04:46.126668+0000","last_became_peered":"2024-09-13T07:04:46.126668+0000","last_unstale":"2024-09-13T07:05:09.502658+0000","last_undegraded":"2024-09-13T07:05:09.502658+0000","last_fullsized":"2024-09-13T07:05:09.502658+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'618","last_scrub_stamp":"2024-09-13T07:05:09.502635+0000","last_deep_scrub":"222'618","last_deep_scrub_stamp":"2024-09-13T07:05:09.502635+0000","last_clean_scrub_stamp":"2024-09-13T07:05:09.502635+0000","objects_scrubbed":19,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:10:25.728996+0000","scrub_duration":0.024057370000000002,"objects_trimmed":0,"snaptrim_duration":0.001721534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5009,"num_read_kb":48072,"num_write":666,"num_write_kb":15004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":128514,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:25.491450+0000","last_change":"2024-09-13T07:05:25.491765+0000","last_active":"2024-09-13T07:05:25.491450+0000","last_peered":"2024-09-13T07:05:25.491450+0000","last_clean":"2024-09-13T07:05:25.491450+0000","last_became_active":"2024-09-13T07:04:57.325955+0000","last_became_peered":"2024-09-13T07:04:57.325955+0000","last_unstale":"2024-09-13T07:05:25.491450+0000","last_undegraded":"2024-09-13T07:05:25.491450+0000","last_fullsized":"2024-09-13T07:05:25.491450+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:05:25.491392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:05:25.491392+0000","last_clean_scrub_stamp":"2024-09-13T07:05:25.491392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:11:49.607496+0000","scrub_duration":0.0037360309999999999,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"222'394","reported_seq":1827,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:15.931694+0000","last_change":"2024-09-13T07:05:15.931865+0000","last_active":"2024-09-13T07:05:15.931694+0000","last_peered":"2024-09-13T07:05:15.931694+0000","last_clean":"2024-09-13T07:05:15.931694+0000","last_became_active":"2024-09-13T07:04:49.132150+0000","last_became_peered":"2024-09-13T07:04:49.132150+0000","last_unstale":"2024-09-13T07:05:15.931694+0000","last_undegraded":"2024-09-13T07:05:15.931694+0000","last_fullsized":"2024-09-13T07:05:15.931694+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'394","last_scrub_stamp":"2024-09-13T07:05:15.931650+0000","last_deep_scrub":"222'394","last_deep_scrub_stamp":"2024-09-13T07:05:15.931650+0000","last_clean_scrub_stamp":"2024-09-13T07:05:15.931650+0000","objects_scrubbed":17,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:40:54.304331+0000","scrub_duration":0.023585517,"objects_trimmed":28,"snaptrim_duration":0.088725134999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1176,"num_read_kb":23362,"num_write":393,"num_write_kb":13842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":2874205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:09.983924+0000","last_change":"2024-09-13T07:05:09.984085+0000","last_active":"2024-09-13T07:05:09.983924+0000","last_peered":"2024-09-13T07:05:09.983924+0000","last_clean":"2024-09-13T07:05:09.983924+0000","last_became_active":"2024-09-13T07:04:46.129069+0000","last_became_peered":"2024-09-13T07:04:46.129069+0000","last_unstale":"2024-09-13T07:05:09.983924+0000","last_undegraded":"2024-09-13T07:05:09.983924+0000","last_fullsized":"2024-09-13T07:05:09.983924+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:05:09.983897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:05:09.983897+0000","last_clean_scrub_stamp":"2024-09-13T07:05:09.983897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:25.703309+0000","scrub_duration":0.0020093709999999998,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"222'720","reported_seq":5298,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:25.873960+0000","last_change":"2024-09-13T07:05:25.874066+0000","last_active":"2024-09-13T07:05:25.873960+0000","last_peered":"2024-09-13T07:05:25.873960+0000","last_clean":"2024-09-13T07:05:25.873960+0000","last_became_active":"2024-09-13T07:04:38.218110+0000","last_became_peered":"2024-09-13T07:04:38.218110+0000","last_unstale":"2024-09-13T07:05:25.873960+0000","last_undegraded":"2024-09-13T07:05:25.873960+0000","last_fullsized":"2024-09-13T07:05:25.873960+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'720","last_scrub_stamp":"2024-09-13T07:05:25.873939+0000","last_deep_scrub":"222'720","last_deep_scrub_stamp":"2024-09-13T07:05:25.873939+0000","last_clean_scrub_stamp":"2024-09-13T07:05:25.873939+0000","objects_scrubbed":12,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:52:50.964394+0000","scrub_duration":0.022412522000000001,"objects_trimmed":40,"snaptrim_duration":0.101155064,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6847,"num_read_kb":24040,"num_write":729,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"222'619","reported_seq":6663,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:10.407317+0000","last_change":"2024-09-13T07:05:10.407478+0000","last_active":"2024-09-13T07:05:10.407317+0000","last_peered":"2024-09-13T07:05:10.407317+0000","last_clean":"2024-09-13T07:05:10.407317+0000","last_became_active":"2024-09-13T07:03:06.744590+0000","last_became_peered":"2024-09-13T07:03:06.744590+0000","last_unstale":"2024-09-13T07:05:10.407317+0000","last_undegraded":"2024-09-13T07:05:10.407317+0000","last_fullsized":"2024-09-13T07:05:10.407317+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'619","last_scrub_stamp":"2024-09-13T07:05:10.407290+0000","last_deep_scrub":"222'619","last_deep_scrub_stamp":"2024-09-13T07:05:10.407290+0000","last_clean_scrub_stamp":"2024-09-13T07:05:10.407290+0000","objects_scrubbed":9,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:08:37.238248+0000","scrub_duration":0.015106461999999999,"objects_trimmed":0,"snaptrim_duration":0.006933257,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":6314,"num_read_kb":32619,"num_write":703,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":11860709,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"222'3","reported_seq":188,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:15.883266+0000","last_change":"2024-09-13T07:05:15.883583+0000","last_active":"2024-09-13T07:05:15.883266+0000","last_peered":"2024-09-13T07:05:15.883266+0000","last_clean":"2024-09-13T07:05:15.883266+0000","last_became_active":"2024-09-13T07:04:46.125291+0000","last_became_peered":"2024-09-13T07:04:46.125291+0000","last_unstale":"2024-09-13T07:05:15.883266+0000","last_undegraded":"2024-09-13T07:05:15.883266+0000","last_fullsized":"2024-09-13T07:05:15.883266+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'3","last_scrub_stamp":"2024-09-13T07:05:15.883217+0000","last_deep_scrub":"222'3","last_deep_scrub_stamp":"2024-09-13T07:05:15.883217+0000","last_clean_scrub_stamp":"2024-09-13T07:05:15.883217+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:04.573276+0000","scrub_duration":0.0066720950000000003,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"222'626","reported_seq":14070,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:20.980133+0000","last_change":"2024-09-13T07:05:20.980246+0000","last_active":"2024-09-13T07:05:20.980133+0000","last_peered":"2024-09-13T07:05:20.980133+0000","last_clean":"2024-09-13T07:05:20.980133+0000","last_became_active":"2024-09-13T07:04:50.122952+0000","last_became_peered":"2024-09-13T07:04:50.122952+0000","last_unstale":"2024-09-13T07:05:20.980133+0000","last_undegraded":"2024-09-13T07:05:20.980133+0000","last_fullsized":"2024-09-13T07:05:20.980133+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'626","last_scrub_stamp":"2024-09-13T07:05:20.980111+0000","last_deep_scrub":"222'626","last_deep_scrub_stamp":"2024-09-13T07:05:20.980111+0000","last_clean_scrub_stamp":"2024-09-13T07:05:20.980111+0000","objects_scrubbed":11,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:21:52.626071+0000","scrub_duration":0.017791412999999999,"objects_trimmed":0,"snaptrim_duration":0.00084358800000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13923,"num_read_kb":43420,"num_write":574,"num_write_kb":22169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":8978985,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":146,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:08.739367+0000","last_change":"2024-09-13T07:05:08.739585+0000","last_active":"2024-09-13T07:05:08.739367+0000","last_peered":"2024-09-13T07:05:08.739367+0000","last_clean":"2024-09-13T07:05:08.739367+0000","last_became_active":"2024-09-13T07:02:44.090715+0000","last_became_peered":"2024-09-13T07:02:44.090715+0000","last_unstale":"2024-09-13T07:05:08.739367+0000","last_undegraded":"2024-09-13T07:05:08.739367+0000","last_fullsized":"2024-09-13T07:05:08.739367+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:05:08.739321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:05:08.739321+0000","last_clean_scrub_stamp":"2024-09-13T07:05:08.739321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:14.140079+0000","scrub_duration":0.0024062810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"222'389","reported_seq":7143,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:10.571648+0000","last_change":"2024-09-13T07:05:10.571648+0000","last_active":"2024-09-13T07:05:10.571648+0000","last_peered":"2024-09-13T07:05:10.571648+0000","last_clean":"2024-09-13T07:05:10.571648+0000","last_became_active":"2024-09-13T07:04:54.129673+0000","last_became_peered":"2024-09-13T07:04:54.129673+0000","last_unstale":"2024-09-13T07:05:10.571648+0000","last_undegraded":"2024-09-13T07:05:10.571648+0000","last_fullsized":"2024-09-13T07:05:10.571648+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'389","last_scrub_stamp":"2024-09-13T07:05:10.571482+0000","last_deep_scrub":"222'389","last_deep_scrub_stamp":"2024-09-13T07:05:10.571482+0000","last_clean_scrub_stamp":"2024-09-13T07:05:10.571482+0000","objects_scrubbed":13,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:36:37.820768+0000","scrub_duration":0.019009884000000001,"objects_trimmed":26,"snaptrim_duration":0.064131435000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6959,"num_read_kb":35900,"num_write":367,"num_write_kb":8191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":3448599,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:11.290601+0000","last_change":"2024-09-13T07:05:11.290831+0000","last_active":"2024-09-13T07:05:11.290601+0000","last_peered":"2024-09-13T07:05:11.290601+0000","last_clean":"2024-09-13T07:05:11.290601+0000","last_became_active":"2024-09-13T07:04:57.320044+0000","last_became_peered":"2024-09-13T07:04:57.320044+0000","last_unstale":"2024-09-13T07:05:11.290601+0000","last_undegraded":"2024-09-13T07:05:11.290601+0000","last_fullsized":"2024-09-13T07:05:11.290601+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:05:11.290548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:05:11.290548+0000","last_clean_scrub_stamp":"2024-09-13T07:05:11.290548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:30:36.116602+0000","scrub_duration":0.0031320139999999998,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"222'524","reported_seq":4299,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:12.460731+0000","last_change":"2024-09-13T07:05:12.460887+0000","last_active":"2024-09-13T07:05:12.460731+0000","last_peered":"2024-09-13T07:05:12.460731+0000","last_clean":"2024-09-13T07:05:12.460731+0000","last_became_active":"2024-09-13T07:04:51.124840+0000","last_became_peered":"2024-09-13T07:04:51.124840+0000","last_unstale":"2024-09-13T07:05:12.460731+0000","last_undegraded":"2024-09-13T07:05:12.460731+0000","last_fullsized":"2024-09-13T07:05:12.460731+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'524","last_scrub_stamp":"2024-09-13T07:05:12.460702+0000","last_deep_scrub":"222'524","last_deep_scrub_stamp":"2024-09-13T07:05:12.460702+0000","last_clean_scrub_stamp":"2024-09-13T07:05:12.460702+0000","objects_scrubbed":13,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:54:37.903549+0000","scrub_duration":0.018966169000000001,"objects_trimmed":28,"snaptrim_duration":0.071371410999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6314,"num_read_kb":38281,"num_write":477,"num_write_kb":24168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"222'2","reported_seq":208,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:20.899358+0000","last_change":"2024-09-13T07:05:20.899712+0000","last_active":"2024-09-13T07:05:20.899358+0000","last_peered":"2024-09-13T07:05:20.899358+0000","last_clean":"2024-09-13T07:05:20.899358+0000","last_became_active":"2024-09-13T07:04:38.218537+0000","last_became_peered":"2024-09-13T07:04:38.218537+0000","last_unstale":"2024-09-13T07:05:20.899358+0000","last_undegraded":"2024-09-13T07:05:20.899358+0000","last_fullsized":"2024-09-13T07:05:20.899358+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'2","last_scrub_stamp":"2024-09-13T07:05:20.899240+0000","last_deep_scrub":"222'2","last_deep_scrub_stamp":"2024-09-13T07:05:20.899240+0000","last_clean_scrub_stamp":"2024-09-13T07:05:20.899240+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:36.898876+0000","scrub_duration":0.0072699080000000003,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"222'119","reported_seq":357,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:19.310544+0000","last_change":"2024-09-13T07:05:19.310678+0000","last_active":"2024-09-13T07:05:19.310544+0000","last_peered":"2024-09-13T07:05:19.310544+0000","last_clean":"2024-09-13T07:05:19.310544+0000","last_became_active":"2024-09-13T07:04:57.324203+0000","last_became_peered":"2024-09-13T07:04:57.324203+0000","last_unstale":"2024-09-13T07:05:19.310544+0000","last_undegraded":"2024-09-13T07:05:19.310544+0000","last_fullsized":"2024-09-13T07:05:19.310544+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'119","last_scrub_stamp":"2024-09-13T07:05:19.310519+0000","last_deep_scrub":"222'119","last_deep_scrub_stamp":"2024-09-13T07:05:19.310519+0000","last_clean_scrub_stamp":"2024-09-13T07:05:19.310519+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:22:09.450744+0000","scrub_duration":0.0070434039999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"222'797","reported_seq":5579,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:21.629984+0000","last_change":"2024-09-13T07:05:21.630211+0000","last_active":"2024-09-13T07:05:21.629984+0000","last_peered":"2024-09-13T07:05:21.629984+0000","last_clean":"2024-09-13T07:05:21.629984+0000","last_became_active":"2024-09-13T07:04:53.131823+0000","last_became_peered":"2024-09-13T07:04:53.131823+0000","last_unstale":"2024-09-13T07:05:21.629984+0000","last_undegraded":"2024-09-13T07:05:21.629984+0000","last_fullsized":"2024-09-13T07:05:21.629984+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'797","last_scrub_stamp":"2024-09-13T07:05:21.629950+0000","last_deep_scrub":"222'797","last_deep_scrub_stamp":"2024-09-13T07:05:21.629950+0000","last_clean_scrub_stamp":"2024-09-13T07:05:21.629950+0000","objects_scrubbed":15,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:08:14.575207+0000","scrub_duration":0.022307024000000002,"objects_trimmed":0,"snaptrim_duration":0.0032724400000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5361,"num_read_kb":39438,"num_write":976,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":861185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"222'443","reported_seq":9651,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:13.343322+0000","last_change":"2024-09-13T07:05:13.343789+0000","last_active":"2024-09-13T07:05:13.343322+0000","last_peered":"2024-09-13T07:05:13.343322+0000","last_clean":"2024-09-13T07:05:13.343322+0000","last_became_active":"2024-09-13T07:02:47.511257+0000","last_became_peered":"2024-09-13T07:02:47.511257+0000","last_unstale":"2024-09-13T07:05:13.343322+0000","last_undegraded":"2024-09-13T07:05:13.343322+0000","last_fullsized":"2024-09-13T07:05:13.343322+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'443","last_scrub_stamp":"2024-09-13T07:05:13.343186+0000","last_deep_scrub":"222'443","last_deep_scrub_stamp":"2024-09-13T07:05:13.343186+0000","last_clean_scrub_stamp":"2024-09-13T07:05:13.343186+0000","objects_scrubbed":11,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:44.865233+0000","scrub_duration":0.016793586999999999,"objects_trimmed":0,"snaptrim_duration":0.0059111220000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9849,"num_read_kb":28359,"num_write":451,"num_write_kb":30112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5191019,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"222'1007","reported_seq":4572,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:11.267694+0000","last_change":"2024-09-13T07:05:11.267824+0000","last_active":"2024-09-13T07:05:11.267694+0000","last_peered":"2024-09-13T07:05:11.267694+0000","last_clean":"2024-09-13T07:05:11.267694+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:05:11.267694+0000","last_undegraded":"2024-09-13T07:05:11.267694+0000","last_fullsized":"2024-09-13T07:05:11.267694+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'1007","last_scrub_stamp":"2024-09-13T07:05:11.267673+0000","last_deep_scrub":"222'1007","last_deep_scrub_stamp":"2024-09-13T07:05:11.267673+0000","last_clean_scrub_stamp":"2024-09-13T07:05:11.267673+0000","objects_scrubbed":11,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:06.385003+0000","scrub_duration":0.018687054000000002,"objects_trimmed":0,"snaptrim_duration":0.0016753040000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7385,"num_read_kb":27871,"num_write":1272,"num_write_kb":19033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"222'614","reported_seq":7672,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:14.503868+0000","last_change":"2024-09-13T07:05:14.504037+0000","last_active":"2024-09-13T07:05:14.503868+0000","last_peered":"2024-09-13T07:05:14.503868+0000","last_clean":"2024-09-13T07:05:14.503868+0000","last_became_active":"2024-09-13T07:04:51.126975+0000","last_became_peered":"2024-09-13T07:04:51.126975+0000","last_unstale":"2024-09-13T07:05:14.503868+0000","last_undegraded":"2024-09-13T07:05:14.503868+0000","last_fullsized":"2024-09-13T07:05:14.503868+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'614","last_scrub_stamp":"2024-09-13T07:05:14.503847+0000","last_deep_scrub":"222'614","last_deep_scrub_stamp":"2024-09-13T07:05:14.503847+0000","last_clean_scrub_stamp":"2024-09-13T07:05:14.503847+0000","objects_scrubbed":15,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:26.338342+0000","scrub_duration":0.019705391999999999,"objects_trimmed":0,"snaptrim_duration":0.0020303249999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9098,"num_read_kb":32181,"num_write":560,"num_write_kb":27201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":10029829,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"222'1708","reported_seq":13995,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:21.909506+0000","last_change":"2024-09-13T07:05:21.909646+0000","last_active":"2024-09-13T07:05:21.909506+0000","last_peered":"2024-09-13T07:05:21.909506+0000","last_clean":"2024-09-13T07:05:21.909506+0000","last_became_active":"2024-09-13T07:04:41.239873+0000","last_became_peered":"2024-09-13T07:04:41.239873+0000","last_unstale":"2024-09-13T07:05:21.909506+0000","last_undegraded":"2024-09-13T07:05:21.909506+0000","last_fullsized":"2024-09-13T07:05:21.909506+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'1708","last_scrub_stamp":"2024-09-13T07:05:21.909484+0000","last_deep_scrub":"222'1708","last_deep_scrub_stamp":"2024-09-13T07:05:21.909484+0000","last_clean_scrub_stamp":"2024-09-13T07:05:21.909484+0000","objects_scrubbed":8,"log_size":1708,"log_dups_size":0,"ondisk_log_size":1708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:14:22.762681+0000","scrub_duration":0.018664949,"objects_trimmed":30,"snaptrim_duration":0.091557893000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21059,"num_read_kb":55383,"num_write":2606,"num_write_kb":25105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"222'674","reported_seq":6960,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:12.353383+0000","last_change":"2024-09-13T07:05:12.353650+0000","last_active":"2024-09-13T07:05:12.353383+0000","last_peered":"2024-09-13T07:05:12.353383+0000","last_clean":"2024-09-13T07:05:12.353383+0000","last_became_active":"2024-09-13T07:04:57.323373+0000","last_became_peered":"2024-09-13T07:04:57.323373+0000","last_unstale":"2024-09-13T07:05:12.353383+0000","last_undegraded":"2024-09-13T07:05:12.353383+0000","last_fullsized":"2024-09-13T07:05:12.353383+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'674","last_scrub_stamp":"2024-09-13T07:05:12.353308+0000","last_deep_scrub":"222'674","last_deep_scrub_stamp":"2024-09-13T07:05:12.353308+0000","last_clean_scrub_stamp":"2024-09-13T07:05:12.353308+0000","objects_scrubbed":12,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:58.097908+0000","scrub_duration":0.018580025,"objects_trimmed":0,"snaptrim_duration":0.002404799,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8026,"num_read_kb":31428,"num_write":564,"num_write_kb":31570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9245169,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"222'571","reported_seq":3705,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:21.389562+0000","last_change":"2024-09-13T07:05:21.389690+0000","last_active":"2024-09-13T07:05:21.389562+0000","last_peered":"2024-09-13T07:05:21.389562+0000","last_clean":"2024-09-13T07:05:21.389562+0000","last_became_active":"2024-09-13T07:04:57.325924+0000","last_became_peered":"2024-09-13T07:04:57.325924+0000","last_unstale":"2024-09-13T07:05:21.389562+0000","last_undegraded":"2024-09-13T07:05:21.389562+0000","last_fullsized":"2024-09-13T07:05:21.389562+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'571","last_scrub_stamp":"2024-09-13T07:05:21.389538+0000","last_deep_scrub":"222'571","last_deep_scrub_stamp":"2024-09-13T07:05:21.389538+0000","last_clean_scrub_stamp":"2024-09-13T07:05:21.389538+0000","objects_scrubbed":10,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:55.462231+0000","scrub_duration":0.017440310000000001,"objects_trimmed":0,"snaptrim_duration":0.0086238349999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4232,"num_read_kb":25664,"num_write":732,"num_write_kb":10506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1357199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"222'925","reported_seq":3847,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:22.437352+0000","last_change":"2024-09-13T07:05:22.437529+0000","last_active":"2024-09-13T07:05:22.437352+0000","last_peered":"2024-09-13T07:05:22.437352+0000","last_clean":"2024-09-13T07:05:22.437352+0000","last_became_active":"2024-09-13T07:04:57.326822+0000","last_became_peered":"2024-09-13T07:04:57.326822+0000","last_unstale":"2024-09-13T07:05:22.437352+0000","last_undegraded":"2024-09-13T07:05:22.437352+0000","last_fullsized":"2024-09-13T07:05:22.437352+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'925","last_scrub_stamp":"2024-09-13T07:05:22.437321+0000","last_deep_scrub":"222'925","last_deep_scrub_stamp":"2024-09-13T07:05:22.437321+0000","last_clean_scrub_stamp":"2024-09-13T07:05:22.437321+0000","objects_scrubbed":16,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:40:34.230142+0000","scrub_duration":0.023200728,"objects_trimmed":0,"snaptrim_duration":0.0075911479999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5743,"num_read_kb":24275,"num_write":1279,"num_write_kb":27074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":68557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"222'429","reported_seq":2623,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:12.291803+0000","last_change":"2024-09-13T07:05:12.292273+0000","last_active":"2024-09-13T07:05:12.291803+0000","last_peered":"2024-09-13T07:05:12.291803+0000","last_clean":"2024-09-13T07:05:12.291803+0000","last_became_active":"2024-09-13T07:03:06.743141+0000","last_became_peered":"2024-09-13T07:03:06.743141+0000","last_unstale":"2024-09-13T07:05:12.291803+0000","last_undegraded":"2024-09-13T07:05:12.291803+0000","last_fullsized":"2024-09-13T07:05:12.291803+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'429","last_scrub_stamp":"2024-09-13T07:05:12.291731+0000","last_deep_scrub":"222'429","last_deep_scrub_stamp":"2024-09-13T07:05:12.291731+0000","last_clean_scrub_stamp":"2024-09-13T07:05:12.291731+0000","objects_scrubbed":10,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:24:12.994684+0000","scrub_duration":0.017150891000000001,"objects_trimmed":0,"snaptrim_duration":0.0012751349999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2106,"num_read_kb":12966,"num_write":446,"num_write_kb":14235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":10481695,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"222'603","reported_seq":8356,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:13.354227+0000","last_change":"2024-09-13T07:05:13.354501+0000","last_active":"2024-09-13T07:05:13.354227+0000","last_peered":"2024-09-13T07:05:13.354227+0000","last_clean":"2024-09-13T07:05:13.354227+0000","last_became_active":"2024-09-13T07:04:57.325398+0000","last_became_peered":"2024-09-13T07:04:57.325398+0000","last_unstale":"2024-09-13T07:05:13.354227+0000","last_undegraded":"2024-09-13T07:05:13.354227+0000","last_fullsized":"2024-09-13T07:05:13.354227+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'603","last_scrub_stamp":"2024-09-13T07:05:13.354174+0000","last_deep_scrub":"222'603","last_deep_scrub_stamp":"2024-09-13T07:05:13.354174+0000","last_clean_scrub_stamp":"2024-09-13T07:05:13.354174+0000","objects_scrubbed":11,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:09.354402+0000","scrub_duration":0.017712631,"objects_trimmed":0,"snaptrim_duration":0.0046356670000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8997,"num_read_kb":22165,"num_write":633,"num_write_kb":13453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":102776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"222'936","reported_seq":9138,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:09.556058+0000","last_change":"2024-09-13T07:05:09.556351+0000","last_active":"2024-09-13T07:05:09.556058+0000","last_peered":"2024-09-13T07:05:09.556058+0000","last_clean":"2024-09-13T07:05:09.556058+0000","last_became_active":"2024-09-13T07:04:47.130438+0000","last_became_peered":"2024-09-13T07:04:47.130438+0000","last_unstale":"2024-09-13T07:05:09.556058+0000","last_undegraded":"2024-09-13T07:05:09.556058+0000","last_fullsized":"2024-09-13T07:05:09.556058+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'936","last_scrub_stamp":"2024-09-13T07:05:09.556009+0000","last_deep_scrub":"222'936","last_deep_scrub_stamp":"2024-09-13T07:05:09.556009+0000","last_clean_scrub_stamp":"2024-09-13T07:05:09.556009+0000","objects_scrubbed":9,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:35:06.312941+0000","scrub_duration":0.015400743999999999,"objects_trimmed":0,"snaptrim_duration":0.001076933,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9860,"num_read_kb":45806,"num_write":1253,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":5924852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"222'599","reported_seq":4712,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:15.330673+0000","last_change":"2024-09-13T07:05:15.330898+0000","last_active":"2024-09-13T07:05:15.330673+0000","last_peered":"2024-09-13T07:05:15.330673+0000","last_clean":"2024-09-13T07:05:15.330673+0000","last_became_active":"2024-09-13T07:04:57.325425+0000","last_became_peered":"2024-09-13T07:04:57.325425+0000","last_unstale":"2024-09-13T07:05:15.330673+0000","last_undegraded":"2024-09-13T07:05:15.330673+0000","last_fullsized":"2024-09-13T07:05:15.330673+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'599","last_scrub_stamp":"2024-09-13T07:05:15.330642+0000","last_deep_scrub":"222'599","last_deep_scrub_stamp":"2024-09-13T07:05:15.330642+0000","last_clean_scrub_stamp":"2024-09-13T07:05:15.330642+0000","objects_scrubbed":11,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:41.366391+0000","scrub_duration":0.01768716,"objects_trimmed":0,"snaptrim_duration":0.0051571560000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5130,"num_read_kb":30367,"num_write":654,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":12828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":408,"num_object_clones":0,"num_object_copies":816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":408,"num_whiteouts":404,"num_read":239446,"num_read_kb":1058874,"num_write":24086,"num_write_kb":567966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5260,"num_bytes_recovered":115290013,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},"log_size":21120,"ondisk_log_size":21120,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78600,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293240,"statfs":{"total":96636764160,"available":96556277760,"internally_reserved":0,"allocated":3276800,"data_stored":1887770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77199819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73999999999999999,"15min":0.70999999999999996},"min":{"1min":0.60399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1100000000000001,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":1.014},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.90600000000000003,"15min":0.78000000000000003},"min":{"1min":0.71099999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0720000000000001,"5min":12.583,"15min":12.583},"last":0.86899999999999999}]},{"osd":1,"last update":"Fri Sep 13 07:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77000000000000002,"15min":0.73799999999999999},"min":{"1min":0.53100000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.98699999999999999,"5min":2.016,"15min":2.016},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.97299999999999998,"15min":0.92100000000000004},"min":{"1min":0.56499999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.97999999999999998,"5min":12.481,"15min":12.481},"last":0.93400000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.88300000000000001,"15min":0.86899999999999999},"min":{"1min":0.53100000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2130000000000001,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.84699999999999998,"15min":0.86899999999999999},"min":{"1min":0.495,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2330000000000001,"5min":1.593,"15min":1.593},"last":1.052}]},{"osd":3,"last update":"Fri Sep 13 07:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91200000000000003},"min":{"1min":0.64900000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.198,"5min":2.379,"15min":2.379},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.92100000000000004,"15min":0.93799999999999994},"min":{"1min":0.75900000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1719999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.095}]},{"osd":4,"last update":"Fri Sep 13 07:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.04,"15min":0.96799999999999997},"min":{"1min":0.56999999999999995,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.296,"5min":12.622,"15min":12.622},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.093,"5min":1.0049999999999999,"15min":1.0069999999999999},"min":{"1min":0.69399999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3500000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.073}]},{"osd":5,"last update":"Fri Sep 13 07:03:03 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0129999999999999,"15min":1.0309999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.427,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.024,"15min":1.038},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3069999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Sep 13 07:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.1739999999999999,"15min":1.113},"min":{"1min":0.66500000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1719999999999999,"5min":12.555999999999999,"15min":12.555999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.0980000000000001,"15min":1.1020000000000001},"min":{"1min":0.65500000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3879999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.89700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33736,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338104,"statfs":{"total":96636764160,"available":96602218496,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.78900000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.024,"5min":1.357,"15min":1.357},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.81399999999999995,"15min":0.86599999999999999},"min":{"1min":0.42599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.2230000000000001,"5min":1.538,"15min":1.538},"last":0.98499999999999999}]},{"osd":1,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78000000000000003,"15min":0.82299999999999995},"min":{"1min":0.53800000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.995,"5min":1.484,"15min":1.484},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.82499999999999996,"15min":0.83999999999999997},"min":{"1min":0.52100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.087,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.26}]},{"osd":2,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77300000000000002,"15min":0.80900000000000005},"min":{"1min":0.52200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0800000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.85599999999999998,"15min":0.91900000000000004},"min":{"1min":0.59199999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96899999999999997,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.89700000000000002}]},{"osd":3,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.91000000000000003,"15min":0.94499999999999995},"min":{"1min":0.68300000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.266,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.92700000000000005,"15min":0.93899999999999995},"min":{"1min":0.49199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.345,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":1.0720000000000001}]},{"osd":4,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.878,"15min":0.89000000000000001},"min":{"1min":0.47099999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.429,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.95099999999999996,"15min":1.0069999999999999},"min":{"1min":0.59099999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.125,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.0409999999999999}]},{"osd":5,"last update":"Fri Sep 13 07:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92700000000000005,"15min":0.95099999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.141,"5min":1.579,"15min":1.579},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0349999999999999,"15min":1.0549999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3420000000000001,"5min":1.857,"15min":1.857},"last":1.111}]},{"osd":7,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.0169999999999999,"15min":1.091},"min":{"1min":0.63500000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.224,"5min":1.607,"15min":1.607},"last":1.2190000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.006,"15min":1.081},"min":{"1min":0.45400000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4570000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.82999999999999996}]}]},{"osd":5,"up_from":201,"seq":863288426508,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35724,"kb_used_data":2628,"kb_used_omap":35,"kb_used_meta":33052,"kb_avail":94336116,"statfs":{"total":96636764160,"available":96600182784,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36852,"internal_metadata":33845260},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69836,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302004,"statfs":{"total":96636764160,"available":96565252096,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73999999999999999,"15min":0.76600000000000001},"min":{"1min":0.33800000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.98999999999999999,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.96799999999999997,"15min":0.95499999999999996},"min":{"1min":0.45100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.1559999999999999,"5min":8.2089999999999996,"15min":8.2089999999999996},"last":0.89000000000000001}]},{"osd":1,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.92600000000000005,"15min":0.81799999999999995},"min":{"1min":0.49099999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1060000000000001,"5min":10.932,"15min":10.932},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84099999999999997,"15min":0.87},"min":{"1min":0.53200000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.272,"5min":2.101,"15min":2.101},"last":1.038}]},{"osd":2,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.76600000000000001,"15min":0.81000000000000005},"min":{"1min":0.42799999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.999,"5min":1.071,"15min":1.071},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.0680000000000001,"15min":0.91600000000000004},"min":{"1min":0.58099999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.282,"5min":12.968999999999999,"15min":12.968999999999999},"last":1.0029999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.875,"15min":0.91100000000000003},"min":{"1min":0.61099999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.26,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.65400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.96799999999999997}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":0.95499999999999996,"15min":0.873},"min":{"1min":0.56899999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":8.2550000000000008,"5min":8.2550000000000008,"15min":8.2550000000000008},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96199999999999997,"15min":0.96999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.198,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.099}]},{"osd":6,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83099999999999996,"15min":0.86199999999999999},"min":{"1min":0.44600000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1839999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.0940000000000001,"15min":1.0289999999999999},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3420000000000001,"5min":10.885,"15min":10.885},"last":1.179}]},{"osd":7,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.93300000000000005,"15min":0.93500000000000005},"min":{"1min":0.57499999999999996,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5589999999999999,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.077},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.92900000000000005,"15min":0.95899999999999996},"min":{"1min":0.51300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.2869999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55880,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315960,"statfs":{"total":96636764160,"available":96579543040,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.621,"15min":0.63300000000000001},"min":{"1min":0.39800000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.80500000000000005,"5min":1.581,"15min":1.581},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.67000000000000004},"min":{"1min":0.40200000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.048,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66300000000000003,"15min":0.71799999999999997},"min":{"1min":0.34999999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.997,"5min":1.577,"15min":1.577},"last":0.503},{"interface":"front","average":{"1min":0.746,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.036,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.68200000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69199999999999995,"15min":0.68500000000000005},"min":{"1min":0.433,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.099,"5min":1.145,"15min":1.145},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.78300000000000003,"15min":0.81799999999999995},"min":{"1min":0.40100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.92400000000000004,"5min":1.375,"15min":1.375},"last":0.62}]},{"osd":4,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.91800000000000004,"15min":0.93300000000000005},"min":{"1min":0.59499999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1539999999999999,"5min":1.488,"15min":1.488},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.94499999999999995,"15min":0.95699999999999996},"min":{"1min":0.68000000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.246,"5min":1.512,"15min":1.512},"last":0.92900000000000005}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3149999999999999,"5min":1.369,"15min":1.369},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4299999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.80300000000000005}]},{"osd":6,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.98599999999999999,"15min":1.014},"min":{"1min":0.71299999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.119,"5min":1.538,"15min":1.538},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.004,"15min":1.02},"min":{"1min":0.73999999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2350000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.98399999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0820000000000001,"15min":1.087},"min":{"1min":0.67800000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.3160000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.093},{"interface":"front","average":{"1min":1.107,"5min":1.036,"15min":1},"min":{"1min":0.79100000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":0.94699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33736,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338104,"statfs":{"total":96636764160,"available":96602218496,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.63800000000000001,"15min":0.64300000000000002},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.80100000000000005,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65500000000000003,"15min":0.63700000000000001},"min":{"1min":0.39700000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.82199999999999995}]},{"osd":1,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.81599999999999995,"15min":0.98399999999999999},"min":{"1min":0.38900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84899999999999998,"5min":6.867,"15min":6.867},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88400000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.69399999999999995,"15min":0.70999999999999996},"min":{"1min":0.51300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.92900000000000005,"5min":1.234,"15min":1.234},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79700000000000004,"15min":0.79100000000000004},"min":{"1min":0.44900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.72999999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91800000000000004,"15min":0.93999999999999995},"min":{"1min":0.64600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.98899999999999999,"15min":1.0029999999999999},"min":{"1min":0.57299999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.667,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.97099999999999997,"15min":0.96899999999999997},"min":{"1min":0.58299999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1,"15min":1.004},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.4670000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.84199999999999997}]},{"osd":6,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.01,"15min":1.008},"min":{"1min":0.70899999999999996,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.611,"5min":2.109,"15min":2.109},"last":1.046},{"interface":"front","average":{"1min":1.004,"5min":1.0529999999999999,"15min":1.071},"min":{"1min":0.70199999999999996,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.714,"5min":1.748,"15min":1.748},"last":0.86099999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0369999999999999,"15min":1.0349999999999999},"min":{"1min":0.80100000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.024,"5min":1.0329999999999999,"15min":1.0369999999999999},"min":{"1min":0.75900000000000001,"5min":0.505,"15min":0.505},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":0.90600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29644,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342196,"statfs":{"total":96636764160,"available":96606408704,"internally_reserved":0,"allocated":2691072,"data_stored":1296855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.58699999999999997},"min":{"1min":0.33100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.034,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66200000000000003,"15min":0.64700000000000002},"min":{"1min":0.33400000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.1240000000000001,"5min":1.77,"15min":1.77},"last":0.62}]},{"osd":2,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60399999999999998,"15min":0.58399999999999996},"min":{"1min":0.45800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81799999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.29399999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0489999999999999,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.70699999999999996}]},{"osd":3,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76400000000000001,"15min":0.73399999999999999},"min":{"1min":0.45200000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1180000000000001,"5min":1.171,"15min":1.171},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.76400000000000001,"15min":0.73699999999999999},"min":{"1min":0.42999999999999999,"5min":0.316,"15min":0.316},"max":{"1min":1.0089999999999999,"5min":2.141,"15min":2.141},"last":0.78000000000000003}]},{"osd":4,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.91500000000000004,"15min":0.90000000000000002},"min":{"1min":0.501,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.599,"5min":1.925,"15min":1.925},"last":0.997},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1719999999999999,"15min":1.0089999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3049999999999999,"5min":13.701000000000001,"15min":13.701000000000001},"last":0.86299999999999999}]},{"osd":5,"last update":"Fri Sep 13 07:03:01 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.006,"15min":1.01},"min":{"1min":0.66400000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3400000000000001,"5min":1.48,"15min":1.48},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.98899999999999999,"15min":0.995},"min":{"1min":0.624,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.409,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.014},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.377,"5min":1.575,"15min":1.575},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0580000000000001,"15min":1.048},"min":{"1min":0.63200000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.544,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.0249999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.647,"15min":1.2869999999999999},"min":{"1min":0.76700000000000002,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.2999999999999998,"5min":59.652000000000001,"15min":59.652000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.5920000000000001,"15min":1.24},"min":{"1min":0.70099999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.7869999999999999,"5min":59.679000000000002,"15min":59.679000000000002},"last":1.1850000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38408,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333432,"statfs":{"total":96636764160,"available":96597434368,"internally_reserved":0,"allocated":3276800,"data_stored":1887185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.7000000000000002,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.34699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.268,"5min":1.738,"15min":1.738},"last":0.96799999999999997}]},{"osd":2,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.71899999999999997,"15min":0.68700000000000006},"min":{"1min":0.30199999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.752,"15min":0.78800000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0069999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.70599999999999996}]},{"osd":3,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.71399999999999997,"15min":0.71299999999999997},"min":{"1min":0.38300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89800000000000002,"5min":1.635,"15min":1.635},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.002,"5min":0.871,"15min":0.84799999999999998},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":3.738,"5min":3.738,"15min":3.738},"last":0.78500000000000003}]},{"osd":4,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.88100000000000001,"15min":0.93000000000000005},"min":{"1min":0.53200000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.077,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.032},{"interface":"front","average":{"1min":1.073,"5min":0.95599999999999996,"15min":0.95699999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":3.6040000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":0.84699999999999998}]},{"osd":5,"last update":"Fri Sep 13 07:03:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.02,"15min":1.034},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1870000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.873,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.254,"5min":1.724,"15min":1.724},"last":0.93700000000000006}]},{"osd":6,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.028,"15min":1.0449999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":4.2359999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":1.022},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.03,"15min":1.0700000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5609999999999999,"5min":2.024,"15min":2.024},"last":1.0640000000000001}]},{"osd":7,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.0429999999999999,"15min":1.081},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.23,"5min":2,"15min":2},"last":1.137},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0409999999999999,"15min":1.1120000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.226,"5min":1.744,"15min":1.744},"last":1.121}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:05:30.539 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:30.539 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T07:01:18.255898+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=1, tm_sec=18, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=7, tm_min=5, tm_sec=4, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T07:05:30.540 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T07:05:50.540 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T07:05:50.846 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-09-13T07:05:50.848 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-09-13T07:05:50.859 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-13T07:05:50.678579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":412,"num_object_clones":0,"num_object_copies":824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":412,"num_whiteouts":404,"num_read":239562,"num_read_kb":1058974,"num_write":24247,"num_write_kb":570384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5266,"num_bytes_recovered":116470768,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21270,"ondisk_log_size":21270,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":375564,"kb_used_data":22156,"kb_used_omap":46,"kb_used_meta":353297,"kb_avail":754599156,"statfs":{"total":773094113280,"available":772709535744,"internally_reserved":0,"allocated":22687744,"data_stored":11560198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48002,"internal_metadata":361776254},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003635"},"pg_stats":[{"pgid":"3.1f","version":"222'428","reported_seq":6844,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:19.439494+0000","last_change":"2024-09-13T07:05:19.439642+0000","last_active":"2024-09-13T07:05:19.439494+0000","last_peered":"2024-09-13T07:05:19.439494+0000","last_clean":"2024-09-13T07:05:19.439494+0000","last_became_active":"2024-09-13T07:04:48.131331+0000","last_became_peered":"2024-09-13T07:04:48.131331+0000","last_unstale":"2024-09-13T07:05:19.439494+0000","last_undegraded":"2024-09-13T07:05:19.439494+0000","last_fullsized":"2024-09-13T07:05:19.439494+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'428","last_scrub_stamp":"2024-09-13T07:05:19.439473+0000","last_deep_scrub":"222'428","last_deep_scrub_stamp":"2024-09-13T07:05:19.439473+0000","last_clean_scrub_stamp":"2024-09-13T07:05:19.439473+0000","objects_scrubbed":13,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:40:58.178088+0000","scrub_duration":0.019357694000000002,"objects_trimmed":30,"snaptrim_duration":0.098858737000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6370,"num_read_kb":30920,"num_write":459,"num_write_kb":15662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":4960629,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"222'394","reported_seq":4145,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:10.464066+0000","last_change":"2024-09-13T07:05:10.464191+0000","last_active":"2024-09-13T07:05:10.464066+0000","last_peered":"2024-09-13T07:05:10.464066+0000","last_clean":"2024-09-13T07:05:10.464066+0000","last_became_active":"2024-09-13T07:04:51.125903+0000","last_became_peered":"2024-09-13T07:04:51.125903+0000","last_unstale":"2024-09-13T07:05:10.464066+0000","last_undegraded":"2024-09-13T07:05:10.464066+0000","last_fullsized":"2024-09-13T07:05:10.464066+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'394","last_scrub_stamp":"2024-09-13T07:05:10.464043+0000","last_deep_scrub":"222'394","last_deep_scrub_stamp":"2024-09-13T07:05:10.464043+0000","last_clean_scrub_stamp":"2024-09-13T07:05:10.464043+0000","objects_scrubbed":15,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:37:33.981404+0000","scrub_duration":0.018767491000000001,"objects_trimmed":2,"snaptrim_duration":0.001432102,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4275,"num_read_kb":38815,"num_write":373,"num_write_kb":7930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"222'506","reported_seq":2609,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:28.809411+0000","last_change":"2024-09-13T07:05:28.809572+0000","last_active":"2024-09-13T07:05:28.809411+0000","last_peered":"2024-09-13T07:05:28.809411+0000","last_clean":"2024-09-13T07:05:28.809411+0000","last_became_active":"2024-09-13T07:04:38.221697+0000","last_became_peered":"2024-09-13T07:04:38.221697+0000","last_unstale":"2024-09-13T07:05:28.809411+0000","last_undegraded":"2024-09-13T07:05:28.809411+0000","last_fullsized":"2024-09-13T07:05:28.809411+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'506","last_scrub_stamp":"2024-09-13T07:05:28.809389+0000","last_deep_scrub":"222'506","last_deep_scrub_stamp":"2024-09-13T07:05:28.809389+0000","last_clean_scrub_stamp":"2024-09-13T07:05:28.809389+0000","objects_scrubbed":17,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:28:35.428304+0000","scrub_duration":0.023179965,"objects_trimmed":28,"snaptrim_duration":0.073210949999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3758,"num_read_kb":13816,"num_write":504,"num_write_kb":5767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"222'514","reported_seq":5618,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:24.889576+0000","last_change":"2024-09-13T07:05:24.889718+0000","last_active":"2024-09-13T07:05:24.889576+0000","last_peered":"2024-09-13T07:05:24.889576+0000","last_clean":"2024-09-13T07:05:24.889576+0000","last_became_active":"2024-09-13T07:04:38.225836+0000","last_became_peered":"2024-09-13T07:04:38.225836+0000","last_unstale":"2024-09-13T07:05:24.889576+0000","last_undegraded":"2024-09-13T07:05:24.889576+0000","last_fullsized":"2024-09-13T07:05:24.889576+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'514","last_scrub_stamp":"2024-09-13T07:05:24.889555+0000","last_deep_scrub":"222'514","last_deep_scrub_stamp":"2024-09-13T07:05:24.889555+0000","last_clean_scrub_stamp":"2024-09-13T07:05:24.889555+0000","objects_scrubbed":14,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:26:35.513305+0000","scrub_duration":0.021224962,"objects_trimmed":34,"snaptrim_duration":0.078226893000000006,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":6179,"num_read_kb":24975,"num_write":551,"num_write_kb":10621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"222'392","reported_seq":6416,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:17.397897+0000","last_change":"2024-09-13T07:05:17.398063+0000","last_active":"2024-09-13T07:05:17.397897+0000","last_peered":"2024-09-13T07:05:17.397897+0000","last_clean":"2024-09-13T07:05:17.397897+0000","last_became_active":"2024-09-13T07:04:48.128577+0000","last_became_peered":"2024-09-13T07:04:48.128577+0000","last_unstale":"2024-09-13T07:05:17.397897+0000","last_undegraded":"2024-09-13T07:05:17.397897+0000","last_fullsized":"2024-09-13T07:05:17.397897+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'392","last_scrub_stamp":"2024-09-13T07:05:17.397875+0000","last_deep_scrub":"222'392","last_deep_scrub_stamp":"2024-09-13T07:05:17.397875+0000","last_clean_scrub_stamp":"2024-09-13T07:05:17.397875+0000","objects_scrubbed":11,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:23:04.467810+0000","scrub_duration":0.017356244,"objects_trimmed":0,"snaptrim_duration":0.0065506189999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6468,"num_read_kb":53020,"num_write":414,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":148740,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"222'1131","reported_seq":8636,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:12.389358+0000","last_change":"2024-09-13T07:05:12.389483+0000","last_active":"2024-09-13T07:05:12.389358+0000","last_peered":"2024-09-13T07:05:12.389358+0000","last_clean":"2024-09-13T07:05:12.389358+0000","last_became_active":"2024-09-13T07:04:50.121731+0000","last_became_peered":"2024-09-13T07:04:50.121731+0000","last_unstale":"2024-09-13T07:05:12.389358+0000","last_undegraded":"2024-09-13T07:05:12.389358+0000","last_fullsized":"2024-09-13T07:05:12.389358+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'1131","last_scrub_stamp":"2024-09-13T07:05:12.389336+0000","last_deep_scrub":"222'1131","last_deep_scrub_stamp":"2024-09-13T07:05:12.389336+0000","last_clean_scrub_stamp":"2024-09-13T07:05:12.389336+0000","objects_scrubbed":12,"log_size":1131,"log_dups_size":0,"ondisk_log_size":1131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:41.586354+0000","scrub_duration":0.019611713999999999,"objects_trimmed":0,"snaptrim_duration":0.002988971,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":11548,"num_read_kb":34808,"num_write":1366,"num_write_kb":17528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":3976247,"num_keys_recovered":80,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"222'639","reported_seq":7719,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:16.950966+0000","last_change":"2024-09-13T07:05:16.951134+0000","last_active":"2024-09-13T07:05:16.950966+0000","last_peered":"2024-09-13T07:05:16.950966+0000","last_clean":"2024-09-13T07:05:16.950966+0000","last_became_active":"2024-09-13T07:04:50.123234+0000","last_became_peered":"2024-09-13T07:04:50.123234+0000","last_unstale":"2024-09-13T07:05:16.950966+0000","last_undegraded":"2024-09-13T07:05:16.950966+0000","last_fullsized":"2024-09-13T07:05:16.950966+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'639","last_scrub_stamp":"2024-09-13T07:05:16.950943+0000","last_deep_scrub":"222'639","last_deep_scrub_stamp":"2024-09-13T07:05:16.950943+0000","last_clean_scrub_stamp":"2024-09-13T07:05:16.950943+0000","objects_scrubbed":11,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:59:06.483229+0000","scrub_duration":0.017176568999999999,"objects_trimmed":0,"snaptrim_duration":0.001498135,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9140,"num_read_kb":20172,"num_write":660,"num_write_kb":17799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":1288675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"222'716","reported_seq":7838,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:18.598082+0000","last_change":"2024-09-13T07:05:18.598296+0000","last_active":"2024-09-13T07:05:18.598082+0000","last_peered":"2024-09-13T07:05:18.598082+0000","last_clean":"2024-09-13T07:05:18.598082+0000","last_became_active":"2024-09-13T07:04:51.126595+0000","last_became_peered":"2024-09-13T07:04:51.126595+0000","last_unstale":"2024-09-13T07:05:18.598082+0000","last_undegraded":"2024-09-13T07:05:18.598082+0000","last_fullsized":"2024-09-13T07:05:18.598082+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'716","last_scrub_stamp":"2024-09-13T07:05:18.598053+0000","last_deep_scrub":"222'716","last_deep_scrub_stamp":"2024-09-13T07:05:18.598053+0000","last_clean_scrub_stamp":"2024-09-13T07:05:18.598053+0000","objects_scrubbed":19,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:53:50.938518+0000","scrub_duration":0.024486260999999999,"objects_trimmed":2,"snaptrim_duration":0.001501437,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8486,"num_read_kb":30532,"num_write":647,"num_write_kb":16391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":13265380,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"222'906","reported_seq":12754,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:11.533393+0000","last_change":"2024-09-13T07:05:11.533393+0000","last_active":"2024-09-13T07:05:11.533393+0000","last_peered":"2024-09-13T07:05:11.533393+0000","last_clean":"2024-09-13T07:05:11.533393+0000","last_became_active":"2024-09-13T07:04:45.140362+0000","last_became_peered":"2024-09-13T07:04:45.140362+0000","last_unstale":"2024-09-13T07:05:11.533393+0000","last_undegraded":"2024-09-13T07:05:11.533393+0000","last_fullsized":"2024-09-13T07:05:11.533393+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'906","last_scrub_stamp":"2024-09-13T07:05:11.533245+0000","last_deep_scrub":"222'906","last_deep_scrub_stamp":"2024-09-13T07:05:11.533245+0000","last_clean_scrub_stamp":"2024-09-13T07:05:11.533245+0000","objects_scrubbed":13,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:41.832464+0000","scrub_duration":0.022800498999999998,"objects_trimmed":34,"snaptrim_duration":0.082577740999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":15171,"num_read_kb":70673,"num_write":1118,"num_write_kb":35612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":8426052,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"222'856","reported_seq":12720,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:31.738513+0000","last_change":"2024-09-13T07:05:31.738802+0000","last_active":"2024-09-13T07:05:31.738513+0000","last_peered":"2024-09-13T07:05:31.738513+0000","last_clean":"2024-09-13T07:05:31.738513+0000","last_became_active":"2024-09-13T07:04:47.126092+0000","last_became_peered":"2024-09-13T07:04:47.126092+0000","last_unstale":"2024-09-13T07:05:31.738513+0000","last_undegraded":"2024-09-13T07:05:31.738513+0000","last_fullsized":"2024-09-13T07:05:31.738513+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'856","last_scrub_stamp":"2024-09-13T07:05:31.738446+0000","last_deep_scrub":"222'856","last_deep_scrub_stamp":"2024-09-13T07:05:31.738446+0000","last_clean_scrub_stamp":"2024-09-13T07:05:31.738446+0000","objects_scrubbed":9,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:37.781982+0000","scrub_duration":0.017385817000000001,"objects_trimmed":0,"snaptrim_duration":0.00075556199999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":14283,"num_read_kb":24697,"num_write":1275,"num_write_kb":8273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1104561,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"222'457","reported_seq":2679,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:18.321703+0000","last_change":"2024-09-13T07:05:18.321841+0000","last_active":"2024-09-13T07:05:18.321703+0000","last_peered":"2024-09-13T07:05:18.321703+0000","last_clean":"2024-09-13T07:05:18.321703+0000","last_became_active":"2024-09-13T07:04:57.321478+0000","last_became_peered":"2024-09-13T07:04:57.321478+0000","last_unstale":"2024-09-13T07:05:18.321703+0000","last_undegraded":"2024-09-13T07:05:18.321703+0000","last_fullsized":"2024-09-13T07:05:18.321703+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'457","last_scrub_stamp":"2024-09-13T07:05:18.321677+0000","last_deep_scrub":"222'457","last_deep_scrub_stamp":"2024-09-13T07:05:18.321677+0000","last_clean_scrub_stamp":"2024-09-13T07:05:18.321677+0000","objects_scrubbed":5,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:40.758192+0000","scrub_duration":0.011351843,"objects_trimmed":0,"snaptrim_duration":0.001895851,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2526,"num_read_kb":26928,"num_write":433,"num_write_kb":12456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"222'537","reported_seq":5776,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:19.299462+0000","last_change":"2024-09-13T07:05:19.299643+0000","last_active":"2024-09-13T07:05:19.299462+0000","last_peered":"2024-09-13T07:05:19.299462+0000","last_clean":"2024-09-13T07:05:19.299462+0000","last_became_active":"2024-09-13T07:04:46.126170+0000","last_became_peered":"2024-09-13T07:04:46.126170+0000","last_unstale":"2024-09-13T07:05:19.299462+0000","last_undegraded":"2024-09-13T07:05:19.299462+0000","last_fullsized":"2024-09-13T07:05:19.299462+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'537","last_scrub_stamp":"2024-09-13T07:05:19.299439+0000","last_deep_scrub":"222'537","last_deep_scrub_stamp":"2024-09-13T07:05:19.299439+0000","last_clean_scrub_stamp":"2024-09-13T07:05:19.299439+0000","objects_scrubbed":21,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:03.051537+0000","scrub_duration":0.026987895000000001,"objects_trimmed":0,"snaptrim_duration":0.0074505329999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":5002,"num_read_kb":31007,"num_write":502,"num_write_kb":34041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":6788059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":246,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:10.285288+0000","last_change":"2024-09-13T07:05:10.285288+0000","last_active":"2024-09-13T07:05:10.285288+0000","last_peered":"2024-09-13T07:05:10.285288+0000","last_clean":"2024-09-13T07:05:10.285288+0000","last_became_active":"2024-09-13T07:04:43.266020+0000","last_became_peered":"2024-09-13T07:04:43.266020+0000","last_unstale":"2024-09-13T07:05:10.285288+0000","last_undegraded":"2024-09-13T07:05:10.285288+0000","last_fullsized":"2024-09-13T07:05:10.285288+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:05:10.285018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:05:10.285018+0000","last_clean_scrub_stamp":"2024-09-13T07:05:10.285018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:23.358286+0000","scrub_duration":0.002927439,"objects_trimmed":0,"snaptrim_duration":0.00095273899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"222'474","reported_seq":3643,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:16.444642+0000","last_change":"2024-09-13T07:05:16.444877+0000","last_active":"2024-09-13T07:05:16.444642+0000","last_peered":"2024-09-13T07:05:16.444642+0000","last_clean":"2024-09-13T07:05:16.444642+0000","last_became_active":"2024-09-13T07:04:49.125105+0000","last_became_peered":"2024-09-13T07:04:49.125105+0000","last_unstale":"2024-09-13T07:05:16.444642+0000","last_undegraded":"2024-09-13T07:05:16.444642+0000","last_fullsized":"2024-09-13T07:05:16.444642+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'474","last_scrub_stamp":"2024-09-13T07:05:16.444589+0000","last_deep_scrub":"222'474","last_deep_scrub_stamp":"2024-09-13T07:05:16.444589+0000","last_clean_scrub_stamp":"2024-09-13T07:05:16.444589+0000","objects_scrubbed":15,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:00:18.701262+0000","scrub_duration":0.021064025,"objects_trimmed":0,"snaptrim_duration":0.00066392000000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2852,"num_read_kb":36914,"num_write":449,"num_write_kb":17878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4764805,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:11.930609+0000","last_change":"2024-09-13T07:05:11.930844+0000","last_active":"2024-09-13T07:05:11.930609+0000","last_peered":"2024-09-13T07:05:11.930609+0000","last_clean":"2024-09-13T07:05:11.930609+0000","last_became_active":"2024-09-13T07:04:46.127497+0000","last_became_peered":"2024-09-13T07:04:46.127497+0000","last_unstale":"2024-09-13T07:05:11.930609+0000","last_undegraded":"2024-09-13T07:05:11.930609+0000","last_fullsized":"2024-09-13T07:05:11.930609+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:05:11.930565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:05:11.930565+0000","last_clean_scrub_stamp":"2024-09-13T07:05:11.930565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:16.038101+0000","scrub_duration":0.0034661929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"222'618","reported_seq":4801,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:09.502658+0000","last_change":"2024-09-13T07:05:09.502785+0000","last_active":"2024-09-13T07:05:09.502658+0000","last_peered":"2024-09-13T07:05:09.502658+0000","last_clean":"2024-09-13T07:05:09.502658+0000","last_became_active":"2024-09-13T07:04:46.126668+0000","last_became_peered":"2024-09-13T07:04:46.126668+0000","last_unstale":"2024-09-13T07:05:09.502658+0000","last_undegraded":"2024-09-13T07:05:09.502658+0000","last_fullsized":"2024-09-13T07:05:09.502658+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'618","last_scrub_stamp":"2024-09-13T07:05:09.502635+0000","last_deep_scrub":"222'618","last_deep_scrub_stamp":"2024-09-13T07:05:09.502635+0000","last_clean_scrub_stamp":"2024-09-13T07:05:09.502635+0000","objects_scrubbed":19,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:10:25.728996+0000","scrub_duration":0.024057370000000002,"objects_trimmed":0,"snaptrim_duration":0.001721534,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5009,"num_read_kb":48072,"num_write":666,"num_write_kb":15004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":128514,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:25.491450+0000","last_change":"2024-09-13T07:05:25.491765+0000","last_active":"2024-09-13T07:05:25.491450+0000","last_peered":"2024-09-13T07:05:25.491450+0000","last_clean":"2024-09-13T07:05:25.491450+0000","last_became_active":"2024-09-13T07:04:57.325955+0000","last_became_peered":"2024-09-13T07:04:57.325955+0000","last_unstale":"2024-09-13T07:05:25.491450+0000","last_undegraded":"2024-09-13T07:05:25.491450+0000","last_fullsized":"2024-09-13T07:05:25.491450+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:05:25.491392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:05:25.491392+0000","last_clean_scrub_stamp":"2024-09-13T07:05:25.491392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:11:49.607496+0000","scrub_duration":0.0037360309999999999,"objects_trimmed":0,"snaptrim_duration":0.00074624700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"222'394","reported_seq":1827,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:15.931694+0000","last_change":"2024-09-13T07:05:15.931865+0000","last_active":"2024-09-13T07:05:15.931694+0000","last_peered":"2024-09-13T07:05:15.931694+0000","last_clean":"2024-09-13T07:05:15.931694+0000","last_became_active":"2024-09-13T07:04:49.132150+0000","last_became_peered":"2024-09-13T07:04:49.132150+0000","last_unstale":"2024-09-13T07:05:15.931694+0000","last_undegraded":"2024-09-13T07:05:15.931694+0000","last_fullsized":"2024-09-13T07:05:15.931694+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'394","last_scrub_stamp":"2024-09-13T07:05:15.931650+0000","last_deep_scrub":"222'394","last_deep_scrub_stamp":"2024-09-13T07:05:15.931650+0000","last_clean_scrub_stamp":"2024-09-13T07:05:15.931650+0000","objects_scrubbed":17,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:40:54.304331+0000","scrub_duration":0.023585517,"objects_trimmed":28,"snaptrim_duration":0.088725134999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1176,"num_read_kb":23362,"num_write":393,"num_write_kb":13842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":2874205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:09.983924+0000","last_change":"2024-09-13T07:05:09.984085+0000","last_active":"2024-09-13T07:05:09.983924+0000","last_peered":"2024-09-13T07:05:09.983924+0000","last_clean":"2024-09-13T07:05:09.983924+0000","last_became_active":"2024-09-13T07:04:46.129069+0000","last_became_peered":"2024-09-13T07:04:46.129069+0000","last_unstale":"2024-09-13T07:05:09.983924+0000","last_undegraded":"2024-09-13T07:05:09.983924+0000","last_fullsized":"2024-09-13T07:05:09.983924+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:05:09.983897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:05:09.983897+0000","last_clean_scrub_stamp":"2024-09-13T07:05:09.983897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:25.703309+0000","scrub_duration":0.0020093709999999998,"objects_trimmed":0,"snaptrim_duration":0.00066388299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"222'720","reported_seq":5298,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:25.873960+0000","last_change":"2024-09-13T07:05:25.874066+0000","last_active":"2024-09-13T07:05:25.873960+0000","last_peered":"2024-09-13T07:05:25.873960+0000","last_clean":"2024-09-13T07:05:25.873960+0000","last_became_active":"2024-09-13T07:04:38.218110+0000","last_became_peered":"2024-09-13T07:04:38.218110+0000","last_unstale":"2024-09-13T07:05:25.873960+0000","last_undegraded":"2024-09-13T07:05:25.873960+0000","last_fullsized":"2024-09-13T07:05:25.873960+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'720","last_scrub_stamp":"2024-09-13T07:05:25.873939+0000","last_deep_scrub":"222'720","last_deep_scrub_stamp":"2024-09-13T07:05:25.873939+0000","last_clean_scrub_stamp":"2024-09-13T07:05:25.873939+0000","objects_scrubbed":12,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:52:50.964394+0000","scrub_duration":0.022412522000000001,"objects_trimmed":40,"snaptrim_duration":0.101155064,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6847,"num_read_kb":24040,"num_write":729,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"222'619","reported_seq":6664,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:10.407478+0000","last_change":"2024-09-13T07:05:10.407478+0000","last_active":"2024-09-13T07:05:10.407478+0000","last_peered":"2024-09-13T07:05:10.407478+0000","last_clean":"2024-09-13T07:05:10.407478+0000","last_became_active":"2024-09-13T07:03:06.744590+0000","last_became_peered":"2024-09-13T07:03:06.744590+0000","last_unstale":"2024-09-13T07:05:10.407478+0000","last_undegraded":"2024-09-13T07:05:10.407478+0000","last_fullsized":"2024-09-13T07:05:10.407478+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'619","last_scrub_stamp":"2024-09-13T07:05:10.407290+0000","last_deep_scrub":"222'619","last_deep_scrub_stamp":"2024-09-13T07:05:10.407290+0000","last_clean_scrub_stamp":"2024-09-13T07:05:10.407290+0000","objects_scrubbed":9,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:08:37.238248+0000","scrub_duration":0.015106461999999999,"objects_trimmed":0,"snaptrim_duration":0.006933257,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":6314,"num_read_kb":32619,"num_write":703,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":11860709,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"222'3","reported_seq":188,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:15.883266+0000","last_change":"2024-09-13T07:05:15.883583+0000","last_active":"2024-09-13T07:05:15.883266+0000","last_peered":"2024-09-13T07:05:15.883266+0000","last_clean":"2024-09-13T07:05:15.883266+0000","last_became_active":"2024-09-13T07:04:46.125291+0000","last_became_peered":"2024-09-13T07:04:46.125291+0000","last_unstale":"2024-09-13T07:05:15.883266+0000","last_undegraded":"2024-09-13T07:05:15.883266+0000","last_fullsized":"2024-09-13T07:05:15.883266+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'3","last_scrub_stamp":"2024-09-13T07:05:15.883217+0000","last_deep_scrub":"222'3","last_deep_scrub_stamp":"2024-09-13T07:05:15.883217+0000","last_clean_scrub_stamp":"2024-09-13T07:05:15.883217+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:04.573276+0000","scrub_duration":0.0066720950000000003,"objects_trimmed":0,"snaptrim_duration":0.001946349,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"222'626","reported_seq":14070,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:20.980133+0000","last_change":"2024-09-13T07:05:20.980246+0000","last_active":"2024-09-13T07:05:20.980133+0000","last_peered":"2024-09-13T07:05:20.980133+0000","last_clean":"2024-09-13T07:05:20.980133+0000","last_became_active":"2024-09-13T07:04:50.122952+0000","last_became_peered":"2024-09-13T07:04:50.122952+0000","last_unstale":"2024-09-13T07:05:20.980133+0000","last_undegraded":"2024-09-13T07:05:20.980133+0000","last_fullsized":"2024-09-13T07:05:20.980133+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'626","last_scrub_stamp":"2024-09-13T07:05:20.980111+0000","last_deep_scrub":"222'626","last_deep_scrub_stamp":"2024-09-13T07:05:20.980111+0000","last_clean_scrub_stamp":"2024-09-13T07:05:20.980111+0000","objects_scrubbed":11,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:21:52.626071+0000","scrub_duration":0.017791412999999999,"objects_trimmed":0,"snaptrim_duration":0.00084358800000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13923,"num_read_kb":43420,"num_write":574,"num_write_kb":22169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":8978985,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":146,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:08.739367+0000","last_change":"2024-09-13T07:05:08.739585+0000","last_active":"2024-09-13T07:05:08.739367+0000","last_peered":"2024-09-13T07:05:08.739367+0000","last_clean":"2024-09-13T07:05:08.739367+0000","last_became_active":"2024-09-13T07:02:44.090715+0000","last_became_peered":"2024-09-13T07:02:44.090715+0000","last_unstale":"2024-09-13T07:05:08.739367+0000","last_undegraded":"2024-09-13T07:05:08.739367+0000","last_fullsized":"2024-09-13T07:05:08.739367+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:05:08.739321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:05:08.739321+0000","last_clean_scrub_stamp":"2024-09-13T07:05:08.739321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:14.140079+0000","scrub_duration":0.0024062810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"222'389","reported_seq":7143,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:10.571648+0000","last_change":"2024-09-13T07:05:10.571648+0000","last_active":"2024-09-13T07:05:10.571648+0000","last_peered":"2024-09-13T07:05:10.571648+0000","last_clean":"2024-09-13T07:05:10.571648+0000","last_became_active":"2024-09-13T07:04:54.129673+0000","last_became_peered":"2024-09-13T07:04:54.129673+0000","last_unstale":"2024-09-13T07:05:10.571648+0000","last_undegraded":"2024-09-13T07:05:10.571648+0000","last_fullsized":"2024-09-13T07:05:10.571648+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'389","last_scrub_stamp":"2024-09-13T07:05:10.571482+0000","last_deep_scrub":"222'389","last_deep_scrub_stamp":"2024-09-13T07:05:10.571482+0000","last_clean_scrub_stamp":"2024-09-13T07:05:10.571482+0000","objects_scrubbed":13,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:36:37.820768+0000","scrub_duration":0.019009884000000001,"objects_trimmed":26,"snaptrim_duration":0.064131435000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6959,"num_read_kb":35900,"num_write":367,"num_write_kb":8191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":3448599,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:11.290601+0000","last_change":"2024-09-13T07:05:11.290831+0000","last_active":"2024-09-13T07:05:11.290601+0000","last_peered":"2024-09-13T07:05:11.290601+0000","last_clean":"2024-09-13T07:05:11.290601+0000","last_became_active":"2024-09-13T07:04:57.320044+0000","last_became_peered":"2024-09-13T07:04:57.320044+0000","last_unstale":"2024-09-13T07:05:11.290601+0000","last_undegraded":"2024-09-13T07:05:11.290601+0000","last_fullsized":"2024-09-13T07:05:11.290601+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T07:05:11.290548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T07:05:11.290548+0000","last_clean_scrub_stamp":"2024-09-13T07:05:11.290548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:30:36.116602+0000","scrub_duration":0.0031320139999999998,"objects_trimmed":0,"snaptrim_duration":0.00054438100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"222'524","reported_seq":4299,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:12.460731+0000","last_change":"2024-09-13T07:05:12.460887+0000","last_active":"2024-09-13T07:05:12.460731+0000","last_peered":"2024-09-13T07:05:12.460731+0000","last_clean":"2024-09-13T07:05:12.460731+0000","last_became_active":"2024-09-13T07:04:51.124840+0000","last_became_peered":"2024-09-13T07:04:51.124840+0000","last_unstale":"2024-09-13T07:05:12.460731+0000","last_undegraded":"2024-09-13T07:05:12.460731+0000","last_fullsized":"2024-09-13T07:05:12.460731+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'524","last_scrub_stamp":"2024-09-13T07:05:12.460702+0000","last_deep_scrub":"222'524","last_deep_scrub_stamp":"2024-09-13T07:05:12.460702+0000","last_clean_scrub_stamp":"2024-09-13T07:05:12.460702+0000","objects_scrubbed":13,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:54:37.903549+0000","scrub_duration":0.018966169000000001,"objects_trimmed":28,"snaptrim_duration":0.071371410999999996,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6314,"num_read_kb":38281,"num_write":477,"num_write_kb":24168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"222'2","reported_seq":208,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:20.899358+0000","last_change":"2024-09-13T07:05:20.899712+0000","last_active":"2024-09-13T07:05:20.899358+0000","last_peered":"2024-09-13T07:05:20.899358+0000","last_clean":"2024-09-13T07:05:20.899358+0000","last_became_active":"2024-09-13T07:04:38.218537+0000","last_became_peered":"2024-09-13T07:04:38.218537+0000","last_unstale":"2024-09-13T07:05:20.899358+0000","last_undegraded":"2024-09-13T07:05:20.899358+0000","last_fullsized":"2024-09-13T07:05:20.899358+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'2","last_scrub_stamp":"2024-09-13T07:05:20.899240+0000","last_deep_scrub":"222'2","last_deep_scrub_stamp":"2024-09-13T07:05:20.899240+0000","last_clean_scrub_stamp":"2024-09-13T07:05:20.899240+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:36.898876+0000","scrub_duration":0.0072699080000000003,"objects_trimmed":0,"snaptrim_duration":0.00059277099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"222'119","reported_seq":357,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:19.310544+0000","last_change":"2024-09-13T07:05:19.310678+0000","last_active":"2024-09-13T07:05:19.310544+0000","last_peered":"2024-09-13T07:05:19.310544+0000","last_clean":"2024-09-13T07:05:19.310544+0000","last_became_active":"2024-09-13T07:04:57.324203+0000","last_became_peered":"2024-09-13T07:04:57.324203+0000","last_unstale":"2024-09-13T07:05:19.310544+0000","last_undegraded":"2024-09-13T07:05:19.310544+0000","last_fullsized":"2024-09-13T07:05:19.310544+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'119","last_scrub_stamp":"2024-09-13T07:05:19.310519+0000","last_deep_scrub":"222'119","last_deep_scrub_stamp":"2024-09-13T07:05:19.310519+0000","last_clean_scrub_stamp":"2024-09-13T07:05:19.310519+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:22:09.450744+0000","scrub_duration":0.0070434039999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"222'797","reported_seq":5579,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:21.629984+0000","last_change":"2024-09-13T07:05:21.630211+0000","last_active":"2024-09-13T07:05:21.629984+0000","last_peered":"2024-09-13T07:05:21.629984+0000","last_clean":"2024-09-13T07:05:21.629984+0000","last_became_active":"2024-09-13T07:04:53.131823+0000","last_became_peered":"2024-09-13T07:04:53.131823+0000","last_unstale":"2024-09-13T07:05:21.629984+0000","last_undegraded":"2024-09-13T07:05:21.629984+0000","last_fullsized":"2024-09-13T07:05:21.629984+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'797","last_scrub_stamp":"2024-09-13T07:05:21.629950+0000","last_deep_scrub":"222'797","last_deep_scrub_stamp":"2024-09-13T07:05:21.629950+0000","last_clean_scrub_stamp":"2024-09-13T07:05:21.629950+0000","objects_scrubbed":15,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:08:14.575207+0000","scrub_duration":0.022307024000000002,"objects_trimmed":0,"snaptrim_duration":0.0032724400000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5361,"num_read_kb":39438,"num_write":976,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":861185,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"222'443","reported_seq":9652,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:13.343789+0000","last_change":"2024-09-13T07:05:13.343789+0000","last_active":"2024-09-13T07:05:13.343789+0000","last_peered":"2024-09-13T07:05:13.343789+0000","last_clean":"2024-09-13T07:05:13.343789+0000","last_became_active":"2024-09-13T07:02:47.511257+0000","last_became_peered":"2024-09-13T07:02:47.511257+0000","last_unstale":"2024-09-13T07:05:13.343789+0000","last_undegraded":"2024-09-13T07:05:13.343789+0000","last_fullsized":"2024-09-13T07:05:13.343789+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'443","last_scrub_stamp":"2024-09-13T07:05:13.343186+0000","last_deep_scrub":"222'443","last_deep_scrub_stamp":"2024-09-13T07:05:13.343186+0000","last_clean_scrub_stamp":"2024-09-13T07:05:13.343186+0000","objects_scrubbed":11,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:56:44.865233+0000","scrub_duration":0.016793586999999999,"objects_trimmed":0,"snaptrim_duration":0.0059111220000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9849,"num_read_kb":28359,"num_write":451,"num_write_kb":30112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5191019,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"222'1007","reported_seq":4573,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:11.267824+0000","last_change":"2024-09-13T07:05:11.267824+0000","last_active":"2024-09-13T07:05:11.267824+0000","last_peered":"2024-09-13T07:05:11.267824+0000","last_clean":"2024-09-13T07:05:11.267824+0000","last_became_active":"2024-09-13T07:01:19.285710+0000","last_became_peered":"2024-09-13T07:01:19.285710+0000","last_unstale":"2024-09-13T07:05:11.267824+0000","last_undegraded":"2024-09-13T07:05:11.267824+0000","last_fullsized":"2024-09-13T07:05:11.267824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'1007","last_scrub_stamp":"2024-09-13T07:05:11.267673+0000","last_deep_scrub":"222'1007","last_deep_scrub_stamp":"2024-09-13T07:05:11.267673+0000","last_clean_scrub_stamp":"2024-09-13T07:05:11.267673+0000","objects_scrubbed":11,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:52:06.385003+0000","scrub_duration":0.018687054000000002,"objects_trimmed":0,"snaptrim_duration":0.0016753040000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":7385,"num_read_kb":27871,"num_write":1272,"num_write_kb":19033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"222'614","reported_seq":7672,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:14.503868+0000","last_change":"2024-09-13T07:05:14.504037+0000","last_active":"2024-09-13T07:05:14.503868+0000","last_peered":"2024-09-13T07:05:14.503868+0000","last_clean":"2024-09-13T07:05:14.503868+0000","last_became_active":"2024-09-13T07:04:51.126975+0000","last_became_peered":"2024-09-13T07:04:51.126975+0000","last_unstale":"2024-09-13T07:05:14.503868+0000","last_undegraded":"2024-09-13T07:05:14.503868+0000","last_fullsized":"2024-09-13T07:05:14.503868+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'614","last_scrub_stamp":"2024-09-13T07:05:14.503847+0000","last_deep_scrub":"222'614","last_deep_scrub_stamp":"2024-09-13T07:05:14.503847+0000","last_clean_scrub_stamp":"2024-09-13T07:05:14.503847+0000","objects_scrubbed":15,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:26.338342+0000","scrub_duration":0.019705391999999999,"objects_trimmed":0,"snaptrim_duration":0.0020303249999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9098,"num_read_kb":32181,"num_write":560,"num_write_kb":27201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":10029829,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"222'1708","reported_seq":13995,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:21.909506+0000","last_change":"2024-09-13T07:05:21.909646+0000","last_active":"2024-09-13T07:05:21.909506+0000","last_peered":"2024-09-13T07:05:21.909506+0000","last_clean":"2024-09-13T07:05:21.909506+0000","last_became_active":"2024-09-13T07:04:41.239873+0000","last_became_peered":"2024-09-13T07:04:41.239873+0000","last_unstale":"2024-09-13T07:05:21.909506+0000","last_undegraded":"2024-09-13T07:05:21.909506+0000","last_fullsized":"2024-09-13T07:05:21.909506+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'1708","last_scrub_stamp":"2024-09-13T07:05:21.909484+0000","last_deep_scrub":"222'1708","last_deep_scrub_stamp":"2024-09-13T07:05:21.909484+0000","last_clean_scrub_stamp":"2024-09-13T07:05:21.909484+0000","objects_scrubbed":8,"log_size":1708,"log_dups_size":0,"ondisk_log_size":1708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:14:22.762681+0000","scrub_duration":0.018664949,"objects_trimmed":30,"snaptrim_duration":0.091557893000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21059,"num_read_kb":55383,"num_write":2606,"num_write_kb":25105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"222'674","reported_seq":6960,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:12.353383+0000","last_change":"2024-09-13T07:05:12.353650+0000","last_active":"2024-09-13T07:05:12.353383+0000","last_peered":"2024-09-13T07:05:12.353383+0000","last_clean":"2024-09-13T07:05:12.353383+0000","last_became_active":"2024-09-13T07:04:57.323373+0000","last_became_peered":"2024-09-13T07:04:57.323373+0000","last_unstale":"2024-09-13T07:05:12.353383+0000","last_undegraded":"2024-09-13T07:05:12.353383+0000","last_fullsized":"2024-09-13T07:05:12.353383+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'674","last_scrub_stamp":"2024-09-13T07:05:12.353308+0000","last_deep_scrub":"222'674","last_deep_scrub_stamp":"2024-09-13T07:05:12.353308+0000","last_clean_scrub_stamp":"2024-09-13T07:05:12.353308+0000","objects_scrubbed":12,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:58.097908+0000","scrub_duration":0.018580025,"objects_trimmed":0,"snaptrim_duration":0.002404799,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8026,"num_read_kb":31428,"num_write":564,"num_write_kb":31570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9245169,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"222'571","reported_seq":3705,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:21.389562+0000","last_change":"2024-09-13T07:05:21.389690+0000","last_active":"2024-09-13T07:05:21.389562+0000","last_peered":"2024-09-13T07:05:21.389562+0000","last_clean":"2024-09-13T07:05:21.389562+0000","last_became_active":"2024-09-13T07:04:57.325924+0000","last_became_peered":"2024-09-13T07:04:57.325924+0000","last_unstale":"2024-09-13T07:05:21.389562+0000","last_undegraded":"2024-09-13T07:05:21.389562+0000","last_fullsized":"2024-09-13T07:05:21.389562+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'571","last_scrub_stamp":"2024-09-13T07:05:21.389538+0000","last_deep_scrub":"222'571","last_deep_scrub_stamp":"2024-09-13T07:05:21.389538+0000","last_clean_scrub_stamp":"2024-09-13T07:05:21.389538+0000","objects_scrubbed":10,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:55.462231+0000","scrub_duration":0.017440310000000001,"objects_trimmed":0,"snaptrim_duration":0.0086238349999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4232,"num_read_kb":25664,"num_write":732,"num_write_kb":10506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":1357199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"222'925","reported_seq":3847,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:22.437352+0000","last_change":"2024-09-13T07:05:22.437529+0000","last_active":"2024-09-13T07:05:22.437352+0000","last_peered":"2024-09-13T07:05:22.437352+0000","last_clean":"2024-09-13T07:05:22.437352+0000","last_became_active":"2024-09-13T07:04:57.326822+0000","last_became_peered":"2024-09-13T07:04:57.326822+0000","last_unstale":"2024-09-13T07:05:22.437352+0000","last_undegraded":"2024-09-13T07:05:22.437352+0000","last_fullsized":"2024-09-13T07:05:22.437352+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'925","last_scrub_stamp":"2024-09-13T07:05:22.437321+0000","last_deep_scrub":"222'925","last_deep_scrub_stamp":"2024-09-13T07:05:22.437321+0000","last_clean_scrub_stamp":"2024-09-13T07:05:22.437321+0000","objects_scrubbed":16,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:40:34.230142+0000","scrub_duration":0.023200728,"objects_trimmed":0,"snaptrim_duration":0.0075911479999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5743,"num_read_kb":24275,"num_write":1279,"num_write_kb":27074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":68557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"222'429","reported_seq":2623,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:12.291803+0000","last_change":"2024-09-13T07:05:12.292273+0000","last_active":"2024-09-13T07:05:12.291803+0000","last_peered":"2024-09-13T07:05:12.291803+0000","last_clean":"2024-09-13T07:05:12.291803+0000","last_became_active":"2024-09-13T07:03:06.743141+0000","last_became_peered":"2024-09-13T07:03:06.743141+0000","last_unstale":"2024-09-13T07:05:12.291803+0000","last_undegraded":"2024-09-13T07:05:12.291803+0000","last_fullsized":"2024-09-13T07:05:12.291803+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'429","last_scrub_stamp":"2024-09-13T07:05:12.291731+0000","last_deep_scrub":"222'429","last_deep_scrub_stamp":"2024-09-13T07:05:12.291731+0000","last_clean_scrub_stamp":"2024-09-13T07:05:12.291731+0000","objects_scrubbed":10,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:24:12.994684+0000","scrub_duration":0.017150891000000001,"objects_trimmed":0,"snaptrim_duration":0.0012751349999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2106,"num_read_kb":12966,"num_write":446,"num_write_kb":14235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":10481695,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"222'603","reported_seq":8356,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:13.354227+0000","last_change":"2024-09-13T07:05:13.354501+0000","last_active":"2024-09-13T07:05:13.354227+0000","last_peered":"2024-09-13T07:05:13.354227+0000","last_clean":"2024-09-13T07:05:13.354227+0000","last_became_active":"2024-09-13T07:04:57.325398+0000","last_became_peered":"2024-09-13T07:04:57.325398+0000","last_unstale":"2024-09-13T07:05:13.354227+0000","last_undegraded":"2024-09-13T07:05:13.354227+0000","last_fullsized":"2024-09-13T07:05:13.354227+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'603","last_scrub_stamp":"2024-09-13T07:05:13.354174+0000","last_deep_scrub":"222'603","last_deep_scrub_stamp":"2024-09-13T07:05:13.354174+0000","last_clean_scrub_stamp":"2024-09-13T07:05:13.354174+0000","objects_scrubbed":11,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:09.354402+0000","scrub_duration":0.017712631,"objects_trimmed":0,"snaptrim_duration":0.0046356670000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8997,"num_read_kb":22165,"num_write":633,"num_write_kb":13453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":102776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"222'936","reported_seq":9139,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:09.556351+0000","last_change":"2024-09-13T07:05:09.556351+0000","last_active":"2024-09-13T07:05:09.556351+0000","last_peered":"2024-09-13T07:05:09.556351+0000","last_clean":"2024-09-13T07:05:09.556351+0000","last_became_active":"2024-09-13T07:04:47.130438+0000","last_became_peered":"2024-09-13T07:04:47.130438+0000","last_unstale":"2024-09-13T07:05:09.556351+0000","last_undegraded":"2024-09-13T07:05:09.556351+0000","last_fullsized":"2024-09-13T07:05:09.556351+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'936","last_scrub_stamp":"2024-09-13T07:05:09.556009+0000","last_deep_scrub":"222'936","last_deep_scrub_stamp":"2024-09-13T07:05:09.556009+0000","last_clean_scrub_stamp":"2024-09-13T07:05:09.556009+0000","objects_scrubbed":9,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:35:06.312941+0000","scrub_duration":0.015400743999999999,"objects_trimmed":0,"snaptrim_duration":0.001076933,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9860,"num_read_kb":45806,"num_write":1253,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":5924852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"222'599","reported_seq":4712,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-13T07:05:15.330673+0000","last_change":"2024-09-13T07:05:15.330898+0000","last_active":"2024-09-13T07:05:15.330673+0000","last_peered":"2024-09-13T07:05:15.330673+0000","last_clean":"2024-09-13T07:05:15.330673+0000","last_became_active":"2024-09-13T07:04:57.325425+0000","last_became_peered":"2024-09-13T07:04:57.325425+0000","last_unstale":"2024-09-13T07:05:15.330673+0000","last_undegraded":"2024-09-13T07:05:15.330673+0000","last_fullsized":"2024-09-13T07:05:15.330673+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"222'599","last_scrub_stamp":"2024-09-13T07:05:15.330642+0000","last_deep_scrub":"222'599","last_deep_scrub_stamp":"2024-09-13T07:05:15.330642+0000","last_clean_scrub_stamp":"2024-09-13T07:05:15.330642+0000","objects_scrubbed":11,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:41.366391+0000","scrub_duration":0.01768716,"objects_trimmed":0,"snaptrim_duration":0.0051571560000000004,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5130,"num_read_kb":30367,"num_write":654,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":12828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":408,"num_object_clones":0,"num_object_copies":816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":408,"num_whiteouts":404,"num_read":239446,"num_read_kb":1058874,"num_write":24086,"num_write_kb":567966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5260,"num_bytes_recovered":115290013,"num_keys_recovered":387,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},"log_size":21146,"ondisk_log_size":21146,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78600,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293240,"statfs":{"total":96636764160,"available":96556277760,"internally_reserved":0,"allocated":3276800,"data_stored":1887770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":77199819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73999999999999999,"15min":0.70999999999999996},"min":{"1min":0.60399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1100000000000001,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.90600000000000003,"15min":0.78000000000000003},"min":{"1min":0.71099999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0720000000000001,"5min":12.583,"15min":12.583},"last":0.69099999999999995}]},{"osd":1,"last update":"Fri Sep 13 07:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77000000000000002,"15min":0.73799999999999999},"min":{"1min":0.53100000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.98699999999999999,"5min":2.016,"15min":2.016},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.97299999999999998,"15min":0.92100000000000004},"min":{"1min":0.56499999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.97999999999999998,"5min":12.481,"15min":12.481},"last":0.93999999999999995}]},{"osd":2,"last update":"Fri Sep 13 07:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.88300000000000001,"15min":0.86899999999999999},"min":{"1min":0.53100000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2130000000000001,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.84699999999999998,"15min":0.86899999999999999},"min":{"1min":0.495,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2330000000000001,"5min":1.593,"15min":1.593},"last":0.72299999999999998}]},{"osd":3,"last update":"Fri Sep 13 07:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91200000000000003},"min":{"1min":0.64900000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.198,"5min":2.379,"15min":2.379},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.92100000000000004,"15min":0.93799999999999994},"min":{"1min":0.75900000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1719999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.04}]},{"osd":4,"last update":"Fri Sep 13 07:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.04,"15min":0.96799999999999997},"min":{"1min":0.56999999999999995,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.296,"5min":12.622,"15min":12.622},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.093,"5min":1.0049999999999999,"15min":1.0069999999999999},"min":{"1min":0.69399999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3500000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.159}]},{"osd":5,"last update":"Fri Sep 13 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.115}]},{"osd":6,"last update":"Fri Sep 13 07:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.1739999999999999,"15min":1.113},"min":{"1min":0.66500000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1719999999999999,"5min":12.555999999999999,"15min":12.555999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.099,"5min":1.0980000000000001,"15min":1.1020000000000001},"min":{"1min":0.65500000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3879999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.0009999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33736,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338104,"statfs":{"total":96636764160,"available":96602218496,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.78900000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.024,"5min":1.357,"15min":1.357},"last":0.627},{"interface":"front","average":{"1min":0.747,"5min":0.81399999999999995,"15min":0.86599999999999999},"min":{"1min":0.42599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.2230000000000001,"5min":1.538,"15min":1.538},"last":0.76500000000000001}]},{"osd":1,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78000000000000003,"15min":0.82299999999999995},"min":{"1min":0.53800000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.995,"5min":1.484,"15min":1.484},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.876,"5min":0.82499999999999996,"15min":0.83999999999999997},"min":{"1min":0.52100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.087,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.68600000000000005}]},{"osd":2,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77300000000000002,"15min":0.80900000000000005},"min":{"1min":0.52200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0800000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.85599999999999998,"15min":0.91900000000000004},"min":{"1min":0.59199999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96899999999999997,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.0389999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.91000000000000003,"15min":0.94499999999999995},"min":{"1min":0.68300000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.266,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.92700000000000005,"15min":0.93899999999999995},"min":{"1min":0.49199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.345,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":0.97799999999999998}]},{"osd":4,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.878,"15min":0.89000000000000001},"min":{"1min":0.47099999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.429,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.95099999999999996,"15min":1.0069999999999999},"min":{"1min":0.59099999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.125,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.95499999999999996}]},{"osd":5,"last update":"Fri Sep 13 07:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":1.0820000000000001}]},{"osd":7,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.0169999999999999,"15min":1.091},"min":{"1min":0.63500000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.224,"5min":1.607,"15min":1.607},"last":1.111},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.006,"15min":1.081},"min":{"1min":0.45400000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4570000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.1359999999999999}]}]},{"osd":5,"up_from":201,"seq":863288426512,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35724,"kb_used_data":2628,"kb_used_omap":35,"kb_used_meta":33052,"kb_avail":94336116,"statfs":{"total":96636764160,"available":96600182784,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36852,"internal_metadata":33845260},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.78300000000000003}]},{"osd":1,"last update":"Fri Sep 13 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.044},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.91800000000000004}]},{"osd":2,"last update":"Fri Sep 13 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.73299999999999998}]},{"osd":3,"last update":"Fri Sep 13 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.85899999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.68300000000000005}]},{"osd":6,"last update":"Fri Sep 13 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Sep 13 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.99399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69836,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302004,"statfs":{"total":96636764160,"available":96565252096,"internally_reserved":0,"allocated":2691072,"data_stored":1297689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73999999999999999,"15min":0.76600000000000001},"min":{"1min":0.33800000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.98999999999999999,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.96799999999999997,"15min":0.95499999999999996},"min":{"1min":0.45100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.1559999999999999,"5min":8.2089999999999996,"15min":8.2089999999999996},"last":0.92900000000000005}]},{"osd":1,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.92600000000000005,"15min":0.81799999999999995},"min":{"1min":0.49099999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1060000000000001,"5min":10.932,"15min":10.932},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84099999999999997,"15min":0.87},"min":{"1min":0.53200000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.272,"5min":2.101,"15min":2.101},"last":1.103}]},{"osd":2,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.76600000000000001,"15min":0.81000000000000005},"min":{"1min":0.42799999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.999,"5min":1.071,"15min":1.071},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.0680000000000001,"15min":0.91600000000000004},"min":{"1min":0.58099999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.282,"5min":12.968999999999999,"15min":12.968999999999999},"last":0.71899999999999997}]},{"osd":3,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.875,"15min":0.91100000000000003},"min":{"1min":0.61099999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.26,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.65400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.194}]},{"osd":5,"last update":"Fri Sep 13 07:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.88900000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83099999999999996,"15min":0.86199999999999999},"min":{"1min":0.44600000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1839999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.0940000000000001,"15min":1.0289999999999999},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3420000000000001,"5min":10.885,"15min":10.885},"last":0.997}]},{"osd":7,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.93300000000000005,"15min":0.93500000000000005},"min":{"1min":0.57499999999999996,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5589999999999999,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.4950000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.92900000000000005,"15min":0.95899999999999996},"min":{"1min":0.51300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.97499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55880,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315960,"statfs":{"total":96636764160,"available":96579543040,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.621,"15min":0.63300000000000001},"min":{"1min":0.39800000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.80500000000000005,"5min":1.581,"15min":1.581},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.67000000000000004},"min":{"1min":0.40200000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.048,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.77400000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66300000000000003,"15min":0.71799999999999997},"min":{"1min":0.34999999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.997,"5min":1.577,"15min":1.577},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.036,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69199999999999995,"15min":0.68500000000000005},"min":{"1min":0.433,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.099,"5min":1.145,"15min":1.145},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.78300000000000003,"15min":0.81799999999999995},"min":{"1min":0.40100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.92400000000000004,"5min":1.375,"15min":1.375},"last":0.74299999999999999}]},{"osd":4,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.91800000000000004,"15min":0.93300000000000005},"min":{"1min":0.59499999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1539999999999999,"5min":1.488,"15min":1.488},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.94499999999999995,"15min":0.95699999999999996},"min":{"1min":0.68000000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.246,"5min":1.512,"15min":1.512},"last":0.96999999999999997}]},{"osd":5,"last update":"Fri Sep 13 07:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.85299999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.98599999999999999,"15min":1.014},"min":{"1min":0.71299999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.119,"5min":1.538,"15min":1.538},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.004,"15min":1.02},"min":{"1min":0.73999999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2350000000000001,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.0389999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:05:10 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0820000000000001,"15min":1.087},"min":{"1min":0.67800000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.3160000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.107,"5min":1.036,"15min":1},"min":{"1min":0.79100000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1}]}]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33736,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338104,"statfs":{"total":96636764160,"available":96602218496,"internally_reserved":0,"allocated":2686976,"data_stored":1297670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.63800000000000001,"15min":0.64300000000000002},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.80100000000000005,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65500000000000003,"15min":0.63700000000000001},"min":{"1min":0.39700000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.65300000000000002}]},{"osd":1,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.81599999999999995,"15min":0.98399999999999999},"min":{"1min":0.38900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84899999999999998,"5min":6.867,"15min":6.867},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88400000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.81599999999999995}]},{"osd":3,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.69399999999999995,"15min":0.70999999999999996},"min":{"1min":0.51300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.92900000000000005,"5min":1.234,"15min":1.234},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79700000000000004,"15min":0.79100000000000004},"min":{"1min":0.44900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.78900000000000003}]},{"osd":4,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91800000000000004,"15min":0.93999999999999995},"min":{"1min":0.64600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.4159999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.98899999999999999,"15min":1.0029999999999999},"min":{"1min":0.57299999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.667,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.874}]},{"osd":5,"last update":"Fri Sep 13 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.486}]},{"osd":6,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.01,"15min":1.008},"min":{"1min":0.70899999999999996,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.611,"5min":2.109,"15min":2.109},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.004,"5min":1.0529999999999999,"15min":1.071},"min":{"1min":0.70199999999999996,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.714,"5min":1.748,"15min":1.748},"last":1.5609999999999999}]},{"osd":7,"last update":"Fri Sep 13 07:05:04 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0369999999999999,"15min":1.0349999999999999},"min":{"1min":0.80100000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.024,"5min":1.0329999999999999,"15min":1.0369999999999999},"min":{"1min":0.75900000000000001,"5min":0.505,"15min":0.505},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.516}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29644,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342196,"statfs":{"total":96636764160,"available":96606408704,"internally_reserved":0,"allocated":2691072,"data_stored":1296855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.58699999999999997},"min":{"1min":0.33100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.034,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.625},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66200000000000003,"15min":0.64700000000000002},"min":{"1min":0.33400000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.1240000000000001,"5min":1.77,"15min":1.77},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60399999999999998,"15min":0.58399999999999996},"min":{"1min":0.45800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81799999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.29399999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0489999999999999,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76400000000000001,"15min":0.73399999999999999},"min":{"1min":0.45200000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1180000000000001,"5min":1.171,"15min":1.171},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.76400000000000001,"15min":0.73699999999999999},"min":{"1min":0.42999999999999999,"5min":0.316,"15min":0.316},"max":{"1min":1.0089999999999999,"5min":2.141,"15min":2.141},"last":0.84099999999999997}]},{"osd":4,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.91500000000000004,"15min":0.90000000000000002},"min":{"1min":0.501,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.599,"5min":1.925,"15min":1.925},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1719999999999999,"15min":1.0089999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3049999999999999,"5min":13.701000000000001,"15min":13.701000000000001},"last":1.1950000000000001}]},{"osd":5,"last update":"Fri Sep 13 07:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.2430000000000001}]},{"osd":6,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.014},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.377,"5min":1.575,"15min":1.575},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0580000000000001,"15min":1.048},"min":{"1min":0.63200000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.544,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.0820000000000001}]},{"osd":7,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.647,"15min":1.2869999999999999},"min":{"1min":0.76700000000000002,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.2999999999999998,"5min":59.652000000000001,"15min":59.652000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.5920000000000001,"15min":1.24},"min":{"1min":0.70099999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.7869999999999999,"5min":59.679000000000002,"15min":59.679000000000002},"last":1.266}]}]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38408,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333432,"statfs":{"total":96636764160,"available":96597434368,"internally_reserved":0,"allocated":3276800,"data_stored":1887185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.7000000000000002,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":0.62},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.34699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.268,"5min":1.738,"15min":1.738},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.71899999999999997,"15min":0.68700000000000006},"min":{"1min":0.30199999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.752,"15min":0.78800000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0069999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.0529999999999999}]},{"osd":3,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.71399999999999997,"15min":0.71299999999999997},"min":{"1min":0.38300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89800000000000002,"5min":1.635,"15min":1.635},"last":0.49399999999999999},{"interface":"front","average":{"1min":1.002,"5min":0.871,"15min":0.84799999999999998},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":3.738,"5min":3.738,"15min":3.738},"last":0.69699999999999995}]},{"osd":4,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.88100000000000001,"15min":0.93000000000000005},"min":{"1min":0.53200000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.077,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.073,"5min":0.95599999999999996,"15min":0.95699999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":3.6040000000000001,"5min":3.6040000000000001,"15min":3.6040000000000001},"last":0.94399999999999995}]},{"osd":5,"last update":"Fri Sep 13 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":3.6480000000000001,"5min":3.6480000000000001,"15min":3.6480000000000001},"last":0.752},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.85199999999999998}]},{"osd":6,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.028,"15min":1.0449999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":4.2359999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.03,"15min":1.0700000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5609999999999999,"5min":2.024,"15min":2.024},"last":0.97799999999999998}]},{"osd":7,"last update":"Fri Sep 13 07:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.0429999999999999,"15min":1.081},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.23,"5min":2,"15min":2},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0409999999999999,"15min":1.1120000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.226,"5min":1.744,"15min":1.744},"last":1.1180000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1175,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11571,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T07:05:50.863 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T07:05:51.182 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T07:05:51.182 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T07:05:51.182 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T07:05:51.182 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:05:57.285 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T07:05:57.285 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T07:05:57.285 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:05:57.378 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T07:05:57.379 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T07:05:57.379 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:05:57.462 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T07:05:57.462 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T07:05:57.462 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:05:58.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-13T07:06:03.563 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T07:06:03.564 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T07:06:03.564 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:06:03.662 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T07:06:03.663 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T07:06:03.663 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:06:03.753 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T07:06:03.754 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T07:06:03.754 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:06:04.376 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-13T07:06:04.376 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-13T07:06:09.855 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T07:06:09.856 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T07:06:09.856 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:06:09.947 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T07:06:09.947 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T07:06:09.948 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T07:06:09.948 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:06:09.976 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T07:06:09.976 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T07:06:09.977 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:06:09.982 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-13T07:06:09.982 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-13T07:06:10.009 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T07:06:10.009 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T07:06:10.009 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T07:06:10.010 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:06:10.050 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T07:06:10.050 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T07:06:10.050 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:06:10.091 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T07:06:10.091 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T07:06:10.091 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T07:06:10.136 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T07:06:10.136 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T07:06:10.136 DEBUG:teuthology.orchestra.run.smithi118:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-13T07:06:10.157 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-13T07:06:10.158 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T07:06:10.631 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-13T07:06:10.631 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T07:06:10.833 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-13T07:06:10.833 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T07:06:11.026 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi118.front.sepia.ceph.com 2024-09-13T07:06:11.026 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T07:06:11.188 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-13T07:06:11.189 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T07:06:11.526 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-13T07:06:11.526 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T07:06:11.718 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-13T07:06:11.718 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T07:06:11.901 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-13T07:06:11.901 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T07:06:12.080 INFO:tasks.ceph:Archiving mon data... 2024-09-13T07:06:12.081 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900895/data/mon.a.tgz 2024-09-13T07:06:12.081 DEBUG:teuthology.orchestra.run.smithi118:> mktemp 2024-09-13T07:06:12.089 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.BHUSxLpTRw 2024-09-13T07:06:12.089 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.BHUSxLpTRw 2024-09-13T07:06:12.252 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.BHUSxLpTRw 2024-09-13T07:06:12.273 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.BHUSxLpTRw is 475KB 2024-09-13T07:06:12.335 DEBUG:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.BHUSxLpTRw 2024-09-13T07:06:12.344 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900895/data/mon.c.tgz 2024-09-13T07:06:12.345 DEBUG:teuthology.orchestra.run.smithi118:> mktemp 2024-09-13T07:06:12.395 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.kJ3zQyKO0V 2024-09-13T07:06:12.395 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.kJ3zQyKO0V 2024-09-13T07:06:12.550 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.kJ3zQyKO0V 2024-09-13T07:06:12.574 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.kJ3zQyKO0V is 634KB 2024-09-13T07:06:12.638 DEBUG:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.kJ3zQyKO0V 2024-09-13T07:06:12.645 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900895/data/mon.b.tgz 2024-09-13T07:06:12.645 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-09-13T07:06:12.652 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.NJLohU6oKL 2024-09-13T07:06:12.652 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.NJLohU6oKL 2024-09-13T07:06:12.807 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.NJLohU6oKL 2024-09-13T07:06:12.829 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.NJLohU6oKL is 634KB 2024-09-13T07:06:12.900 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.NJLohU6oKL 2024-09-13T07:06:12.907 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T07:06:12.907 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T07:06:12.909 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T07:06:12.924 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T07:06:12.924 DEBUG:teuthology.orchestra.run.smithi118:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T07:06:12.926 DEBUG:teuthology.orchestra.run.smithi164:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T07:06:12.954 INFO:teuthology.orchestra.run.smithi118.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T07:06:13.003 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T07:06:13.009 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T07:06:13.011 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900895/remote/smithi118/crash 2024-09-13T07:06:13.012 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T07:06:13.029 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900895/remote/smithi164/crash 2024-09-13T07:06:13.030 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T07:06:13.046 INFO:tasks.ceph:Compressing logs... 2024-09-13T07:06:13.046 DEBUG:teuthology.orchestra.run.smithi118:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T07:06:13.049 DEBUG:teuthology.orchestra.run.smithi164:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T07:06:14.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-13T07:06:30.182 INFO:tasks.ceph:Archiving logs... 2024-09-13T07:06:30.183 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900895/remote/smithi118/log 2024-09-13T07:06:30.184 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T07:06:31.314 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900895/remote/smithi164/log 2024-09-13T07:06:31.315 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T07:06:33.107 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T07:06:33.117 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T07:06:33.117 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T07:06:33.119 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T07:06:33.191 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T07:06:33.200 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T07:06:33.200 DEBUG:teuthology.orchestra.run.smithi118:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T07:06:33.202 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T07:06:33.224 INFO:teuthology.orchestra.run.smithi164.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T07:06:33.224 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================== 2024-09-13T07:06:33.224 INFO:teuthology.orchestra.run.smithi164.stdout:*hv01.front.sepi 67.205.162.81 3 u 39 64 377 0.068 -1.484 1.605 2024-09-13T07:06:33.224 INFO:teuthology.orchestra.run.smithi164.stdout:+hv02.front.sepi 96.245.170.99 3 u 39 64 377 0.064 1.538 2.054 2024-09-13T07:06:33.224 INFO:teuthology.orchestra.run.smithi164.stdout:+hv03.front.sepi 74.6.168.72 3 u 34 64 377 0.079 -1.752 2.070 2024-09-13T07:06:33.224 INFO:teuthology.orchestra.run.smithi164.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T07:06:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T07:06:33.226 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================== 2024-09-13T07:06:33.226 INFO:teuthology.orchestra.run.smithi118.stdout:+hv01.front.sepi 67.205.162.81 3 u 36 64 377 0.090 -1.593 0.940 2024-09-13T07:06:33.226 INFO:teuthology.orchestra.run.smithi118.stdout:+hv02.front.sepi 96.245.170.99 3 u 42 64 377 0.103 3.513 1.146 2024-09-13T07:06:33.226 INFO:teuthology.orchestra.run.smithi118.stdout:*hv03.front.sepi 74.6.168.72 3 u 42 64 377 0.087 0.057 0.966 2024-09-13T07:06:33.226 INFO:teuthology.orchestra.run.smithi118.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T07:06:33.226 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T07:06:33.234 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T07:06:33.235 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T07:06:33.243 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T07:06:33.251 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T07:06:33.259 INFO:teuthology.task.internal:Duration was 843.381056 seconds 2024-09-13T07:06:33.259 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T07:06:33.267 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T07:06:33.267 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T07:06:33.269 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T07:06:33.323 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T07:06:33.324 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi118.front.sepia.ceph.com 2024-09-13T07:06:33.324 DEBUG:teuthology.orchestra.run.smithi118:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T07:06:33.337 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-09-13T07:06:33.337 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T07:06:33.350 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T07:06:33.351 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T07:06:33.353 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T07:06:33.392 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T07:06:33.392 DEBUG:teuthology.orchestra.run.smithi118:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T07:06:33.437 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T07:06:33.526 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T07:06:33.536 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T07:06:33.536 DEBUG:teuthology.orchestra.run.smithi118:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T07:06:33.547 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T07:06:33.563 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T07:06:33.572 DEBUG:teuthology.orchestra.run.smithi118:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T07:06:33.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T07:06:33.587 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = core 2024-09-13T07:06:33.620 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-09-13T07:06:33.640 DEBUG:teuthology.orchestra.run.smithi118:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T07:06:33.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:06:33.662 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T07:06:33.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T07:06:33.668 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T07:06:33.678 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T07:06:33.679 DEBUG:teuthology.misc:Transferring archived files from smithi118:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900895/remote/smithi118 2024-09-13T07:06:33.679 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T07:06:33.712 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900895/remote/smithi164 2024-09-13T07:06:33.713 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T07:06:33.738 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T07:06:33.738 DEBUG:teuthology.orchestra.run.smithi118:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T07:06:33.750 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T07:06:33.784 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T07:06:33.794 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T07:06:33.795 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T07:06:33.803 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T07:06:33.803 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T07:06:33.805 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T07:06:33.811 INFO:teuthology.orchestra.run.smithi118.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 07:06 /home/ubuntu/cephtest 2024-09-13T07:06:33.813 INFO:teuthology.orchestra.run.smithi164.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 07:06 /home/ubuntu/cephtest 2024-09-13T07:06:33.815 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T07:06:33.825 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T07:06:33.846 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} duration: 843.3810560703278 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-13T07:06:33.846 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T07:06:33.927 INFO:teuthology.run:pass